On: Duplicates appearing in HN top?

[dbul]

As you look into it, it would be pretty cool to know the steps you took.

[pg → dbul]

0. View frontpage with horror. Remind self not to panic.

1. Back out latest release, which contained a bunch of low-level changes, and restart the server.

2. Use the server repl to analyze what's happening. Notice that the problem is duplication in the list of top stories, because its len is 180 and len:dedup of it is 157. Fix the immediate problem with

    (zap dedup ranked-stories*)

Now there are only 157 top stories, but no one will notice.

2. The list of top stories is kept in order by reinserting things into it when they get upvoted. Hypothesis: this code is not recognizing duplicates.

3. Here's the operator that does that:

    (def reinsert-sorted (test elt seq)
      (if (no seq)
           (list elt)
          (is elt (car seq))
           (reinsert-sorted test elt (cdr seq))
          (test elt (car seq))
           (cons elt (rem elt seq))          
           (cons (car seq) (reinsert-sorted test elt (cdr seq)))))

4. The expression where it should get rid of duplicates is

    (rem elt seq)

5. Why is that not working? Because I just stupidly expanded the Arc testify operator to treat tables as tests, like functions. What used to be

    (def testify (x)
      (if (isa x 'fn) x [is _ x]))

had become

    (def testify (x)
      (if (in (type x) 'fn 'table) x [is _ x]))

I thought I'd rarely want to find or remove a particular table from a list. I'd be much more likely to want to use one like a fn. Doh. I forgot all the items in News are tables.

6. Revert to old definition of testify, test on localhost, commit code, restart server again.

7. Be really thankful that the problem was not some other much more frightening stuff we changed today, like the definition of atomic.

[brlewis → pg]

Test suites are your friend.

[pg → brlewis]

You are my test suite, friend.

[nopassrecover → pg]

Best response ever.

[dbul → pg]

This is great. Even though it wasn't just for me, I offer my gratitude: thank you.

[nopassrecover → pg]

Really quick response + detailed solution - great work!

[jmonegro → dbul]

Not a bad idea. I would also like to know how this happened.