
TheSixtyOne.com's Jobs Page 'Voting Rings' Problem - rams
http://www.thesixtyone.com/#/info/settings/jobs/
======
patio11
Ooh, that sounds fun.

In terms of process design, I think you could take a hint or two from the
Netflix guys here. Currently you're distributing a single test case. That
takes very little work to accomplish: good first step! Consider creating two
test cases: one we'll call the probe case, and one we'll call the real case.
The real case is kept on your servers and never gets exposed directly to the
participants.

Make a really freaking simple way for you to check your work on the probe case
automatically, without needing to speak to you guys. For example, you could
make a simple REST API or even a web page with an upload/submit button. Then
run whatever your evaluation function is against the submitted solution for
the probe case, and tell folks how good their submission was. For an extra ten
minutes of work you can add a leaderboard. (Add a leaderboard!) You can
optionally rate-limit this if you're worried about people overfitting to the
probe via taking a lot of measurements.

This has a better user experience and will get you more viral exposure because
a) you've now totally decoupled your ability to run code from people's ability
to participate in the contest and b) people can now get immediate feedback
that their participation is valued and producing results. Including a
leaderboard also gives folks an emotional hook both for participating and for
spreading word of your contest: "Oh cripes guys, the top submission for this
problem was done in Ruby. Let's show them the awesome power of functional
programming. For the greater glory of Lisp!"

Of course, you can feel free on putting whatever restrictions you want on code
which you individually invite for offline exposure to the real test case.

~~~
threepointone
Now you have 2 problems :)

------
carterschonwald
Isn't the problem format a teeny bit underspecified, as in shouldn't the input
format also indicate who submitted the article?

~~~
chronomex
It could be, or depending on how the problem is specified, it could be
entirely superfluous. If the accounts that submit articles vote according to
the general population's general vote distribution, then knowing the submitter
of the article does not help finding a voting ring.

------
JoelPM
Did anyone write a solution? I wrote one for fun (not looking for a job) and
am interested in comparing findings. Here's the biggest ring I found (assuming
there isn't some huge flaw in my code):

114317 117417 116711 104623 55965 61543 117666 111454 106702 109719 59735
98844 79232 99084 56944 42023 116674 113113 106336 106877 114731 107692 106107
116790 116420 98958 111863 87465 102051 59239 36249 61496 108406 93936 117489
102314 114890 40853 43144 112933 61454 93895 99996 107278 100940 106030 98559
28959 36931 8629 65535 61589 113214 108605 100944 99853

------
Tichy
Is that job still open? I toyed with the idea of applying, but then my gf was
a bit shocked by the idea of moving to the states. Still, it sounds like fun.

I started coding the exercise, my first approach would have been a clustering
algorithm. For various reasons I decided to write my own (though with a
standard algorithm) and didn't have enough spare time to debug it properly.
Should have just used an existing package... Anyway, if the job is still open,
I might give it another shot.

------
buro9
That's the best job application I've seen in a while and something that I've
no doubt interests many of us. I think I'll take a stab at their interview
problem just for my own amusement. Shame I won't be moving to California
otherwise I'd consider going for the job.

------
dangrossman
If I were looking for work I'd have some fun with this. Machine learning is a
fun area.

~~~
DanielBMarkham
Would you do it that way?

I'd just brute force it, assuming the entries would support it.

Of course, I always like using a shotgun where a fly-swatter might work :)

~~~
jules
What do you mean by brute force? How would you even determine whether a
solution is good?

------
DanielBMarkham
This does sound like a neat problem -- and one which I'll be solving soon in
my application.

Don't want to give away too much, but looks like a basic statistics text might
be a good place to start....

~~~
Perceval
So we'd have to set up the null hypothesis that the voting patterns are simply
random. What probability distribution are we going to use? Gaussian
distribution is probably the wrong one to use, since online voting and online
popularity usually conforms better to a power law distribution. So bootstrap a
distribution from the data, and then test for correlations? How do we control
for the deliberately random behavior of the voting circle sockpuppets?

~~~
rms
A Bayesian approach probably works much better

------
rwhitman
Wouldn't this be a problem that was better solved without software?

I mean if its actual people doing the voting, rather than just bots, shouldn't
this be handled in a less software-focused way?

What I'd say is treat it like police detective work, have someone go
undercover, act as if it was like any other human 'crime' syndicate...

~~~
Perceval
1) There is the problem of identification, which seems to be the object of
this exercise. TheSixtyOne wants to have a program that can red flag certain
correlated behaviors that they might not otherwise notice or be able to
quantify. You can't really go undercover if you can't spot the criminals in
the first place.

2) There is the question of labor. I doubt TheSixtyOne has a large enough
staff to devote one or more employees to the task of infiltration.
Infiltration is not the easiest skill to just pick up. Nor are voting rings
actually made up of multiple people. As the test prompt states, it could just
be one person with multiple sockpuppets. How do you infiltrate a one-person
voting ring?

~~~
dasil003
_How do you infiltrate a one-person voting ring?_

This should go on a list of startup koans somewhere.

------
trafficlight
James took the simple way out with this one. He just took down all of the
voting and commenting features. This drives away all of his users and now he
doesn't have to worry about voting rings.

That's a bingo!

