

Designing Incentives for Crowdsourcing Workers - wslh
http://blog.crowdflower.com/2011/05/designing-incentives-for-crowdsourcing-workers/

======
forcer
So this is the winning task description incentive:

Bayesian Truth Serum or BTS (ﬁnancial) “For the following ﬁve questions, we
will also ask you to predict the responses of other workers who complete this
task. There is no incentive to misreport what you truly believe to be your
answers as well as others’ answers. You will have a higher probability of
winning a lottery (bonus payment) if you submit answers that are more
surprisingly common than collectively predicted.” 8

I would be interested on how to interpret this :)

------
todayiamme
There's a selection bias in operation over here.

The real question is what type of workers do they actually attract? I am
desperate and out looking for a job and for me all of these freelancing sites
are a last resort.

Why?

First of all, there are arbitrary restrictions placed upon the workers (and
I'm guessing they're present for the companies as well) limiting in some cases
not just the amount of bids, but how I can bid, how much I can bid, what I can
write and what I can't, my profile picture and what not. A lot of it is
understandable as fraud detection, but what really gets to me is the
environment.

The problem with bidding in such an environment is that workers are chosen on
the basis of cost. Yes, they put stars next to each worker's name and
recommendations from previous employers are there, but do you really think
that anyone looks through all of that?

No.

I think that there's a heavy selection bias on the side of the employers as
well. People who emphasise on quality wouldn't go to such outsourcing sites
for critical things like their code or their copy they'll either ask around in
their network. Or, if push comes to shove they'd rather do such things
themselves than risk botching it up.

So, the users of this site are people who're trying to save a buck while
getting and aiming for average quality, at best. Hence, there is a focus of
bidding on cost instead of quality.

Further, since we bid on the basis of cost people as the number of workers
increases the average bid decreases. This means that I will have to squeeze in
more work to make the same amount of money. This means that the quality of my
performance per job will automatically decrease leading to greater pain all
around.

So, what happens is an avalanche. The best workers, who are trying to improve
their craft, will intend to compete on the basis of quality. Whereas the
implicit selection pressure will be on the basis of cost. So, either they
reduce the quality per job (and increase the number of jobs) or they bide
their time and open the doors to greener pastures and thus leave. As the site
will grow this will happen again and again until you reach the lowest common
denominator and that's their problem.

Plus, add to the fact that my margins will be heavily dictated by them I've
safely labelled this as a last resort.

