

Mechanical Turk: Now with 40.92% Spam. - Panos
http://behind-the-enemy-lines.blogspot.com/2010/12/mechanical-turk-now-with-4092-spam.html

======
andrewljohnson
At first, I thought MTurk was good for getting translation work done, and
checked by others. It worked OK, but I had to be clever to avoid the cheaters
who would ignore instructions not to use machine translation. After refining
my tactics, talking to a lot of workers, and fending off the crooks, I got a
few decent translations (for my iPhone app).

Then I tried a new tactic. I hired some pros from various translation
websites. All were more expensive (money-wise) than MT, but the results were
better, and I spent a lot less time. Some were even close to MTurk prices.

So, I don't use MTurk anymore. Maybe if I had a more mindless task I'd try
again, but it is a lot of hassle.

I know this isn't exactly what the author means by "spam" - but even the
workers on that site are often dishonest. It's a total black market.

~~~
trotsky
It seems to be a failed social experiment to me, there is definitely a
negative feedback loop there. Bad HITS, low wages, withheld payments, bad
workers, sweatshops. Hard to imagine it will get any better.

I wonder if you reversed the process and had projects bid on workers if it
could counter the race to the bottom effect. Include a system where work could
be divided into common types and include worker metrics and feedback that
captured any specialization. Perhaps allow some workers to fill a managerial
role, where they acquire large blocks and have a trusted group they farm out
work to. Presumably wages would go up but bad work would go down, perhaps
offsetting it somewhat.

~~~
byoung2
It sounds like you have a great idea for a startup to bring some competition
to this area.

------
gcv
Fascinating.

My experience with MTurk suggests that Amazon has mostly abandoned the
product. Amazon still uses it for product classification, but does not
participate in the community forums (even on the official AWS MTurk developer
forum). No new features have appeared in several years, and enhancement
requests have fallen on deaf ears.

That said, MTurk still works. The API still does what its documentation
claims. People still perform tasks. Payment still works.

~~~
nmcfarl
I have to say that although it feels that way, lately they've been sort of
active.

They've been using twitter since October at: <http://twitter.com/amazonmturk>

And they _just_ launched launched a blog at:
<http://mechanicalturk.typepad.com/>

Largely I attribute this to Natala Menezes, their newish Product Manager.

~~~
natala
Nathan, thank you for the kind words -- but I really can't take any credit.
Just know we are actively listening and we take all the feedback, even the
controversial bits, very seriously.

~~~
khafra
The obvious question: do you have a computer program set to alert you if
"natala" appears on a tech site in a mturk context, or a mechanical turk task
set with a 10 cent payment to email you whenever you're mentioned?

~~~
natala
;-) No. I found out the old fashioned way -- a friend saw the post and sent it
to me. He reads YCombinator news much more frequently than me! It does help to
have a unique name...

------
staunch
It's unbelievable that no one has swooped in to crush MTurk given how little
Amazon has done to develop it since 2005(!)

~~~
aresant
I think this is one of those examples where nobody's rushing in because no
matter how you slice it, the product is just not that useful.

I've used it a few times and because the eco system is truly about hiring
dirt, dirt cheap labor you kind of get what you pay for.

Beyond that the revenue model sucks - the concept of running millions of
micro-transactions and having a little piece of all the action is compelling,
but again because the product use is so limited it's difficult to make
signifigant revenue.

~~~
iwwr
Last I checked, MTurk still accepted only US credit cards. This puts a dent in
the potential size of the ecosystem.

If the purpose is dirt-cheap labor, they should open up the door to markets
where $cents matter.

~~~
thetable
I'm pretty sure I recently used a non-US credit card to pay for MTurk. They
require you live in the US, but they don't seem to verify that.

~~~
iwwr
They verify for valid zip codes and they prune false addresses from time to
time. You have to lie to Amazon, or know someone in the US.

------
ig1
I think mturk is vastly underdeveloped, there are all sorts of things people
could be using it for, but don't simply because they don't know about it or
it's too complex.

Before I started my current startup, creating an mturk competitor was
something I considered, but the fundamental problem is handling the payments.
Most payment processors wouldn't touch you due to the risk of fraud without a
huge (>1mm) deposit.

Paying individuals small amount of money is hard, you pretty much have to use
paypal, and if you're lucky enough not to have your account suspended they'll
completely rip you on the charges. And you'll still carry all the risk of
people trying to use your system for money laundering.

