

Mechanical Turk changes how we understand labor - shalmanese
http://blog.bumblebeelabs.com/mechanical-turk-changes-how-we-understand-labor

======
TomOfTTB
I love Mechanical Turk. We used it at my agency where we were having a big
problem with time clock fraud (a.k.a. people were asking their friends to
punch them out and then leaving hours early)

So I built a very simple add on to our timeclock (which is essentially an ELO
touch screen and a PC) and had it use a web cam to take a picture of each
person punching. Then we created a program that showed turk users a person's
badge picture and then the picture taken at the punch and asked "Is this the
same person?" (with answers of Yes/No/Maybe) Doing that virtually eliminated
the problem overnight.

I agree with the author that M.T. won't completely change the market. But what
it will do is allow people to monitize their spare seconds and in doing so
make a lot of the "no-brainer" grunt work that companies have to do a lot
cheaper

~~~
derefr
You just inspired me for a CAPTCHA replacement. Simply asking questions can't
work because we either have to have a preformulated DB of questions and
answers (which could be cracked easily) or have to generate the question from
a simple enough formula that a computer could derive the answer. How about,
instead, we generate CAPTCHA questions on-demand _from_ humans? Or, even
better, don't bother with an _automated_ Turing Test at all: just pay people
to have a short conversation with the possible bot, and then say whether they
think they're human or not.

~~~
shalmanese
Except MTurk and other similar services make the entire concept of CAPTCHA's
obsolete with the ability to pay people to solve any human distinguishing
task.

~~~
derefr
Actially, I believe this becomes a variation on the AI-Box experiment: ask the
tester to not just distinguish the humanity of the challenger, but also to
determine whether the challenger is a user with an honest goal to use the
website, or an MT-employed opponent. The spammer can then only win if they can
convince the tester to "let them out."

Interestingly, with both the challenger and tester being average humans that
are badly- or un-trained on being on either side of a social engineering
attack, the effectiveness of this technique will depend totally on the
statistical ratio of cunning to cynicism in the average human being.

------
dxjones
The Calorie Tracking example (in the blog posting) illustrates that Mechanical
Turk relies heavily on tremendous inequities in wealth.

A rich, fat, lazy person snaps a picture of his lavish meal with a fancy
iPhone, and rather than take a brief moment to ponder the calories, … uploads
it for, not just one, but _three_ other people to ponder and estimate the
calories and report back. The most dissimilar answer is rejected; that person
is not compensated.

This only works if there is a very large pool of very poor people who are
willing to do these mundane tasks for (fractions of) pennies. These “workers”
certainly can’t afford an iPhone; they probably can’t even afford the meal
they are looking at.

~~~
shalmanese
There’s a common perception that Mechanical Turk thrives on exploiting 3rd
world labor. This doesn’t appear to be the case.

From: [http://behind-the-enemy-
lines.blogspot.com/2008/03/mechanica...](http://behind-the-enemy-
lines.blogspot.com/2008/03/mechanical-turk-demographics.html) and [http://asc-
parc.blogspot.com/2008/07/mechanical-turk-demogra...](http://asc-
parc.blogspot.com/2008/07/mechanical-turk-demographics.html) 82% of turkers
are from the US, Canada or the UK and over 75% have a Bachelor’s degree or
higher.

~~~
rms
Yes, the average wage is $4/hour and most people spend it as Amazon credit on
plasma TVs and home theaters.

------
sergeo
As a developer of a calorie tracking service, also with an iPhone app in the
App Store, I would point out that the example with calorie tracking does not
work in RL.

1\. There are hundreds of food chains in the US, with widely varying recipes,
serving sizes, and nutritional contents. Even being familiar with the subject
area, I generally won't be able to recognize a food. Even if I see a burger
photo - how would I know the brand, flavor, whether it has cheese in it, etc.
The error rate will be huge, rendering the service useless, and even worse -
misleading.

2\. You cannot estimate portion size on a photo, not having anything to
compare with. This also results in significant errors.

I am not arguing against other uses of MT, just pointing out that this is a
not very well thought through example.

MT is applicable only for very rudimentary tasks, requiring absolutely zero
qualifications and training. There are fewer such tasks around than it looks
at the first glance, as this example demonstrates.

~~~
tectonic
What is your calorie tracking app?

~~~
sergeo
www.mynetdiary.com , also see in the App Store

------
ars
This story makes me really sad. You know why? Because it means that humans are
now so common and cheap that we can use them for menial labor.

People are now cheaper than machines :(

This is a reversal of 2000 years of progress.

~~~
paulgb
Mechanical Turk is generally used for things that machines still can't do with
nearly the accuracy that humans can. Image recognition, facial recognition,
translation, dictation, etc.

No matter how much hardware you throw at those problems, you can't (yet) match
what a human brain can do in a split second. I don't think that's depressing.

