
Inside Amazon's clickworker platform - jsrn
http://www.techrepublic.com/article/inside-amazons-clickworker-platform-how-half-a-million-people-are-training-ai-for-pennies-per-task/
======
drieddust
Amazon is setting a wrong precedence which others are following[1]. When Bezos
says "Your Margin is my opportunity.", he fails to mention who will be the
loser. They have started a race to bottom which means employees/workers will
be squeezed until their bones are dry.

I think this can be fixed simply by measuring companies on employment they
generate and median tax payed per employee apart from the corporate tax. If
they fall behind on these metrics, their products should be taxed heavily. If
companies don't agree, they lose access to the market which will allow for
local responsible companies to emerge and compete.

I am just wondering if my thinking is simplistic or this is a problem our
leadership does not want to solve this issue?

[1]
[http://www.livemint.com/Sundayapp/bAgZIMlCQu8weSTF7j750L/Sig...](http://www.livemint.com/Sundayapp/bAgZIMlCQu8weSTF7j750L/Sign-
here-please-the-life-of-a-delivery-boy.html)

edit: grammer

~~~
bgirard
It's an interesting idea. In practice I bet it would push business for more
automation and to for splitting into a highly profitable R&D entity with
highly payed workers and 'independent' contracting firms that are barely
profitable under which low wage employees would work under.

~~~
drieddust
I really want to put more thoughts into this so your input is appreciated.

Not to nitpick but what you described is happening today. If companies are
measured on all of the metric then I how do they escape. If they move their
work to contracting firms, won't they also be evaluated on the same metrics.

------
Sanddancer
My mother had a job like this 30 years ago. She worked data entry as a
keypunch operator. However, it was a full time job, above minimum wage,
provided insurance and vacation time. Mechanical Turk relies on the race to
the bottom, the fact that some people need any job to keep from being on the
street. Of course companies love the gig economy, it keeps them from having to
think of their workers as people.

~~~
Erik816
Alternative interpretation: 30 years ago this kind of job could't easily be
done by someone in a third world country for a fraction of the price. Now that
it can, the price has adjusted.

~~~
PostOnce
Out of sight, out of mind. It's not like those people have the same standard
of living but just with lower living costs.

You know a WHOLE LOT of people would rather make 500k a year than 300k even if
it means their employees are damn near homeless and malnourished. See: any biz
with min wage employees in the US.

Plus you're not bound by the same labor laws if you hire from overseas, only
public outrage if the public ever finds out.

edit: that is to say, instead of thinking of it as "the price has adjusted",
we can think of it as "greed is no longer constrained by the difficulty of
overseas communication". Our laws can't contain greed, if we can just export
it to where we have no jurisdiction.

~~~
wjossey
> You know a WHOLE LOT of people would rather make 500k a year than 300k even
> if it means their employees are damn near homeless and malnourished. See:
> any biz with min wage employees in the US.

Nah, sorry. That's just over the top and not true. I worked two separate
minimum wage jobs before I was 20, and both had owner / operators who made no
where near that kind of money. One was an entrepreneur who was losing money
every year, but believed he had a good product that could grow with franchises
(and it did). The other ended up having his collection of businesses go under
due to Netflix (they were video stores).

Minimum wage does not imply improperly priced labor. While it is not a living
wage in much of the United States, to assume it is purely exploitive is also
an overdramatized statement.

I'm not exceptionally well versed in minimum wage theory, so my opinions are
based on listening to numerous Econtalk podcasts on the subject
([https://goo.gl/IbKS1F](https://goo.gl/IbKS1F)). Some argue for an
elimination of the minimum wage, others for a vast expansion at the federal
level, others for a broader expansion at the state level.

My personal observation is that the minimum wage discussion is interesting,
but mostly just the tip of the ice berg and serves as more of a distraction
around more meaningful conversations. 1\. Minimum wage accounts for less than
2% of the hourly workforce in the United States
([https://goo.gl/Q9S2Mf](https://goo.gl/Q9S2Mf)). Looking at it on a state by
state basis, that number increases; however, let's focus on the federal wage
for now. 2\. There are more than seven times more people in the United States
(14M+) who are either unemployed (seeking but can't find work) or
underemployed (have work, but are seeking more) ::
[https://data.bls.gov/timeseries/LNS13327709](https://data.bls.gov/timeseries/LNS13327709)
3\. There are over 60M Americans who work hourly jobs in the United States,
and 4.6M will have to find new work each year due to regular turnover in the
market (I don't have my citations handy for this one, have it in a document
that is missing the citation, but I'm confident on the figure).

For a worker working 32 hours at the 8.25 minimum wage, they can expect to
make $264 / week. Give them a 10% raise, and they're at $9.075, or $290.40 /
week. Or, alternatively, let's give that worker better access to full
employment, and they reach 40 hours, but only make $8.25 / hour, or $330 /
week.

Given an average number of weeks of 52.1429 per year our example workers can
expect to make:

Fully Employed Minimum Wage: $17,207.15 Under Employed Higher Wage: $15,142.29

In this very simple example (with some assumptions that are admittedly
contrived), the fully employed minimum wage worker is in fact, better off.
They are still impoverished, they still have a tough life and will not be able
to have many luxuries; however, full employment, in this example, is more
optimal when purely looking at salary.

Full employment, mixed with increasing wages, and decreasing prices of goods
and services, is the combination we should all be striving for.

~~~
PostOnce
"Minimum wage does not imply improperly priced labor" even if it's not a
living wage? If it's not a living wage, it's not proper, is it? What are you
saying?

Have you read anything about the McDonalds and other franchises paying their
employees in cards which carry transaction and withdrawal fees ON TOP OF the
fact they're already making min wage?

Even if you make $10/hr which isn't min wage, and you're fully employed,
you're still probably not breaking even with rent/food/car/insurance/medical,
you'll be sharing a duplex with some other miserable bastard or living at
home, or getting a subsidy from the government because apparently it's someone
other than your employers job to see that you can afford to pay your rent!

Do you know that 40% of workers in America earn under $20K/yr? If your rent is
$900/mo, then over half your income (haha we didn't even consider taxes yet)
goes to rent, and you still haven't payed for power or food or anything.

You keep telling yourself it's fine not to pay people a fair wage, or that we
should be struggling to lower prices or to employ people for more hours --
whatever you have to do to dance around the fact that we should be paying
people a fair wage.

Every job I ever had before I started serious software as a freelancer paid
fuck all ($10/hr or less, after 2000) regardless of how much the owner made.
Many jobs, many different industries, same story -- and that's not anecdotal,
this is happening to the entire country, see again the stat of how many
Americans earn under 20K.

It's not like it's impossible to pay a living wage. I could go on and on with
statistics about profit margins and wealth concentrating and everything -- we
as a society are choosing to damn ourselves and our neighbors JUST IN CASE we
ever happen to get rich ourselves, or we are already rich. We're greedy.

~~~
pas
Do people on min. wage have to pay taxes? And how much?

Also, what is the total income of those who work min. wage? (So what are the
other sources of income, or the monetary equivalent of regular support they
receive?)

It's also fine to pay just a security guard and rent a machine that can make
burgers and pay the franchise mothership delivery service a tiny bit more to
not just unload the cargo but to load it into the burgermachine.

Yes, we ought to pay people a fair wage. People should look after people a lot
better than we do today.

And please do, show us that the majority of those making under 20K/year work
somewhere where the company is making huge profits _and_ that the work they do
is AI-hard.

And of course we're greedy, just look how many people are still clinging to
concepts such as nation, fate, hard work and so on.

~~~
orly_bookz
I believe someone making minimum wage will get all of their Federal taxes
back, so they could probably just claim 99 exemptions and never pay that money
in the first place. I'm not sure about state taxes as that may well vary from
place to place.

------
zaroth
75% of Turkers are Americans. And I didn't see it in the OP but I would guess
that similarly 75% of companies issuing HITs are American.

Now I don't know the exact wording of minimum wage law, but I'm pretty sure if
you hire an independent contractor through a 3rd party staffing company, let's
say to clean houses. And then you set a fixed price on the service, say you
will pay $20 for each house cleaned, knowing it takes 4 hours on average to
clean a house. And then you let anyone who wants it take the job. This would
be illegal.

So, I honestly don't know how mTurk is allowed to operate the way they do.
American companies should not be able to hire American workers to perform
labor at below minimum wage, just because they use an API to do it.

I'm willing to bet if someone was willing to spell it all out in front of the
right AG or the right class action legal team that there would be significant
settlements to be had.

~~~
mtw
From what I understand, most of the american turkers are stay-at-home moms.
They have a family to cater to and various obligations that prevent them to
pursue a full-time job. Mechanical Turk gives them the possibility to work
anytime, at their own rhytm. 10mn there, 2 hours, early am, middle of the
night. I'm not sure if there are jobs that can offer this flexibility.

~~~
zaroth
I don't believe that working in fits and spurts is a written exception to the
minimum wage law.

To be covered by the Federal minimum wage, one must be a "covered nonexempt"
employee. There are two ways in which an employee can be covered by the law:
"enterprise coverage" and "individual coverage." [1]

    
    
      Enterprise Coverage
    
      Employees who work for certain businesses or organizations (or "enterprises") are
      covered by the FLSA. These enterprises, which must have at least two employees, are:
    
        (1) those that have an annual dollar volume of sales or business done of at least
            $500,000
        (2) hospitals, businesses providing medical or nursing care for residents, schools and
            preschools, and government agencies
    
      Individual Coverage
    
      Even when there is no enterprise coverage, employees are protected by the FLSA if their
      work regularly involves them in commerce between States ("interstate commerce"). The FLSA
      covers individual workers who are "engaged in commerce or in the production of goods for
      commerce."
    
      Examples of employees who are involved in interstate commerce include those who: produce
      goods (such as a worker assembling components in a factory or a secretary typing letters
      in an office) that will be sent out of state, regularly make telephone calls to persons
      located in other States, handle records of interstate transactions, travel to other States
      on their jobs, and do janitorial work in buildings where goods are produced for shipment
      outside the State.
    

What I think that means is either Turkers are covered by FLSA if the HIT comes
from a business which has more than $500k of annual sales volume.

Or Turkers are covered under FLSA because their work involves commerce between
states. Because their work itself is literally commerce between states on its
face -- the writen work product is sent over the Internet -- I don't see how
you weasel out of that one.

I don't see how Turkers could be exempt from FLSA, but IANAL.

[1] -
[https://www.dol.gov/whd/regs/compliance/whdfs14.htm](https://www.dol.gov/whd/regs/compliance/whdfs14.htm)

------
siliconc0w
Amazon could bundle Turk with it's other AI efforts to automatically train AIs
alongside Turk problems to eventually take over the task or only farm it out
when the model's confidence is outside a particular threshold.

~~~
twblalock
A lot of companies do that already, for machine learning dataset training.

------
tie_
It's a global platform for cheap human labor. I find it completely
unsurprising that living off cheap labor alone would not result in a
particularly good living standard, especially not in some of the most
expensive locations in the world, and especially not when competing with cheap
labor from across the globe.

~~~
BoorishBears
I'm always torn when I see stories like this. On one hand I feel for someone
making less than minimum wage doing a job, but on the other I realize that
these people are competing with people who come from places without a minimum
wage at all.

The same thing happens on freelancer sites. Jobs for pennies on the dollar of
their worth are fought over by people barely making it in the US, and people
who are enjoying a rather livable stream of income in other countries.

To me the answer is to tell yourself that it's not work but a hobby that
happens to pay, and at least seek out work that pays minimum wage in your
area, if that applies.

But easier said than done I'm sure.

~~~
Fnoord
> but on the other I realize that these people are competing with people who
> come from places without a minimum wage at all.

"Places", or "countries" might have low minimum wage compared to US or even
below the average of world. But no minimum wage at all seems unlikely.

See
[https://en.wikipedia.org/wiki/List_of_minimum_wages_by_count...](https://en.wikipedia.org/wiki/List_of_minimum_wages_by_country)

I wonder if there is a 18+ check involved.

> To me the answer is to tell yourself that it's not work but a hobby that
> happens to pay, and at least seek out work that pays minimum wage in your
> area, if that applies.

But it isn't a hobby; it is paid; hence work.

From Wikipedia: "A Hobby is a regular activity that is done for enjoyment,
typically during one's leisure time."

The type of labour on Amazon Turk is extremely repetitive just like (physical)
hard labour. The goal also isn't pleasure, and the income isn't a gift; it is
part of one's income (I wonder if it is being tracked though).

I also find the name Amazon Turk offensive. I am aware that the word Turk has
multiple meanings, but in a good part of the 90s in Europe cheap labour and
Turkish people went hand in hand. It was the kind of labour the native
Europeans didn't want to do (e.g. garbagemen), it paid very badly.

Now we are at the next step of globalisation and because the competition goes
via the Internet the lowest common denominator becomes standard. Which is very
bad for those who are unemployed, underskilled, homeless etc in higher
civilized countries where the standard of living is higher, but so is things
like rent. Heck, I wouldn't even be able to pay my rent with Amazon Turk.

> The same thing happens on freelancer sites. Jobs for pennies on the dollar
> of their worth are fought over by people barely making it in the US, and
> people who are enjoying a rather livable stream of income in other
> countries.

Yes, and fake freelance like Uber and MLM.

~~~
BoorishBears
I'm (oddly) glad someone else finds the name offensive. I know the history
from the chess playing machine, but I found it felt more strongly tied to the
meaning behind that in Turkish people. I just worried I was being overly
sensitive and no one else would see it that way.

But I'd also say having a stated minimum wage, and having a relevant minimum
wage are two different things. I come from a country on that list with a
stated minimum wage of 1.70$ _a day_ or about 22 cents an hour.

Not only is that incredibly low, I assure you no one is touring villages
looking for hawkers with young helpers that aren't getting paid 1.79 a day,
when the hawkers themselves can't ensure they're getting 1.79 every single
day.

Ironically, it looks like mechanical Turk would actually meet the minimum wage
in the country. And I'm sure the country is not alone there. So for someone
who has no better source of income, I'm sure even after paying for an Internet
cafe they'd come out making income that's livable.

I don't think that it's a bad thing that those wages aren't livable in a
country where our _hourly_ minimum wage exceeds the _daily_ minimum wage.

That's why I'd say _treat_ it like a hobby if you're in a country that's like
the US.

I'm definitely not calling the work described as fun, but treat it as a hobby
in the sense of treating it as something you do because you want to make a few
dollars from the comfort of your computer (and for some reason don't want to
try something that pays better in the long run like learning a skill, say...
programming), not because you have to pay rent next month (again, situation
permitting).

You have to remember, even in the US at minimum wage people still struggle
with rent, so even if mechanical Turk did pay minimum wage per country, it'd
still wouldn't guarantee financial security in many parts of the US

~~~
Fnoord
Thank you for your post and clarifications. They make a lot of sense to me.

I made a mistake in my post:

> in a good part of the 90s in Europe cheap labour and Turkish people went
> hand in hand

I meant to say the 20th century, not the 90s. It occurred from about '60s till
'80s. I'm also glad I'm not the only one who finds the name offensive, and
FWIW I'm not Turkish in any way. Nor do I resent being associated with Turkish
people. However for it to change, two people wouldn't be enough. I also wonder
if the name is a coincidence, or deliberate. If Amazon were a European company
I'd expect the latter, but America doesn't have the same history with Turkey
as Europe does.

As for hobby. I'd just call it what it is: "a badly paid job". One you'd
replace ASAP if you could. You need to also take into account that one
requires a computer which requires maintenance and is going to (partly) die at
some point. Sure, internet bill is likely flat fee and paid already, but
electricity is going to be higher.

------
kozikow
In my company I have my own image labeling team in Poland and I pay $4 per
hour. I can go down to $2.5. Median hourly rate of amazon mechanical turk is
supposedly $1.38[1]. My dataset is harder to label than ordinary images, so it
takes a few hours to ramp up the labeller. I have been contemplating migrating
to mechanical turk for a while. Anyone have some experience to share?

1\.
[http://journals.sagepub.com/doi/full/10.1177/216770261246901...](http://journals.sagepub.com/doi/full/10.1177/2167702612469015)

~~~
IanCal
Is it a major cost to your business? Would it be a large risk if things went
badly?

Also, does your current setup have any problems (perhaps scaling)?

If the answers are no, yes, no then I'd not change a thing.

Otherwise perhaps look at using Turk for labeling and your working crew as
reviewers. It may only take a small number of badly labeled images to cost
more than you save.

------
blazespin
It'll be interesting to watch AI start edging out these jobs. What an
interesting general AI project. Instead of a Turk, there will be a machine
inside..

~~~
avaid1996
Don't know if this would really happen much, unless we are saturated with
labelled datasets. They are used for labelling and not directly responsible
for the actual end classficiation which is what AI is aiming to solve. In
fact, AI needs labelling to work on these classficiations

~~~
visarga
classfic[i]ations

/Just a human doing a spellbot's job.

~~~
sharmi
class[i]fic[i]ations

/Just another human doing a human's job doing a spellbot's job

Makes you wonder about the quality of the job done on MechanicalTurk.

Redoing supposedly done work is my worst nightmare.

~~~
visarga
Haha this is priceless. :-) Thanks

------
novaleaf
A bit of anecdotal insight: I have a relative who is psychologically incapable
of finding a job due to fear of rejection, and will justify not applying for
any job by considering how hard it would be to get a job. Quite literally he
finds it more attractive to get a guaranteed minimum wage job than to apply
and face possible rejection for a high paid job which he is qualified for.

I believe this same kind of pathological behavior is what's causing people
(like described in the article) to stick with AMT jobs.

I wish I knew a way to break this vicious cycle :(

------
WhitneyLand
What a dystopian vibe. They were debating how long this type of work would be
available, but didn't emphasize that ultimately it's a zero sum game.

Is anyone predicting when various jobs will fall to AI? I here about the 3
million truck drivers a lot. However I think some groups like radiologists are
still underestimating the speed at which reading x-rays will become mostly a
job of double checking computer interpretations.

------
JackFr
If I close my eyes and imagine a world where Mechanical Turk doesn't exist,
it's not clear to me how the people in the article are better off.

~~~
dx034
Just look away from the US. Turk is not available in Europe (although other
systems are).

------
eva1984
Shit job or no job. That is a question. And probably one should not ask Amazon
for answer.

------
soufron
I would not call it "being paid".

------
johnnabil
I am confused, i checked most of the HITs and most of them are paying less
than 1$ , is that true? will i work for 1 hour for 0.04$?

