Hacker News new | past | comments | ask | show | jobs | submit login
Who Makes Below Minimum Wage in the Mechanical Turk Sweatshop? (priceonomics.com)
80 points by narad on July 5, 2013 | hide | past | favorite | 91 comments



Mechanical Turk and its ilk are very important. One of the biggest barriers to achievements in machine learning and optimization is a lack of processed and labelled data. A lot of statistical inference is about algorithms and formulae, but in most cases the key issue is having enough data. It doesn't matter how fancy your technology is if you don't have lots and lots of clearly labelled samples.

Mechanical Turk (and crowdflower and a few other services) are important bridges for this. People can do important labeling and cleaning work for a cost that's affordable enough to be practical. Right now most of the work is translation, address finding or quasi spammy tasks. But the service itself has a lot of potential to do things that impact our lives in the future. Mechanical Turk workers can improve startups by helping with personalization and automation tasks, help with science and defense by labeling images and aid in all kinds of language processing and computer vision tasks that could make things cheaper and better for all of us.

It's good to see human capital being utilized, even if it is ont he cheap. For what it's worth, amazon does recommend that people set the pricing on their mturk tasks so that they come out to something close to minimum wage. Though I'm not sure that things like this are what the minimum wage was designed for. Completing online labeling tasks is a far cry from working in an actual sweat shop. It would be nice if there was some sort of exception for technical piece work. Mechanical Turk isn't nearly popular enough right now, but in the future as machine learning becomes more common there will be a serious demand for these tasks. I hope we can find a way to let people do data preparation work while avoiding virtual dickensian conditions of a kind of data processing industrial revolution.


If this type of work is important, then why does it not provide minimum wage? For what reason should Mechanical Turk be an exception?


Minimum wage is a distortion of pricing for human work - somewhat analogous to rent price control. Although you could argue it helps some people, it hurts others such as people who aren't currently working. They may be prepared to work for $6/hr but are prevented from doing so. Similarly an employer may have work that brings in value of $6/hr but are prevented from hiring people to do so. Or maybe they have work that brings in $10/hr and can't hire two people for it.

This is a completely separate discussion than what people need to get by, or what they should be provided. It is far more straightforward to just provide/pay them directly from the government, than pretend that isn't happening and there is no cost by distorting markets and making customers pay anyway, as well as reducing competition.


It is quite hard to think of any jobs in western economies that are only intended to bring in $6/hr revenue. Automation increasingly means that people are used as quality checkers primarily who also do some kind of manual work. You could use a robot but you would still need a pair of eyes to check things. Certain tasks can't be automated but that tends to be in higher value products. You could argue that minimum wage forces companies to be more efficient and focus on high profit activities.


There is an employment market - we don't need to think of the jobs. There will be good old fashioned supply and demand as people and companies find what meets their needs.

Seeing exactly what companies do in reaction to not being able to hire based on supply and demand is quite difficult to measure. For example it can hit the poorer people because companies can't economically provide products for them, hence causing the poorer people to have to spend more on alternatives. Sometimes surveys find that minimum wage increases made no difference, but that then turns out to be survivorship bias - you can't survey companies that went out of business or left to go elsewhere.

The real problem is that market distortions are used to sweep under the rug a social policy, and to pretend that it doesn't cost people anything when it certainly does, and the poor/unemployed the most. If in favour of the social policy then just do it directly.


Legally speaking, the workers are paid as contractors rather than employees. They set their own hours and are not under any obligation to accept work they don't want to do. As such, they don't have to pay payroll taxes and are not subject to laws such as minimum wage.


I am pretty sure contractors have to pay payroll taxes. It is one of the reasons that companies are increasingly classifying employees as contractors, the burden of the payroll tax falls on the contractor entirely (normally it is split 50/50 between employee/employer).


That doesn't make much sense--payroll tax is roughly 15% either way, so with employees you just pay them 7.5% less than a contractor for the same benefit.

The real reason for contracting is that the paperwork is much simpler. You don't need an HR department. And the guy gets automatically laid off after a few months without the incident reflecting on either you or him, so there's less risk.

There's another side to it where you can hire a "contractor" on an n-month term, but instead of paying the contractor directly you go through a middleman who hires the contractor as an employee and provides (shitty) benefits and takes care of taxes.


Or you pay them the same, and skip out on benefits as well as payroll taxes and your ROI is much higher.


you can not "split" taxes between employer and employee. The employer cares only about how much the total cost of employing someone is. The makeup of the cost is irrelevant.


The point is that employers are classifying employees as contractors because they can get away with offering the same 'salary', but avoid paying payroll taxes and benefits. We can get pedantic about 'who is really paying what', but in the end, companies are using this tactic to screw over their employees. There are many cases where taking work as a contractor makes sense (mainly, short-term/part-time work), but people need to be aware of the costs, and take them into consideration when comparing job offers.


> not under any obligation to accept work they don't want to do

The fact they are choosing to do it shows they have no viable alternative, which means things must be pretty bad where they are living


From the article:

> A survey of workers on Mechanical Turk done by Panos Ipeirotis, an Associate Professor at the Stern School of Business at NYU, found that the majority of American workers are young, have at least some college experience, and have household incomes between $25,000 and $60,000.

It sounds like the majority of workers are supplementing their incomes, not relying on it.


Only a small percentage of workers are using MTurk as a primary source of income, so I wouldn't agree with the assertion that "things must be pretty bad".


You think that people working for less than minimum wage is not a problem?


Did you even read the article?

> The rest make under $20 a week by working an hour or two per day. The most commonly cited reasons for using Mechanical Turk are to earn some extra spending money and productively use time that would otherwise be wasted watching television. One academic study that relied on Mechanical Turk workers filling out questionnaires noted that its American participants were “normally distributed in terms of socioeconomic status, with the average participant having completed some college and receiving a financial income between $37,500–49,999 per year.” This does not match up with the perception of exploited sweatshop labor - the poor who must accept low wages due to a lack of alternatives.

I guess it ought to be illegal for people to choose to make productive use of their time.

Next time, please read the article before parroting your ideological commitments.


The bigger point you need to understand is that if minimum wage was enforced at MTurk, it would significantly raise the prices, and many requesters would simply stop using the service. This would result in less work for everyone.


By that logic, no jobs should have a minimum wage, lest it drive up the price of having employees and then there would be less work for everyone.


That's right. It's the common argument against minimum wage, which a lot of people argue.


Given a choice between fewer jobs in which employees are actually able to support themselves and more where they can't, the former seems the more humane option...


How do you plan to enforce a global minimum wage where employers cannot simply shift labor?


I don't have a good answer for that, except to expect that wages wherever the jobs are outsourced to are at a livable standard for that area. Competition and outsourcing are probably inevitable in a global marketplace, but everyone should at least have access to food, clean water, education and decent healthcare.

Of course i'm coming entirely from a labor point of view, and i'll grant my grasp of economics is probably naive to say the least.


If the end goal was not profit, this would not be a problem.

If the end goal was to produce some goods or services that are of benefit to people, while looking after employees, the environment and the local community, the world would be a much better place.

Of course, the stock market might not grow so fast, but that's just made up anyway. People struggling to buy food on paltry wages is not made up.


>If the end goal was not profit, this would not be a problem.

I suggest you try your experiment on sentient creatures that are not motivated by utility. I am unaware of any at this point, however.


Humans managed about 200,000 years of doing exactly that.


I find the past 200,000 years of "management" unacceptable and largely void of progress. Only once sufficient incentive was established in what we now know as a capitalistic global market did we see humans as a species begin to solve existential problems at a race previously reserved for science fiction and fantasy.


Yes.


Once again, I'm very happy I don't live in your country.


Whose minimum wage?


>The fact they are choosing to do it shows they have no viable alternative

Assertion based on facts not found in evidence. Just because they take remote jobs that can be done at their whim whenever they want doesn't mean they can't find a viable alternative. In fact, there aren't many jobs where those qualifications are met.


I think this misses the point of minimum wage. The whole point of minimum wage is that the work is not very important and very commodity-like. Minimum wage protects the importance of people, not any particular type of work.


While I have not used it, I have thought about using it for things that require speed in mass. That is tasks that can be broken into small little chunks. Which if done by a few would take days/weeks/months. But having a virtual workforce of thousands it can get done in minutes/hours/days.

That is why this is slightly different. It is the micro payment for micro work.


How would you regulate it? The jobs aren't specified by the time it takes to do them, they're specified by the task that needs to be accomplished. This is unlike a typical minimum wage job, like in retail, where a cashier just needs to be present.


Minimum wage where? Don't assume that minimum wage in San Jose has anything to do with a living wage in Bangladesh. That's the whole point of this labor arbitrage.


It's good to see human capital being utilized, even if it is ont he cheap.

Chillingly spoken like a true, cold-blooded functionary. Are humans mere "capital"? Is their worth decided by being "utilized"? Is that what the meaning of our existence is reduced to? Perhaps our robotic overlords will agree that such a determination fits each and all of us. Tarpaper shacks, dirty water, food scraps and, every now and then, a glance at the Sun when the heat and clouds of pollution part for a short moment.

Either that or our future could be much less deterministic and free-form through the elimination of such de-liberate prognoses. One way or another that viewpoint of humanity is doomed to extinction. I'll keep my fork packed, friend.


> For what it's worth, amazon does recommend that people set the pricing on their mturk tasks so that they come out to something close to minimum wage

What is the incentive to do an online job and set your prices at the lowest possible legal limit? Surely nearly anything else you could do with a computer would be more valuable? I mean, if you get good enough playing computer games you can make more than minimum wage.


Very few gamers get sponsorships. It is not an easy way to make a living. Your argument is equivalent to telling everyone to forget about minimum wage jobs and practice throwing curveballs, on the off chance that they can score a position as a pitcher in the major leagues.


I think he was referring to the farming and selling of virtual goods.


Farming digital goods can be done by anybody , so you will have to compete with chinese farming groups who are happy with a return that's about break even with what the electricity and broadband would cost in the west.

At least with transcribing english text you have some advantage by being a native english speaker.


No, I'm saying before you waste hours and hours doing tedious tasks on Turks, just get a job at McDonald's. It will pay more and probably be less tedious.


It's the difference between a guaranteed payout right now vs a possible payout down the road. In order to get paid for online games you probably have to practice for quite a few hours for free before you might get good enough to be compensated.


True, but at least you might enjoy it.


Not much...

I am a avid gamer, but I gave up forever in going pro, because practice to reach pro level is tiring, VERY tiring...

One pro player lived unoficially with me (actually, he lived somewhere else but hated his roomates, so he hanged around my house instead), he would arrive from university, sit on the computer, and play until 4 in the morning, eat delivery McDonald every day, drop on the floor (yes, the floor, not a bed, or a cushion, or anything like this), sleep, wake up at 7 and go to university (without taking a bath...)

Sometimes he would get too smelly or clearly lacked decent food, then I and other roomates would kick him out back to his house, and let him return about 3 days later.

He is a player I still don't enjoy playing against (Because he wipe the floor with me), but even with all that amount of ridiculous practice, and winning championships, he still do not got sponsors... Now he went to do something else, and play for fun only (and his performance decreased accordingly actually... during his pro years, I would lose 100% of the matches against him, now I only lose about 70% of them, and I am sure I don't improved, since I did not play for 2 years in a row and since my return I had in total about 30 matches, while a pro player could pull 20 practice matches in a day)


How many hours do you have to practice a day to be a pro?


The barrier to entry for being a worker in Mechanical Turk is extremely low, and the system is very flexible. I could see it being very attractive as "side job" type work to monetize spare hours, rather than as ones' long term primary form of employment.


It's good to see human capital being utilized, even if it is ont he cheap.

It is, in fact, especially convenient for those who buy this human capital to get it on the cheap.

I notice also that you're arguing that this project will allow machine learning to essentially capture this capital and get even more value out of it. Obviously those now selling their skill and knowledge for cheap will not get a cut of that result.


I've been fascinated by the mturk concept since it launched. I sign in from time to time and complete tasks that look interesting.

Unfortunately, the service is flooded with $0.02 spam tasks (Register an account and post ad-copy on a specific forum etc). The highest paying single tasks routinely involve translation/transcription of Arabic.

The most I made from a single task was about $2.30 for 10 minutes, part of which was completion of a pre-screening task to demonstrate a certain level of English reading comprehension.


I used to do tasks now and then, but it feels like Amazon abandoned the platform. Early on there were more interesting tasks, engagement by the team, even improvements now and then. Now it's full of spam and scammers that Amazon doesn't seem to be doing anything about, and the platform doesn't seem to have had any improvements made since 2008 or so. To the extent it still works at all, it's because there's a subculture of "turkers" who use 3rd-party message boards and browser addons to navigate the mess.


The main thing that keeps me from engaging in mturk is finding tasks! I too like the concept, but I too do not speak Arabic and will not spam for pay.

There were some transcriptionist tasks that looked interesting, but the pre-screening task was looking for professional transcriptionist level performance.

Minimum qualifying WPM was like 70, and the acceptable error rate per minute was... zero? One? I'm not that good.


70 seems like a fairly reasonable rate to select for. If you've been touch typing for any measurable number of years I would think you'd at least be in the 70-80 range.


I don't know about you, but while I can type at 70WPM for typing tests, I generally can't sustain it for long periods of time, and my error rate increases.


Concerns are misplaced for many reasons.

1) no supervision

2) no dependency on any particular employer

3) ability to choose what to work on

4) ability to choose own hours

5) intrinsic trade-off between job compensation and the speed at which the job will be completed

6) no requirement that the worker be doing the task exclusively -- the could be watching TV, or chatting with a friend, or playing a video game

7) No possibility of employer coercion

Here's another scenario: say you were paid to watch ads. Should that be subject to minimum wage laws? Well, clearly not: we're routinely paid $0 for the ads we're subjected to.


I'm not sure it's accurate to say that we're not paid to watch ads -- we're not paid a salary -- but often we're given something else. Eg: substantially reduced prices on magazines, newspapers, tv etc (compared to if the sellers were to attempt to maintain a similar margin without ads).

Personally I'm in favour of higher prices and less ads and product placement -- but to say that we're not paid to watch ads sounds a little disingenuous (while technically true).


Like Airbnb takes advantage of a previously untapped supply of places to sleep, like Uber takes advantage of a previously untapped supply of driver's time, Mechanical Turk takes advantage of a previously untapped supply of people's minds. But is it as profitable?

Why isn't there something that makes it super simple for me to take surveys on my phone while I wait somewhere? I'd rather answer some marketing questions for a quarter than play Dots as I wait to meet a friend.


*>"Why isn't there something that makes it super simple for me to take surveys on my phone while I wait somewhere?"

Judging by the going rates on mturk, you aren't going to make a quarter for a simple survey, more like a few cents. In my experience, a task that pays $0.25 likely to take 10+ minutes and require some actual effort.

Not that these are bad things necessarily. I've seen some pattern-recognition type tasks on mturk in the $0.25 range that I'd call fun.

Also, in the case of actual surveys I wonder how the value of a regular survey taker's responses would change over time?

Would it go up by creating an ever more specific profile of yourself or go down as the collector already has data for your demographic?

Are the surveys even necessary? Could someone simply create an open market for personal data?

Just put some subset of your current snapshot up for bid and collect small passive income from accesses to it cutting out the middlemen who currently collect and market this data via their "free" web services.

Fascinating possibilities, imo.


Looks like there's an opportunity to sell an app for this. A key for leveraging the untapped brainpower is presenting Mechanical Turk work in a way akin to many successful games: optimize the interface to a mobile device, make it "instant on" with pre-filtered options (no/minimal login & other logistics), just present the next task as something to do in a minute or so (or skip in a second). Sell that for a buck or $5, pushed as easy for a buyer to earn back fast. Crack for intellectuals?


The desktop interface ought to be optimized as well. You could even go as far as bringing together multiple sites like M-turk to keep your filtered job queue full (standardized API, anyone?)

It should also be possible to write something to check a bunch of bitcoin faucet sites and present tasks as they come available (though I suspect that's less than honest work, heh). Many of those have timers, which are easy to predict, and video watching can be put in the background so you can do things in parallel.


Exactly right: concerns about minimum wage are misplaced when the workers are foreigners. CloudFactory in Nepal pays workers about $1/hour (disclaimer: visited them, think they're cool, no other affiliation). Terrible for US workers, more than being a high school teacher in Nepal.

What concerns me more is how all these systems are difficult to use and built for humans to complete tasks. The next revolution should be letting programmers use AIs.


The next revolution should be letting programmers use AIs.

Because the only thing stopping them from using AI is that they aren't allowed to?


I think he's referring to how mturk and other things like it don't allow the use of AIs, it has to be an actual person.


I should have been more precise by adding in that marketplace.

Last I checked, MTurk disallowed non-human workers, and that's holding back a lot of potential progress, IMNSHO.


While on the subject of mturk, you might find this useful: "Running Experiments Online Using Amazon Mechanical Turk" https://sites.google.com/a/cognitivescience.co/knowledgebase...

Found it the other day via Steve Blank's amazing list of startup tools/resources:

http://steveblank.com/tools-and-blogs-for-entrepreneurs/


I don't think it's just companies online who use "independent contractors" to dodge minimum wage. I believe I've seen it firsthand at Walmart where certain staff are replaced by contracted workers.

Next year the big scam is going to be everyone only works 29 hours. Because at the 30th hour the corporation has to contribute towards health care.


Here in Brazil a stupid law was made that if you have more than 1000 employees you must have 5% of your workforce being impaired/disabled people.

The problem is that for some jobs that is almost impossible (for example, Call Center... you need someone that is not retarded, not blind, not deaf, not quadriplegic, leaving you with paraplegics, but paraplegics, specially those willing to venture shit sidewalks, are nowhere near 5% of the population).

So we have lots of companies with 999 employees. When someone want to expand their company, they instead create a new company (with more 999 employees), and have them both work as contractor to something else or something like that.


Not sure how the 29-hour thing is a "scam". More like a "completely foreseeable consequence and understandable policy".


how do you explain the fact that this was not observed with Massachusetts' very similar health care system? http://www.urban.org/publications/412583.html


That PDF shows nothing about full-time vs part-time downgrading. It only shows employment numbers were not reduced.

In fact by reducing everyone to 29-hours, they could in theory claim to hire MORE people, while carefully saying nothing about them making a living wage or the state paying for their insurance vs the employer.

That PDF is actually depressing when you look at the graphs, uninsurance was only reduced by a little over a third for all that change in policy.


> That PDF shows nothing about full-time vs part-time downgrading.

From the text:

"To capture potential changes in hours worked in response to health reform, we also examined trends in full-time and part-time work over time."

> uninsurance was only reduced by a little over a third for all that change in policy.

this reduction occurred on top of a trend of great increases in uninsurance nationwide at the same time:

> Between 2008 and 2010, Massachusetts, the four comparison states, and the rest of the nation all experienced sharp drops in employer-sponsored coverage, reflecting the impacts of the recession. However, it would appear that health reform mitigated the effects of the recession in Massachusetts on employer-sponsored coverage to some extent, as the level of employer- sponsored coverage in Massachusetts, which was below that of the four comparison states prior to health reform, moved above the level in those states after health reform and has remained at a higher level.


According to your document, the reason is that MA's employer penalty is about 7-10x smaller ($295) than the Obamacare employer penalty (either $2000 or $3000).


The document refers to this disparity but hypothesizes the opposite conclusion based on the fact that fewer firms would be impacted:

> Although there are differences in the details, with penalties for firms that fail to comply higher under the ACA than in Massachusetts but fewer firms likely to be affected nationally, the broad similarities between the ACA and Massachusetts’ reform suggest that we can expect to see patterns in the response by employers under the ACA similar to those observed under health reform in Massachusetts.


There's no next year, this has already been happening for years. The requirement (used?) to be 35 hours or less is considered part time and dd not require insurance. Walmart, which you mentioned, is infamous for making everyone a 'part-timer' to avoid health-insurance and other benefits.


Last time I checked you could do pretty good with the audio transcription tasks if you have the knack for it (minimal stopping/replaying parts). The surveys usually aren't worth it but sometimes you'll get one that's only a couple of questions..


I did Mturk for a few months while traveling. Occasionally I was able to make ~7.00 an hour, but the average was closer to 2.50. The work is mind-numbing and underpaid.


If it's mind-numbing, doesn't that by definition mean it's not worth paying very well?


Standing watch is mind numbing, but can be associated with risk - and isn't always paid terribly low.


Has anyone made an AI that can complete some of the tasks?


I wanted to, but they specifically forbid this.

As the article mentions, the tasks are supposed to be those which cannot be easily automated and require the attention of a human.


What's the reason to forbit automation? If they cannot be automated, then they won't be.


Mturk can be used as a source for automation training.

If you start writing bots to do the tasks, you're now the mostly-blind leading the blind.


That could be addressed by having an optional meatbags-only flag when creating tasks.


I guess it may also be for quality. For example it's not inconceivable that one could write an audio transcription bot that is quite good and may be good enough for a lot of tasks, but still not as good as a human and may fail quite badly at complex tasks.

Once the market gets flooded with bots, the buyers will work out what's going on and start to lower their offers to cover the risk of getting a bot transcription. Then the real humans leave.

And of course, if a task can be fully automated then amazon would prefer to integrate it into their cloud offering and take all of the profit themselves.


Amazon has to deal with the quality of human workers, too. It shouldn't matter who is behind the work.

>And of course, if a task can be fully automated then amazon would prefer to integrate it into their cloud offering and take all of the profit themselves.

And that's what Amazon should do if they could.


Humans are more likely to care about reputation than bot herders who can trivially flood the market with programs which will work 24 hours a day. As soon as you allow bots, it becomes 99% bots very quickly.

You would also get people doing some work themselves to build reputation up and then swapping in a bot.

It would be quite interesting to build a dedicated market for bots. Just let people upload scripts with a documented input/output format and then pay the developer a royalty each time it is run.


From Amazon's perspective it doesn't matter if the user is a bot or a user. An user can also work themselves to build reputation up and then start doing low-quality work to increase pay. If you can't create a market for humans, you can't create a market for humans and bots.


The selling point of MT is that you can get human tasks done, if bots chase the humans out they loose that. It's similar to marketplaces like etsy in that regard.


Who are "they"? Amazon shouldn't care if the turks are humans or bots, they'll make the same dollar in any case. The people who create tasks for turks shouldn't care either: they just want stuff done.

The point of MT is that it's a market for jobs that cannot be easily automated. If someone can automate the task then so be it, it doesn't matter.

The point of MT is not a market place for "tasks that can only be done by humans", it's a market place for "tasks that can't be automated easily, therefore everyone and everything should have an opportunity to complete the task by any means what-so-ever. if you can only complete tasks on drugs then so be it. if you can only complete tasks by writing a bot so be it.". The seller just wants task to be completed cheaply.


The advantage of listing a job on MT vs finding a program to run is that you know it is at least reviewed by a human being who cares somewhat about their reputation within the system.

Once that guarantee goes away you affect the value of all transactions within the market. What is commonly known as a "Lemon market effect".


You may not make minimum wage recycling cans either, so lets shut down those bottle and can returns too!


You can do minimum wage if you write proper english for some of the description stuff.


that was an underpaid job ten years ago when it was called technical writing. barely above minimum in mturk is a sad direction.


The saddest part is the consequences. The quality of technical writing has fallen dramatically over the years.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: