Hacker News new | past | comments | ask | show | jobs | submit login
Data scientists shouldn't use Upwork (racketracer.com)
239 points by data4lyfe 6 days ago | hide | past | web | favorite | 149 comments

> the dataset contained ... demographic attributes like race ... to predict actual work performance

Regardless of the original intent, that's a request to launder a protected class into an opaque "work performance" metric. Also, while they are not on the unfortunately short list of protected classes, using demographics like age is probably going to create similar discrimination problems.

If you want to know about job performance, use data that is at least theoretically related to someone's performance. While machine learning is going to find some sort opf signal in whatever pile of data you feed it, that doesn't mean it actually predicts something complicated like job performance. When you feed it a bunch of general demographics, it's going to find one of the signals that we know are present throughout existing demographic: the institutional discrimination and similar problems of the past.

I spent the whole time reading the article thinking "so you got scammed by people looking to find a way to hide prejudice behind ML? What a surprise..."

Maybe the lesson here is that people doing unethical things will take the opportunity to do more unethical things, like not pay you, and it may well be in your self interest to apply your ethics when choosing a job? Upwork doesn't look great here, but seems like it's hardly the whole problem.

No, that's not the lesson. Upwork the platform scammed him, not the 'unethical' clients. The clients were looking to settle the bill.

I mean, the clients may be unethical too. But more in a not-hiring-minorities kind of way than a scamming-their-contractors kind of way.

That's a pretty wild assumption. They clearly were willing to pay.

I actually thought that line was a joke when I first read it. "We don't hire non-citizens, Hispanic people, or anyone over 30" sounds a lot worse than "we evaluate candidates according to an algorithm trained to predict work performance."

Before tech I did a stint in recruiting. We had a client demand only "white sounding" names. We ignored that and provided any qualified candidate. They complained because they mistook the recruiters name for an applicant. We fired that client the same week. It was a truly disgusting thing to be asked and I'm glad my company made the right call. Any future client that asked that was immediate grounds for client dismissal, i.e. didn't require management approval.

I'm conflicted about this, on the one hand it's completely wrong, but on the other hand would a non-white person even want to work for them? In the end their own biases will cause them to fail.

There are plenty of people who need to work to eat and can't be picky with employers.

Excellent point!

Isn't age a protected class though? At least in the US, discrimination over 40 is covered by the EEOC.

They are protected against certain kinds of discrimination, but no human discriminated against them. It was simply a dispassionate algorithm and an algorithm can't display malice!

Grrr... or that's what they, and probably the courts, would say.

The OP calling it "discrimination laundering" had it right.

> probably the courts, would say

The courts are the ones that created the doctrine of "disparate impact." A criterion can be discriminatory even if it doesn't look discriminatory on its face. And, importantly, it does not require intent to discriminate, only impact.

I don't think you can use "no human discriminated against them" as a defense. It could still be constructive discrimination.

I'm hazy on the details, but it's not a protected class in the same way that e.g. race is. In alleging racial discrimination (in certain legal venues), statistical evidence of racial discrimination is admissible, however statistical evidence of age discrimination is not. E.g. if there are layoffs, and every grey head disappears, that's not grounds to allege discrimination. You'd have to have someone on record saying "we're firing all of the old fucks because they remind me of the inescapability of my own mortality" to litigate.

Someone with a better grasp on the particulars here please chime in :)

Having race in the dataset itself might not be bad, if there was an expectation that the data scientist would use the data to check if the algorithm was biased in some way, but it seems like the customer was pretty terrible sooo who knows.

If you think it is not possible for race be related to job performance you must not have watched much NBA basketball.

You can't make effective prediction off this stereotype. It is useless.

There is nothing inherent in "being black" that leads to greater basketball performance. Also, define "black," good luck.

Being tall, maaaaaybe. But not perfectly so, and there's no useful correlation between tallness and race, considering that the NBA generally just selects for the most skilled basketball players.

Then you get into all the cultural influences that totally shoot any genetic arguments out of the sky, and really just makes trying to do any, uh, science, around genetics and the NBA pointless. In my experience (as a southerner) it's typically just a sort of dog whistle kind of psuedoscientific attempt to set the groundwork that race can be related to other things, such as likelihood to be a criminal, hence why I jump on this argument whenever I see it. Apologies if that wasn't your intent.

In short, "race" (good luck defining even one) is not related to job performance.

>There is nothing inherent in "being black" that leads to greater basketball performance. Also, define "black," good luck.

How so universities define someone as black?

By self-identification.

Aren't blacks genetically predisposed to larger twitch muscles (faster reaction times, etc.)?

Yea, happy to explore this.

First, what do you mean by "black people?"

In America?, any person self described as one, or with a significant (25%+) component of the Niger-Condorfanian cluster as per [1]. I agree the definition is arbitrary but all definitions are. The same with "tall" people. We could discuss all day long who is or isnt tall, or we could define tall as being at least 6ft 2in or whatever.

[1] https://science.sciencemag.org/content/324/5930/1035

This is a good faith attempt and I appreciate that.

First, from your link:

>Our data also provide evidence for shared ancestry among geographically diverse hunter-gatherer populations (Khoesan speakers and Pygmies).

Would we say that pygmies, being considered black, would be likely to make it into the NBA?

Second, can we guarantee that people who self identify black, would agree that the definition of someone else's "blackness" is legitimate? Whose definition do we "choose" to then move ahead with "and these sorts of black people are more likely to get into the NBA?" Given that it's not well agreed upon (is Obama black? Steph Curry? Not as cut and dry as you might assume. Are they "black enough" to have NBA advantage?)

Third, based on your tallness: why draw the line as 6'2"? There are shorter players that were Great. So, how do you deal with (relatively frequent) exceptions? "Tall people are more likely to get into the NBA, except when theyre short but grew up next to a court" ? And the tall People that aren't good at basketball? "Tall People are good at basketball except when they're not" ? And when they're too tall to the point that it's detrimental to their health and can't even play basketball?

Fourth, arbitrary. Exactly. It is arbitrary, and thus I argue scientifically impossible to have a rational discussion about blackness and its influence on likelihood to enter the NBA. Or, rather, develop an effective predictor algorithm based on genes alone.

We could have interesting sociological conversations about basketball courts in predominantly black neighborhoods, black NBA stars inspiring black kids, self-propagating cultural memes about blackness and basketball skills, etc. But we won't get any really good information that allows us to say something like "black people are better at basketball" to where it will be true.

I agree with some things you said others not so much. By the way, being this the Internet, I am not trying to push any weird racial stuff here, I am only trying to argue for the position you can operationalize the race concept based on arbitrary but objective genetical clusters in the human population. This can be useful for myriad things: Anthropology, population genetics, hereditary disease, diet requirements design, human evolutionary history, etc etc. There is nothing wrong with the concept that there are groups of genetic markers who are different among populations in the world.

Regarding pygmies, they are a different cluster from the highly homogenous cluster of Niger Cordofian from which current African American population in America descends. From the article I cited: "High levels of heterogeneous ancestry (i.e., multiple cluster assignments) were observed innearly all African individuals, with the exception of western and central African Niger-Kordofanian speakers (medium orange), who are relatively homogeneous at large K values(Fig. 5C and fig. S15).)".

I see you are also a bit disingineous if you dont think there is a high correlation between athletic abilities and genetics. Gimme a competent geneticist, 200 hundred genomes of 15 years old boys: 100 African Americans and 100 Guatemalans, shortest dudes in the Americas. I am pretty confident that I can pick 2 teams of 5 kids just by their genome and pick the correct winner of a basketball game between them with high probability. Yeah I cannot predict individuals outcome(yet) but population wise I think it is not a crazy stretch.

On the other hand I see your point that there is no crystal clear boundary between any human group, but if you take that position you have to be consistent with it. What I mean is that I assume you are not happy with an universal definition of black (I can understand and even agree with that). But then you are forced to admit the concept of White privilege is vague too. Same reasoning. You could say "White" is a cultural concept, but I could say the same regarding "Black" so we are back to part 1.

Again I suspect we have more points in common that differences regarding this topic, I just wanted to put out there some considerations.

I don't have enough time to give you the response you deserve for the time you've taken, but I do appreciate your insight and will think on it.

As all parts of culture, the concept of white privilege is extremely vague, and I imagine white Milwaukee trailer park residents would be a bit offended to be lectured about their privilege. Hence why I would prefer we spend our time focusing more on socioeconomic factors of humanity rather than genetics - it just seems more useful.

I may return to edit this if I have time today but if not thank you for this discussion.

+1 to both of you for having a polite and coherent discussion about such a divisive topic. I enjoyed reading it.

Guatemalans are short because of chronic malnutrition not because of genetics. Guatemalans born and raised in the us are much taller.

Perhaps, yeah they can add some inches with nutrition. They are still shorter than much poorer countries like Haiti. A rich country like Japan has a short average height so nutrition helps up to a point.

It’s only a couple inches shorter than Americans and a lot of that is still based on how much and the kinds of protein people eat.

Or Sherpa people carrying luggage up Mt Everest...

There's a difference between acknowledging some correlations, and using them as statistical tools in the job market. The latter is thankfully illegal or at least frowned upon.

When they built New York's skyscrapers, they hired predominantly people from the Mohawk tribe to do riveting a three hundred feet above ground because they lacked fear of heights.

At what point does a correlation become useful as a statistical tool?

Every proven correlation is immediately useful as a statistical tool! But we also don't jail people when there's an 80% chance they're guilty. Not everything is about efficiency.

The best athletes are extremely intelligent and driven people. The over-representation of black people in athletics is most definitely related to their under-representation in other fields.

Nah, intelligence is orthogonal to sports skill, that is, it is not correlated. Plenty of dumb and smart sportsmen. Now, mental fortitude, self-belief, ability to perform under pressure, now those things I believe. But none of them are strongly correlated with intelligence.

Or it might just be because basketball is the preferred sport of a lot of black people in the US, so they are more likely to play it and become good at it.

If you look at hockey, it is mostly white. That doesn't mean being white gives you an advantage in playing hockey. It just means way more white people enjoy hockey compared to black people.

> If you look at hockey, it is mostly white. That doesn't mean being white gives you an advantage in playing hockey. It just means way more white people enjoy hockey compared to black people.

I suspect it's more that playing hockey implies a certain socioeconomic stratum (hockey is an expensive sport).

You can play a pickup game of hockey with something to thwack around as a puck, a stick, and some skates, all of which are dirt cheap. I think it's kind of a stretch to make any socioeconomic predictions when the much simpler correlate is that hockey is cheap and easy to play (and thus will have a larger talent pool to draw from) when you live where local ponds freeze over in the winter, and those parts of the country are predominately white.

Did you grow up with any competitive-level hockey players?

The sport is a VERY large time and finance drain on a family.

That's a bit of a non-sequitur. High-end-competitive-level anything is a tremendous time and money sink. But that doesn't mean that the majority of the people that play the sport in question need to be wealthy to participate at the rec-league levels.

> But that doesn't mean that the majority of the people that play the sport in question need to be wealthy to participate at the rec-league levels.

Rec-league hockey requires an ice rink and time on it. Ice rink time is highly demanded and you may be getting it at 6:00AM or 11:00PM. You also need to coordinate teams at those times. And if you want any practice time-you may be getting that at 3:00AM in the morning (not joking).

And, for all of that, you get the joy of shelling out lots of money for the privilege of using the ice on the graveyard shift.

ice hockey is very resource consumptive, both in terms of money and familial time) attention. Ice hockey requires both equipment and ice-time; both are costly and don't have parallels in other sports.

Why does it require more time than any other sport such as basketball?

How many basketball courts exist vs how many ice rinks exist?

Practically every high school and elementary school has a basketball court--that's probably 10,000+ in Canada. It doesn't count any associated with public parks, etc.

Contrast that with ice rinks which are probably about 1000. You have at least a factor of 10 differential--probably more. In addition, ice rinks are vastly higher outlay to keep running.

So, that means real money to use the rink when using a basketball court is, at worst, a nominal fee. Longer travel to play against another team if you're in a league. Fewer coaches which need to be paid more. And, your practice times are in the "dead zones" from 11:00PM to 6:00AM in the morning as the ice rink is looking to make real money from 3:00PM to about 11:00PM.

Given how small a population “best athletes” is (..by definition), I somehow doubt this.

I haven't read pertinent studies in awhile, but it used to be that all top sprinters were of West African ancestry due to a disproportionate amount of fast-twitch muscle fibers in the leg muscles.

I think it is less "intelligence" and more "naturally physically talented"

"According to twin studies, the Big Five personality traits have substantial heritable components explaining 40–60% of the variance" https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5068715/

Trait conscientiousness and IQ predict job performance. Stop being Luddites with this technology.

It's a politically-motivated, Luddite-like mentality with respect to the technology available via psychometric analysis on identical twins.

I freelanced on Upwork for a bit. It was all the worst aspects of having an abusive boss (the platform itself), and all of the worst aspects of freelancing (unpredictability, getting ripped off by shifty clients).

"Creepiest shit ever" is a pretty accurate description of Upwork. If Upwork's attitudes and policies were described to me in the 90s, I would have said it was lame, unrealistic science fiction with an over-the-top villain.

If anyone wants to do freelance work, I'd recommend marketing yourself directly to local businesses, print some business cards, schmooze. It's about as much work and stress as using Upwork, and you don't have a misanthropic middleman taking their cut.

It blows my mind that UpWork does so well as a platform. I have never had a good experience using it. And I don't know anyone who says good things about it.

I think there is a real market opportunity for a startup that can provide a way to connect businesses who need short-term or specific project work (i.e. they don't want employees) with freelancers who want to work in that mode while (and this is important) avoiding the creepiness and flaky nature of UpWork. Conversely, it would also need to avoid being too similar to having the middleman as an employer.

I probably know 50 people who would jump on a platform that hit this sweet spot in a heartbeat.

I feel you should check out Codementor, as to me they are that kind of a platform: https://www.codementor.io

I've tried Upwork in the past and the platform made me miserable.

I've been using Codementor for the last year or so and been very happy with them. For the past year I've worked for a US client and since a few weeks I work for an Australian client.

Why I like Codementor:

- The payments are done on time and they don't extract some platform fee as far as I know for the freelance jobs. Or it's invisible to me (perhaps more likely). What I ask per hour is what I get.

- There seems to be a decent amount of jobs available every month and not too many people respond to the same job as me at the same time. So there's a good chance that if I respond the job, I at least will be invited for a video call with the potential client.

- The rates on the platform seem decent (they advise setting somewhere between 40 - 80 USD, no race to the bottom imo). I'm based in Thailand, so these are rather nice rates for living here - I can save /a lot/ of money every month. My costs of living is currently around 1700 USD per month, everything above I can safe.

- I don't have to install software on my computer that spies on me to make sure I put in the proper amount of hours. And I actually tend to work more hours than I bill my clients, but I don't need software to keep me in check.

If you're charging $80/hour but then work more hours than you charge your effective hourly wage is less than $80/hour. This is an incredibly low wage (less than half) for typical web design work in my area (New England).

I guess it's all a matter of perspective. As stated earlier, in the Netherlands I worked as a freelancer for 70 EUR an hour (probably around 80 USD or so) and there it's not considered low. You'd probably be in the 10% highest earners in the country. And then I'd also probably lose ~35% of my income on income tax. Here in Thailand I pay pretty much no tax and cost of living is cheaper so for me this is a great improvement.

Oh, $80/hr is still well above the top 10% of earners in the US, it's just not very much for web design.

Any tips for freelancers who are recent engineering grads? I have some portfolio items, but I still can’t claim more than 1 year experience

Spend a year working for a consulting company, to learn the ropes. If I were a business looking to have a freelancer build something for me, 'a fresh grad, with less than a year work experience, who has never done this sort of thing before' is not who I'd hire.

I have 9 years experience, and if I were to to go into freelancing (The kind where you solve problems for businesses, as opposed to the kind where you're a 9-5 1099 contractor), the first thing I would do is to take a job at a consulting firm, and spend my time there learning. Figure out what works, and what really, really doesn't.

That job is probably going to suck, but you'll have a much better idea of what to do, what your customers really hate about working with big firms, and your resume will look much better, when you can put down "I did the thing I want you to hire me for, in a consulting firm, for 18 months, I know how this works, and I can do it better."

Very solid advice.

Working as a consultant will also let you build a network that you can use when you want to go independent. Obviously you have to be careful not to poach from your old company, but often there are side projects and jobs too small for a large consultancy that are a good fit for an independent contractor.

When I was a consultant I considered it a slight failure if I didn't get a job offer from the client at the conclusion of each engagement. I never took any of them up on it, but if I'd been in the market it would have been a hell of a way to basically get paid for an extended job interview... and lots of people I know who decided to get out of consulting went over to the client side. That's a pretty standard path, really.

> I have 9 years experience, and if I were to to go into freelancing (The kind where you solve problems for businesses, as opposed to the kind where you're a 9-5 1099 contractor), the first thing I would do is to take a job at a consulting firm, and spend my time there learning. Figure out what works, and what really, really doesn't.

I have to disagree.

I've worked for several tech consulting firms(8 years of experience) and then started my own consulting company 2 years ago. And in my running a successful consulting firm involves 3 key skills(in order of importance).

Sales - This is definitely the hardest and most important skill. And unfortunately is also the skill developers usually have the least experience in. This involves finding leads and closing deals. This will take a long time to do and is emotionally very rough for most developers.

Customer/Relationship Skills - If you're already a nice a polite person who doesn't mind eating the occasional shit sandwich this mostly involves repeatedly learning the lesson that however important you think communication it's more important you think.

Execution - This is mostly involves a combination of being able to correctly identify requirements and technical execution skills. Most developers usually have plenty of experience hear.

The experience most developers lack is sales. And most consulting shops won't put you anywhere near a sales or account management role unless you have previous sales experience or have spent significant time at the company. A much faster route to learn these skills is to just start a company and start doing them. (while reading a bunch of books and talking to anyone you can who has relevant experience)

> Execution - This is mostly involves a combination of being able to correctly identify requirements and technical execution skills. Most developers usually have plenty of experience hear.

How do you think a fresh-out-of-college junior with less than a year of experience going to be at execution?

Knowing nothing but that, I'm going to take a statistical guess, and say: "Probably not great." I'm going to make another statistical guess, and say: "They probably don't even know what they don't know."

Also, joining someplace that's not consultancy-focused (like Google or Facebook) isn't necessarilly going to help with building relevant experience.

Yes, you're going to learn a lot about how to move protobufs around from one distributed system to another, and how to do on-call, and how to work in the cloud.

No, this is not the most useful set of skills to have, when the insurance broker down the street wants to pay you $XYZW to build them a custom SalesForce widget.

Sorry I wasn't disagreeing that the fresh out should get some experience. Just I don't think that working at a consultancy is going to be much more helpful than working any other development job.

> Also, joining someplace that's not consultancy-focused (like Google or Facebook) isn't necessarilly going to help with building relevant experience.

It's soo much more important to have worked for one of these companies. If you have Google on your resume it will work magic. Clients love that. It'll help with leads, it will help with negotiation, and it will help close deals.

Honestly the two things I wish I'd done before I started my company was work for amagoobooksoft and put in some time at a sales job.(cold calling or some type of lead gen)

That's great advice. Even with quite a few years of experience at a large firm as a product manager, I was pretty clueless when I first switched to an industry analyst/consulting firm. This is spite of having been a client of said firms.

There's no way I would have known where to begin if I had just set out on my own.

> Any tips for freelancers who are recent engineering grads?

If you have a relevant degree, why not to apply for a remote job position instead? That would help to build your CV, and allow to switch to contracting later on.

Stack overflow jobs board thing is the best place imo to find remote Dev positions. Harder for new grads.

Angel list might be a place to find remote AND newbie friendly jobs.

Second AngelList for this purpose, though keep in mind the salary trade off to work remote for a new grad is likely gonna be considerable.

I can only speak for myself, doing mostly web dev/maintenance for local businesses at this point in my life.

I started trying to freelance online, with little success. When I mentioned to a dentist that I was freelancing he asked me to fix his old website. A small store owned by a friend asked me to make a website. Later the dentist asked me if I had cards to give to a friend.

About 75% of my business since then has been word-of-mouth. The rest has been me contacting local businesses that don't have websites. Business cards and flyers are supper cheap.

That being said, after three years, I'm only now starting to feel comfortable. There were very long dry spells, in between short periods of intense business, and word-of-mouth takes a very, very long time to operate (hence me reaching out to businesses without websites). I had to depend on my wife financially for stretches. I've also had to learn design, which is a much different skill-set to programming and has been very difficult. The main reason I went down the freelance path is because in my last salaried job, my boss developed substance abuse issues and unpredictable behaviours, which lead me to having a burnout. It's been hard but it's gradually starting to pay off. For all the difficulties, it is much more rewarding than watching all the value I generate go to some guy with a business degree who got through college doing elementary algebra.

Anyway, that was a fun rant, hope it was helpful to someone.

This kind of makes sense in the same way that AirBnb does. There are more users looking for work than employers looking to hire, just like there are more tourists looking for cheap accommodations than landlords willing to rent their space. The platform will cater to the group with the higher bargaining power.

However, the surveillance just seems like an absurd waste of resources. The author quotes $400 for the job, why does it matter how many hours it takes them? If you think $400 is too much, then bid on the project.

Long term upwork Data Scientist here. It seems he had a bad experience, that could have happened on any other place / platform. There are always horror stories about how platforms deal with conflicts (think PayPal, Stripe, Uber...)

Out of >30 contracts I had, only 1 resulted in a dispute, and it was decided in my favor. I do agree the whole monitoring thing is absurd and I generally don't take jobs from clients that demand hours to be logged through the monitoring app. If you want to see how the jungle looks like, please have a go at freelancer.com.

The monitoring app is a hoop I guess. I would hate it too because it's a form of control over the work.

I once had a client who, at the negotiation of the contract, wanted me to work from an "office" (an in-laws quarters) for a specific set of hours each week. His desire for that was because he wanted to make sure he was getting his money's worth; that if I said I was charging him for 10 hours, I was spending 10 hours on the task.

I told him that, along with other stipulations, would qualify me as an employee. He and his fiance/business partner insisted it didn't. That was troubling because his fiance was a lawyer.

I declined to take the project.

> I generally don't take jobs from clients that demand hours to be logged through the monitoring app

Isn’t like 9/10 jobs there require you to use the monitoring?

Not at all. Maybe for low paying jobs.

I'm working on a matchmaker service for a certain niche industry that will be completely free. Hopefully it will catch on. Down with Upwork.

My company uses Upwork (as a services buyers) for hundreds of contracted hours per week, and have been for many years. We’ve never required any contractor to use screen recording for hourly work — that is optional — and have never heard of anyone using a webcam. We’ve also never disputed a single hour of work that a contractor has billed, nor have we heard any of the dozens of contractors we’ve hired mention this as an issue.

I think you might be extrapolating from a single bad data point (never a good idea, especially for a data scientist) and reaching an incorrect conclusion about the Upwork ecosystem.

While I appreciate your perspective, I can imagine there’s a lot of people hiring that have zero idea how developers & data scientists work and think screen recording is a brilliant idea and thus insist on it.

I’ve seen a lot of really shit job postings on Upwork & similar freelancing platforms that confirm the person posting the job has no idea of what they’re doing.

I imagine there could be employers who insist on this sort of thing screen monitoring (which sounds crazy to me) -- but the good thing about this marketplace is that contractors aren't forced into any particular contracts. If they don't like the terms with one buyer, they could "vote with their dollars" and work for another.

Second this, I've been both a nontechnical and a somewhat less nontechnical buyer on Upwork. When I was more nontechnical before working for Gigster (freelance software dev marketplace), I would have said that screen recording is brilliant. After all, how could I trust someone who is doing work I don't understand? At least I can look at their screen and make sure they weren't on HN when they were billing me :)

Somewhat less nontechnical me knows better now, but if I didn't have any idea how to hire still, there's a good chance I would be one of those bad buyers.

If Upwork will disqualify you for payment for not fulfilling a set of rules in a rather than look into the source of the dispute as a misunderstanding from a completion, I find that in bad faith and a terrible customer support experience.

I also have a high rate of success with Upwork for freelancers, from designers to developers, however I believe their recent changes, for example to charging to invite more than 3 candidates, is going to cause many to look elsewhere. They are increasing fees and decreasing usability. I'm open to new options.

I used to work on Upwork and almost all of the projects required screen logging. I think this was before there was any webcam thing in it.

You registered in 2012 and thought you'd use your first ever post - 9 years later - to defend Upwork?

I've been using UpWork for over a year now and never had a dispute so far.

The author is right in saying that UpWork only guarantees payment when time tracking using their app. Where he is mistaken though is where he says that the app requires camera to be on at all times. There is such an option but personally I'd never go for a project like this. And yes, the app will take the screenshot of the stuff you are working on.

I don't get why author's customer would launch a dispute over not completing a 10 minute survey. Given that it was an hourly project I bet there were issues with communication and perhaps the author should have double checked with the client before closing the job.

UpWork is far from being flawless but for me it has been working quite ok.

Yeah, I've been working on Upwork for about 1.5 years now and it has been such a vast improvement over other freelancing sites - both in terms of the size and scope of the projects and in terms of the types of clients available. Not one of my hourly clients has ever insisted that I use the monitoring app; yet I do it anyway because then I don't underbill and the payments are guaranteed.

In my mind, the screenshot every 10 minutes is no more invasive than sitting next to a colleague in an office and if you want to do something private, simply pause the timer and do it on your own time. Upwork is a pretty great site if you're a freelancer.

> I don't get why author's customer would launch a dispute over not completing a 10 minute survey.

... so they don't have to pay for the work?

I’ve spent a sizable amount of money hiring people on upwork and never have disputed a payment. If something isn’t being done correctly, the contract gets terminated and that individual never gets any more work from me. Early on it was very difficult to spot potential problems before the contract began. With experience it became much easier.

It is an imperfect system in an environment rife with problems from both employers and works attempting to cheat their way through. Now they are a public company now and have an incentive to squeeze fees from all ends. For that reason, I rarely hire new workers from upwork any more.

Workers or clients who get screwed on the first job will never come back and they will tell everyone else not to use it. In the long run that will add up.

I'm surprised that the complaint is about getting paid, rather than pointing out that Upwork facilitated an illegal and immoral system to be built. You don't get to use "demographic information" like "race" or "age" or probably many of the columns the post mentions to make hiring decisions. For the sake of the jobs industry, the author should change their post to say this.

I feel there's such a huge potential for serious sampling bias in data like that, I wouldn't be comfortable to train classifiers on it.

Any company with dubious ethics could easily manipulate the data (collection) to match their intentions.

Things like job performance usually have a subjective component, so the training data will already have human bias built in without any intentional manipulation.

Ethics aside, you'd think a company concerned about simply covering its ass would explicitly remove demographic information from the analysis, to avoid the possibility of it influencing the categorizations.

> The central computer notices just about everything. Keeps track of every key you hit on the keyboard, all day long, what time you hit it, down to the microsecond, whether it was the right key or the wrong key, how many mistakes you make and when you make them. You're only required to be at your workstation from eight to five, with a half-hour lunch break and two ten-minute coffee breaks, but if you stuck to that schedule it would definitely be noticed, which is why Y.T.'s mom is sliding into the first unoccupied workstation and signing on to her machine at quarter to seven. Half a dozen other people are already here, signed on to workstations closer to the entrance, but this isn't bad. She can look forward to a reasonably stable career if she can keep up this sort of performance.

Snow Crash, Neal Stephenson

And now, from the convenience of your home...

As a buyer I had a few good experiences finding contractors on Upwork. If “no one should ever use it,” does anyone have a good suggestion for a better place to find good freelance coders for small projects?

I have used Turtle for 2 projects and had an exceptional experience with both.

Very well vetted devs, excellent platform and framework for managing them, and human customer service at (currently underpriced IMO) $55/hr.

The two apps I’ve used Turtle for:

- React site for my business - Ongoing Node integration work at about 10/hr per week.


Thanks for mentioning Turtle! Really appreciate it.

(1) Great talent, (2) great customers, and (3) an easy to follow process (platform) are critical for making freelancing work. UpWork typically misses at least 2/3.

HN has its own dedicated monthly listing: https://news.ycombinator.com/item?id=20325924

Also reach out to your network... I belong to several slack nodes that have channels just for contracts and leads are dropped in there pretty regularly.

Absolutely do not use freelancer.com either. They charged my CC $54 when I closed my account. Shady site.

I think you can find good freelancers everywhere... And also bad ones. Whether upwork is good or not depends on your viewpoint. I think it is just a marketplace like everything else. I personally believe bigger value lies in longer relationships, therefore places like upwork should be just a place to get started.

GitHub worked well for me when I was hiring freelancers in the past and nowadays when I occasionally freelance.

How would you find freelance jobs on GitHub?

Upwork is ruining it for gig economy / freelancing / outsourcing. They create a race to the bottom and prioritize the client over fairness. Fiverr isn't much better.

It's hard for any "we focus on everything" platform to get the specific needs of each vertical (customer + talent side) right. It's why TaskRabbit (focus on everything) never reached the scale of Uber/Lyft (press a button, get a ride).

I'm not deep in the Data Science world, but hopefully there's a platform focused on vetting customers + freelancers and providing a platform for working together easily, fairly, transparently (without the creepiness). Platforms should also decide what kind of work is fine to outsource. Long-term/strategic work the company has to live with for many years (i.e. a core prediction model or fundamental backend arch) shouldn't be outsourced. People who will see the benefits of their work in 5+ years should be the ones working on stuff that'll benefit the company in 5+ years.

I appreciate the mention of Turtle (https://www.turtle.dev/) in this thread already (I'm one of the founders).

For freelancers, Turtle: - Makes sure customers are vetted (just as freelancers are). - Pays for all hours billed (even if customers don't pay). Freelancers are trusted to enter all hours manually. No creepy screenshots. - Provides an easy-to-use platform for task management, chat, video, payments.

We're specifically focused on React, React Native, iOS, and Android dev now. Front-end/mobile is easier to compartmentalize. I'm not sure if that's possible for many forms of data science.

Part-time work is an important part of our economy and will become more important as the world further globalizes (ex: "Remote" now being a hot topic) -- but the infrastructure is seriously lacking. UpWork is a sloppy solution attempt.

I encourage freelancers, customers, and platforms to be honest about what should and shouldn't be outsourced, to make sure there's quality on both sides (great freelancers and shit customers fail), and to make sure there's an easy process to follow (great freelancers, customers, with shit process still fail).

I'm happy to see an alternative to Upwork.

I hope you will consider branching out beyond those particular technologies. For example, I'm a Node dev who has done Angular and React before but would probably rather use Vue on the front end (if it's appropriate) these days. Also have a fair bit of Linux experience which might be an important skill (as well as C++ and many other technologies from 20+ years). And I am thinking I might focus more on AI by learning to use some of the high level APIs out there. I assume there is demand for that type of thing.

I guess one of the main difficult things about branching out beyond React etc. is, how can you necessarily do a weekly demo for something that really isn't visual or interactive or may not have a short term deliverable? I think the simplest answer is just some type of screen share or short video that shows the raw progress, for example just a run-through of data cleanup scripts highlighting some of the data issues being resolved.

Thank you.

We will absolutely branch out, but not sure on timing or specific skillsets.

Fixing what companies (Startups) and freelancers (React, React native, iOS, Android) are on Turtle lets us focus on our platform.

We believe the infrastructure is the biggest missing element in making remote, part-time work successful. Something as simple as how fast or reliably a chat message gets delivered becomes important in asynchronous work. Chat, tasks, meetings, payments, and vetting processes are the core elements of our platform today and these feel critical to any vertical. The vetting processes and meeting templates are the only unique pieces to each vertical.

Since most of our customers are technical, the non-visual demo is not a concern. We are more concerned with distracting ourselves away form focusing on the platform or outsourcing things that shouldn't be outsourced (ex: strategic, long-term impact decisions should not be outsourced. Co-founders should never be outsourced).

Aside from 3/4 of your post being a gigantic ad about your own platform (which if you ever reached the scale of Upwork, I guarantee you’d run into the same issues you mention), how is it really a race to the bottom? You give the price you’re willing to work for, and the buyer agrees to the terms. Just like any other marketplace. I guess Fiverr is a race to the bottom, but that’s because their name and original mission statement (any work for $5) incentives that behavior.

I’ve used Upwork and had good success, but also there have been bad actors. It’s up to the person hiring to do research before they hire, and cut ties by firing if they don’t work out, like pretty much any other workplace. Maybe the vetting process could be better, but Upwork is the best contract to hire marketplace for a variety of work, IMO.

> Aside from 3/4 of your post being a gigantic ad about your own platform

I mention what I think is necessary for a platform in any vertical to work. We just happen to already be doing it for our vertical.

> how is it really a race to the bottom?

- Just about everyone has a 4.5+ star review, so the cheapest 4.5+ star review looks objectively better to a Co trying to cut costs. Just like any review system, it can be gamed.

- Customers get preferential treatment. That's not a fair two-sided marketplace -- that's a race to the bottom, for whoever keeps the customers happy, including the bad ones.

- Look at average salaries/hourly fees for different fields and look at UpWork. That should be the clearest "race to the bottom" evidence.

> Maybe the vetting process could be better, but Upwork is the best contract to hire marketplace for a variety of work, IMO.

They're the biggest. That's about it. Read OP's write-up, ask other freelancers, ask other customers. UpWork is a lame excuse for what part-time, remote work has the potential to become in the economy.

When I vet candidates I do an image search on their profile photo, look at their resume (and other projects they’ve done on the platform), along with the number of hours worked on the platform. Focusing only on a rating system is short-sighted.

Re: it being an imbalanced marketplace in favor of customers, I guess you’d also say the same about eBay?

I understand your motive is driven to compete with Upwork, but it’s the most built-out and legitimate platform out there, and given their size, I trust them over some smaller operation.

P.S.: I read the OP’s naive writeup. He didn’t follow the platform rules and was surprised that it went against him. Shocking. $400 is a lesson to do more research next time.

I agree that if everyone vetted as extensively as you did, UpWork experiences would be more successful. But people don't, and it's unreasonable to assume the same vetting processes for 10 hours of work vs 10+ year careers (FT hires). The platform should be vetting both sides, and they don't really.

I would say the same about eBay. Would you rather bet on eBay or Amazon today?

> given their size, I trust them over some smaller operation

Fair enough, but "trust the biggest fish" is a dangerous assumption.

> P.S.: I read the OP’s naive writeup. He didn’t follow the platform rules and was surprised that it went against him. Shocking. $400 is a lesson to do more research next time. Maybe the platform rules suck?

Any self-respecting professional would think the random screenshots thing is creepy. If talent is vetted and to be trusted, the screenshots/camera thing should be unnecessary.

Archive link: http://archive.is/Is2Uk

The title is "Why data scientists should never use Upwork"

Upwork rules require you to have their desktop app installed and let yourself be filmed for the duration of the task? Holy s

UpWork doesn't do filming using your camera, only takes a picture every ~10 minutes. Also the picture taking is optional and can be disabled by client (Throughout 1 year working on UpWork I never had a project which required of me the camera bit). The screenshot taking is required for payment protection.

> UpWork doesn't do filming using your camera, only takes a picture every ~10 minutes.

And how can I, the person being filmed, verify that?

Probably notice that your webcam light comes on only once every ten minutes, or that your network traffic is consistent with one picture every ten minutes.

Isn't this site for nerds?

You can see the work log online that contains all the screenshots and most probably camera shots too (never had to use it so I'm not 100% sure).

I think the way to know that your camera is on is to look at the little LED next to your camera and hoping UpWork didn't find a way around it.

Get a camera where the activity LED turns on when camera is active.

You can't be precious on Upwork. And there's a strong GIGO factor.

What you have to do is churn a bunch of jobs/contractors until you get the one that really pays off. Don't let the hassles get to you, just keep turning the crank.

A lot of people from India on there posting dumb jobs. Though, dumb jobs can come from anywhere. It takes hours of sifting to find real offers. Most people want you to solve climate change for $1k or build their amazing idea they haven't thought through. It's a dumpster fire but there are real offers on there. With data science, you have to stress that it is research. You can only give probabilities, there's a chance it never works as you suspected or you don't come to a conclusion.

_Nobody_ should use Upwork. You won't get paid what you're worth, and it's the worst of both worlds when you compare it with normal freelancing or working in a regular job.

Site down, we 'll never know why ...

Maybe the site was built by Upwork? ;)

—-just a joke, I have nothing against them

After reading the article, you will.

I’ve probably hired about $150,000 of contracts through up work and never once have we required the monitoring app to be used. However, we have a bit of a different approach, we will “pay” you through upwork for the first month, but if we are satisfied with your work after a month we pay you directly through PayPal or wire transfer and then we cancel the contract on upwork. Out of all of the contractors we have hired (20-30) I can only think of one who wanted complete payment through upwork.

I know how bad upwork is and when we hired we genuinely always set out to get the contractors we worked with off using it so they can keep more money.

Might not want to post that here if you intend on keeping that account - https://www.upwork.com/legal#noncircumvention

There’s an opt-out clause, but I don’t know enough about talent acquisition costs to know whether it’s a good deal. My assumption is that recruiting agencies and internal processes are super expensive, so Upwork might be a cheaper option even with the opt-out fee.

We have such a solid team now and we have lots of other channels for hiring people now but thank you!

Care to share a channel or exchange some knowledge? We buy about 50k hours a year off of upwork, would love to learn how others are doing it. John @ curtisdigital . com

The e-mail address isn't working, neither is info@<website>. You might want to take a look into that.

> This was the first task I ever received and it flickered my curiosity. In the problem set, the dataset contained survey and demographic attributes like race, age, computer proficiency, immigration status….etc.. to predict actual work performance.

> I completed the survey which was asking me workplace situational questions testing my unconscious biases and then quickly rejected the dispute.

This sounds like the "consulting company" was someone running a scientific experiment to see who is willing to create racist/ageist/otherwise unethical models.

what would be a way to start contracting if you have no existing network, no existing clients to get references?

I actually do think Upwork is a good way to start but you could

1) Build a network - go to product meetups (assuming you're a dev) 2) Contact agencies near you and see if they have contract projects you can hop on 3) Use other sites - Toptal, Freelancer, Remote, Gigster, etc.

I found work through Upwork when I was starting out as a dev and actually had a pretty good experience. Just like on the hiring side, there are a lot of bad clients you need to filter through but there are plenty of reasonable people looking to hire on the site.

It did make me nervous having so much of my income tied to Upwork and there were a few things that seemed like minor red flags, but I didn't have any awful experiences with them.

Someone with a few years of experience probably has better avenues to look at for work but it could be a good option if you're looking to get experience and build out a freelance network.

With Upwork, since they're an intermediary and marketplace and hirers are the ones who are paying and providing supply, it's in their interest to offer services that cater to them like time logging and charging connection fees, even if it's unfriendly for freelancers.

Your best best is to take matters in your own hands, and proactively email people you know you can help with your services. This way, there's no intermediary controlling your relationship with your client and you don't have to play the waiting game either (waiting for referrals or networking to pay off, waiting for people to find your website, etc.)

This may seem hard at first glance but let's the take the example of data science work. You can reach out to startups who recently raised a sizable investment round (sign they have the budget and bigger growth goals) and share a few interesting ways you can help them uncover user behavior patterns that would them hit their next milestones.

Further reading:

[1] https://artofemails.com/get-new-clients

[2] https://blog.hubspot.com/marketing/best-guest-post-pitch-exa...

this article has nothing to do with "data scientists". It is just a runt by some guy who used Upwork for the first time, did not follow their (admittedly stupid) rules and got into payment dispute.

Anyone using upwork knows you need to have the software running when you have an hourly contract. Fix contracts are different and upworks judgements can be awful but for hourly as long as you have the software running you will be paid.

Data scientist doesn't read agreement.

The gig economy sucks.... surprise!

What does this have to do with Data Scientists? So you didn’t read up on how to log time, made unnecessary mistakes, and an inflexible collective of likely underpaid service reps didn’t care enough to go beyond the letter of the law. That’s hardly a scam. That’s you not knowing what you’re signing up for and then being upset when you didn’t get the outcome you wanted. Hardly deserves front page news.

> Their rules state that you must have the Upwork Desktop App on you computer which installs some sort of video malware onto your laptop AND have your webcam visible and on at all times so that the customer success agent can audit that you looked like you were working through the webcam while staring at a screen. How is that not the creepiest shit ever?


This is standard industry practice. I remember hearing about one of Upwork's constituent companies, oDesk, doing the same thing 13 years ago. You had to run their time logger program, which would periodically screenshot your desktop and take a webcam photo to ensure you were working.

>Their rules state that you must have the Upwork Desktop App on you computer which installs some sort of video malware onto your laptop AND have your webcam visible and on at all times so that the customer success agent can audit that you looked like you were working through the webcam while staring at a screen. How is that not the creepiest shit ever?

Compare this to the Upwork App's description https://support.upwork.com/hc/en-us/articles/211064098-Log-T... .

It seems pretty reasonable. By default, it snaps a picture every 10 minutes. If you do not want to upload a snapshot, it removes the 10 minutes from your recorded hours. It captures info about how often you used keystrokes and clicked, but not what you typed or clicked.

The benefit of using the app is instant approval and payment.

On my reading, it looks like the author failed to take a 10 minute survey in 3 days, losing his work hours. That seems like a pretty reasonable ask that he failed on.

Yeah no. No kind of spyware is reasonable.

There's also an older post about the same topic: https://hackernoon.com/why-you-should-never-use-upwork-ever-...

This is very typical for Upwork. I have entirely stopped using their platform.

Elance was so much better

Looks like upwork ate them AND oDesk

Well, maybe a data scientist should do some research into a platform's terms before taking on a job there. And read up on a topic properly when writing a blog post.

Nobody should use Upwork, it's filled with scam artists and Upwork doesn't actually adequately block the scammers.

So hidden in the article is the real reason that data scientists might not want to use Upwork:

* They record your screen * Your screen contains data * GDPR.

s/Data scientists shouldn't/No one should/


s/Data scientists/people/

tl;dr OP accidentally left a survey unfinished for the client, this a dispute was made. OP completed survey, client was okay to pay, but OP hadn't logged hours with always-on webcam thus money was refunded to client.

Seems to me the client got the work for nearly free (gaming the system time only) - I see nothing in the article about UPWork requiring the client return work product as consequence of payment refusal.

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact