
The latest trend for tech interviews: Days of unpaid homework - draven
https://work.qz.com/1254663/job-interviews-for-programmers-now-often-come-with-days-of-unpaid-homework/
======
speby
Our industry's hiring practices are absolutely obnoxious. In an ideal world,
you could trust someone's resume. e.g. "I've written large production programs
in C." Ok, great, so then clearly this person knows C so there should be no
need to dissect code or run them thru some linked list algorithm and see if
they know how to use pointers correctly.

Yet, we do. Ok, then what about open source contributions (if they have them)?
Well, sure, but we don't really know, for sure, if that code is their own or
if they indeed truly wrote it, can we? So we can't count on it. It helps but
it can't be an automatic "this person passes the technical" signaler.

So maybe the person shares _actual_ code and walks us through it that they
wrote, even if not open source? Again, we can't know for certain it is their
own code and not some friend who wrote it for them 2 years ago that they claim
is their own. Or was a teammate's, or what-have-you. So we can't count on that
either.

Apart from getting a candidate to actually pump out some code, even trivially,
we don't really have a way to de-facto verify that the person says they can do
what they say they can do, short of a personal reference from someone, say, in
the company already that knows and has worked with the person in the past.

I absolutely hate this practice of having to constantly "re-prove" to the next
employer that someone knows how to write software. It's extremely redundant
and yet we keep on doing it, with no real hope of actually making it better.

~~~
monocasa
I used to hate it, but have now performed enough interviews of supposedly
senior devs with decades of experience.

I've literally seen someone who did firmware for the space shuttle grind for
45 minutes on fizz buzz without making progress. Like, I'd they struggle for
ten minutes or so, I take a step back and say "Ok, screw syntax, let's just
vaguely talk about what needs to happen and mock up some pseudo code.". Even
then, grinding for another half hour with no progress.

~~~
poulsbohemian
I’m going to show some vulnerability… this happened to me this week. I’m
probably like a lot of you here: professional software developer for 20 years,
have shipped countless products in multiple languages / technologies. Work has
appeared on tv and print media multiple times. Have generated millions and
millions in cost savings and revenue for employers and clients.

But I totally failed a technical screen.

It’s been a week of reflection as a result. I nearly canceled the interview to
begin with because the entire process felt like a cattle call - totally cold,
the company had shared no information about themselves or the role. There were
ten minutes of rapid fire (“you have one minute to answer this question.”)
followed by the proclamation that the next 45 minutes would be spent on “as
many coding problems as we can get through.”

Within the first few minutes I was able to describe a general solution to the
problem, but over the next 40 I totally struggled to produce a working
solution. Now, sitting here I could likely write it in 5 minutes. The problem
was:

\--I was completely frustrated by the experience from the beginning

\--I was in a code editor that I wasn’t familiar with

\--I had limited access to documentation

\--An interviewer looking over my shoulder harassing me

\--I realized that while the problem was simple, there were things I’d just
never needed in that particular language before, so didn’t have a complete
understanding of what was available to me.

\--I focus on design before implementation, where the interviewer just wanted
me to jump in and bang out code.

\--I realized that the last time I wrote this particular bit of code was 20+
years ago, as a college freshman, and certainly not in the language I was
using today.

So I agree with the other commenters: we should all check our premises before
we disregard another experienced programmer as as hopeless hack.

I wrote a long blog article this week about what I think the interview process
should be, will be posting it soon. There are ways to fix this process.

~~~
monocasa
So a little more on the interview. I've got a laptop setup with eclipse, all
rigged up with an integration test and filled out function signatures. I show
the interviewee what to press to compile, and the subsequent failing tests.
The code builds in it's initial state, but the tests fail. I also show them on
the desktop that the original source is squirreled away in case they fat
finger and erase everything (I've totally done that too, lol. Source control
is a beautiful thing in the real world). Then I peace out for a few minutes to
give them space (I know I'm close to an order of magnitude worse at typing
when someone is looking over my shoulder).

Given the negative response we've had in the past about take home assignments
for senior devs (their time is pretty valuable, so I try spend the same time
they do), I'm not sure what else to do to accommodate.

~~~
poulsbohemian
Todd, is that you? Sounds a lot like the process used by someone I worked with
years ago.

It sounds to me like you’ve done some careful thinking about this, and it
doesn’t sound unreasonable. One of my beefs really gets down to the interview
process turning into a one-way grilling, dehumanizing the candidate into a
“code monkey.” It sounds to me like you are trying really hard to evaluate
their technical skills, but in a way that is collaborative.

~~~
athenot
> dehumanizing the candidate into a “code monkey.”

Bingo.

A hiring process that attempts to convert the multidimensionality of humans to
a handful of numeric variables is essentially trying to hire the best drone
out there, not the best human fit for the job.

I learn more about candidates from the types of questions they ask me, and
their reply to open-ended questions I ask them, than from any technical
grilling test.

~~~
ConceptJunkie
A dev manager I work with insists that one of the best indicators, along with
all the usual things you try to suss out about a software development
candidate is whether or not he does any software development in his spare
time. Not everyone who's good does, but there's a good correlation between
people who would be good at writing software, and people who do it for fun.

So I would take writing software for fun as a plus, but would not take not
writing software for fun as a minus.

------
newobj
It's unreal that will people will bitch and moan about having to invest a
couple days, maybe even a whole week in cramming/prepping/interviewing, to
perform on-demand, one time, for a job that realistically will earn them
$100k, $200k, $300k, $400k+ a year. Meanwhile people that work harder than us
5 days a week for the entire duration of their career will make 1/2, 1/4,
1/6th, or less of income, with none of the perks, flexibility, mobility, etc.

I mean good lord. People are asking for an objective/verifiable measure of a
skill that's entirely in the domain of your supposed education and/or work
experience. How much more straightforward can things get? Would you prefer
that it be based on ... who you know in the company? How much you can bullshit
your way through a soft skills quiz? How much buddy-buddy rapport you can
establish with your interviewer?

These are objective measures on a set of borderline "well known" problems, ...
I can't imagine a simpler thing to prepare for. There is no guessing. The
amount of hoping someone just "likes" you is at a bare minimum. The impact of
resume fluffing is minimized.

I literally can't think of anything more fair or objective.

I can't even fathom people that aren't willing to make these miniscule one-
time efforts where there are literally buckets of money waiting on the other
side of the gate. Money that most non-tech people will never ever dream of
obtaining.

I'll take all my downvotes now, but I'm willing to karmically die on this
cross. :P

~~~
FLUX-YOU
Look at other fields for examples. Few have to interview like devs do. Other
professionals may have continuing education like conferences, but that's not
the same.

Imagine nurses or docs getting asked to take vitals for an interview.

~~~
trowawee
I mean, programmers also don't go through a medical school equivalent or
residency. We don't take boards. I'm married to a doctor and we were together
through her time in medical school and residency. If I had to choose between
skill checks in the interviewing process and the 8 grueling years of hell that
was med school and residency, I will take option 1 without a second of
consideration.

~~~
user5994461
There are developers who went through a master or PhD at a top school, with a
similar difficulty and debt to medical school.

There are companies who will only hire you if you got a degree and relevant
experience. The big tech companies never mention it but most of the workforce
had a long education.

~~~
newobj
I've never seen post-grad requirement for a dev unless it's a cutting edge
research position and they need PhD's from the problem domain.

Requiring a master's for a dev position is just a joke and I wouldn't take a
company seriously that was asking that.

Above not true for roles like PM, where for better or worse an MBA carries
weight.

~~~
user5994461
>>> I've never seen post-grad requirement for a dev unless it's a cutting edge
research position and they need PhD's from the problem domain.

I've seen it in almost every place I worked for in London. The floor is either
only people with degrees or only people with no degrees.

The degree is never announced as a requirement, it's rather selected
automatically during the interview. For instance in programming, some
questions about state machines and graph traversal will do a wonderful filter.

------
danielh
I don't get it. The whole industry is complaining about not being able to hire
developers, yet there seems to be some kind of competition who can come up
with the most awkward hiring process.

Requiring homework is a great way to deter the kind of people companies should
be most interested in hiring: those who are not actively looking for a new
job.

I find this trend especially surprising considering that the U.S. has at-will
employment where you can fire anyone anytime (at least that is my
understanding, I'm from Germany). Why not get people at a desk quickly and
evaluate them on the job?

~~~
abhinai
Just that firing someone is legal does not mean it is easy. Especially if you
have been working closely with the said person for months. And then there are
people who are very nice human beings but their performance just isn't good
enough. Firing them is hard and pretty bad for team morale.

I'd much rather do that extra work during the hiring process than have to deal
with a bad hire later.

~~~
EliRivers
Money where your mouth is; are you willing to pay for it? Would you pay the
candidate a reasonable sum, certainly over a hundred Euros a day (after tax
etc.) for their pre-interview homework? Given that the cost of a bad hire is
orders of magnitude more than that, this seems like a really good deal for the
company.

~~~
gaius
_are you willing to pay for it? Would you pay the candidate a reasonable sum_

This would be additional income complicating tax, and many companies restrict
employees from taking on outside paid work without approval.

No matter how you slice it, these multi-hour homeworks are a stupid idea. And
the real reason for them is to discourage candidates to justify getting
indentured labour instead. Or to discriminate against older workers with more
existing time commitments.

~~~
BrendanD
The homework is, in part, a personality test to see how much the company will
be able to take advantage of an applicant. This may or may not not be the
intent, but it is a result.

~~~
pnw_hazor
AKA hazing.

------
pwthornton
If you're going to expect people to do real work, you need to pay them.

This is common in a field like journalism. Want to do a one or two day trial
of a reporter? Pay them to write for you! Have them work in a real environment
and do real work for publication.

If you're not willing to pay people for their time, and it's not an executive
position, asking for days of a person's time is asinine. Also, if the homework
isn't really real world, I still don't think it's of value. Making interview
processes longer doesn't make them more rigorous. Only rigor makes a process
more rigorous.

What I tend to do is have people do contract work for me, and if they do good
work and people like working with them, I'll offer them a full-time position.
Another way that companies get at this is with internships. They are a low-
cost way to scout talent.

I find that a lot of companies are mistaking theater for rigor. If you ask
someone to do a coding assignment, and it's not the kind of work that your
company would actually do, how much value do you get out of it? Can someone
work with your APIs? Are they comfortable with your languages and systems?

I have also found that longer interview processes that have more people
involved introduce noise, not rigor. This is why I generally try to be
involved in the entire interview process for people I am hiring.

Want to work on my team? I'll screen the resumes, I'll do the phone interviews
and then I'll bring you in in person. I'll be in on all of the interviews you
have with various people. It's my job to hire great people to work for me --
not recruiters, HR or random people at the company. I involve HR at the end to
get a sense for a person's personality, not at the beginning.

As a manager, hiring is arguably my most important skill. A manager that can't
hire is going to have trouble building a good team that works well together.
So, getting the hiring process right is critical.

~~~
deepGem
Exactly. Just a week or two of contract work will tell you if the candidate is
a fit or not. A 30 minute phone screen will tell you if a candidate is worthy
of a contract. I think companies don't do this because a. They don't want to
pay and b. They don't get to gloat 'we have a super rigorous hiring process'

~~~
theptip
How many engineers would accept an offer which is conditional on "a week or
two of contract work"?

Most of the engineers that I've hired came from another job (not all though),
and so they need a confirmed offer before they are willing to jump ship.

So this practice introduces its own selection bias; you'd only be sampling
from the pool of currently un-/fun-employed, and those who hate their current
job enough to quit without a safe landing.

I'd vastly prefer to have a week or two of paid contract work from each
candidate before making the decision, but it feels insulting to even ask.

~~~
ksk
I wonder if having multiple ways you can hire would be beneficial here. Behind
Door number 1 is a confirmed job offer after 5 hours of grueling technical
interviews. Behind Door number 2 is a 30 day probation offer after an hour
long soft skills + tech interview, etc..

Maybe the lesson here is that the "optimal" strategy is to simply not abstract
the problem. Its not "a person" you're hiring, its Bob. And Bob is more
comfortable doing X or Y or Z, and Bob is best tested for his skills in ABC
manner, etc.

~~~
mifreewil
This is the most reasonable thing I’ve read in this thread. Totally makes
sense.

------
maxk42
This literally happened to me last week. On a Friday night before my
girlfriend's birthday they sent me -- without warning -- an assignment that
"should take about 8 hours". I told them to give the job to someone hungrier
than me which was the politest way of telling someone to stuff it I could
think of.

The company in question advertises here on HN's "Who's hiring?" threads, but
the role was brought to my attention by a recruiter.

Here's a little tip: If you expect to take more than 2 - 3 hours of someone's
time, you should be paying them. Contract-to-hire and trial contracts are very
common in the industry and most people have no problem with them. If you
require a significant time commitment to interview someone then you should be
respecting their time. I don't want to work for any company that doesn't
respect my time.

~~~
exodust
But you didn't say how long they gave you to do it. Could you hand in the
assignment a week later?

I've never been given a homework assignment for a job interview, but I would
love it. I absolutely hate the tests they give you on the spot, the last one I
did was awful and made me angry! I would much prefer an 8 hour homework
assignment. That pressure would be fun, and I get to work with my tools in my
comfort zone.

I spend 8 hours playing video games (over a few days), so putting that aside
for a challenge with possible good job at the end seems okay to me.

~~~
maxk42
They wanted it done by Monday. Then I got to choose which day of my weekend I
wanted to sacrifice: I chose none and I'll begin my new position at a company
that respects my time Monday.

To be clear: I've been given take-home "challenges" that take 2 - 3 hours
each. Including at the company I wound up going with. I have no problem with
these. If you expect me to sacrifice a full day of my time then you can pay me
or find someone more desperate than I am.

~~~
exodust
Okay, they wanted it by Monday, yeh that's rough.

------
iandanforth
I'll take a 3-day take-home project over a whiteboard interview or live coding
session any day. Working at my pace with my tools in my space? Yes please.

The article makes two good points re: bias and lack of research. To the first
I think it's up to us as interviewees to mention this in all of our calls. "Do
you have a non-take-home option? For some groups this could be an exceptional
burden."

To the second there's no real answer other than a lot of companies need to try
it out to see how they fare and then when they have a basic idea of logistics
that work run a study on efficacy / predictive power.

~~~
markbnj
Came here to say almost exactly this. I can't speak to the bias issue. I do
know that spending three days doing a little chat app last time around landed
me one of the best gigs I've had. The process was painless, I would have spent
that time coding and refreshing skills anyway, and the project was actually
fun. I also know 100% that if the company I work for encountered a candidate
who needed more time to work on the project due to child care or other
responsibilities that would be quickly and happily accommodated. I wouldn't
invest this kind of time as a first step, obviously, and would be turned off
by any company that expected me to. I undertook this project after two 30
minute conversations had convinced both parties that there was a fit, and the
project was a next step.

~~~
rhapsodic
_> I undertook this project after two 30 minute conversations had convinced
both parties that there was a fit, and the project was a next step._

In this case, I don't see it as unreasonable, because you didn't have to do
the project just to have a shot at someone talking to you in person.

I would never do a homework project as the _first_ step in an application
process. I would do one as the last step, after I had learned enough about the
job and the company to decide that I really wanted it, and they had narrowed
down the field to me and a handful of other candidates.

~~~
minimaxir
> I would never do a homework project as the first step in an application
> process. I would do one as the last step, after I had learned enough about
> the job and the company to decide that I really wanted it, and they had
> narrowed down the field to me and a handful of other candidates.

When I was job hunting last year, the steps typically went like this.

1) 30 min phone call

2) Homework

3) Algorithm Exam

4) Onsite

I typically got rejected at #3 which means the homework meant nothing.

~~~
rhapsodic
_> I typically got rejected at #3 which means the homework meant nothing._

It got you to step #3, but if I were applying to that company, I would not
have done the homework. When they told me of the homework assignment, I would
have asked, "assuming I do it, what comes next?" And when they replied "an
algorithm exam, followed by an onsite interview", my reply would have been,
"homework is the last step, or no thank you."

Then again, I have almost two decades of experience and a well-established
track record of accomplishments. And I currently have a solid, well-paying
job. If I were fresh out of school, trying to land my first programming job, I
would probably be more flexible.

~~~
minimaxir
In disclosure, all the companies were upfront about the full interview
process. That said, in this job market, there isn't much _leverage_ to say no
to a prospective job or to make demands of the interview process unless you're
already a rockstar.

~~~
rhapsodic
_> That said, in this job market, there isn't much leverage to say no to a
prospective job or to make demands of the interview process unless you're
already a rockstar._

I don't consider myself a rockstar, or even a prima donna type who would make
"demands" of the interview process. But I'm confident I could find a job
fairly quickly without jumping through ridiculous hoops just to get an initial
response from a company. From my perspective, the software developer market is
still a seller's market.

YMMV, of course. Maybe you're in an area where there are more qualified
developers than available jobs, and employers have the upper hand to the
extent they can pull off bullshit like this.

------
dosycorp
I don't want to name names but one YC company I interviewed at left me with
the distinct impression that they talked a big game, just to get me to work on
one of their (apparently) "strictly-hiring-related toy projects" as homework.
Homework, "sure", except it: 1) integrated with a real system, 2) mirrored
something they actually did, and 3) they would only agree to proceed if I
signed an IP/work contract and accepted their (sorry, too small) payment of
USD450 for the (not insignificant) project.

I pushed back at the contract, and asked them to ask their lawyer if that was
really necessary. When they insisted on a contract for the homework project I
pulled out of the process. It just felt so wrong.

The worst part, is not that they would seek some way to freelance a small part
of their system, nor that they would actually use ideas from hiring interviews
in their product -- but that (it seems) they would so blatantly lie to someone
they professed to want to hire that this was homework, when in fact it was
not. Even worse was the feeling of disrespect that they wanted to freelance
part of their system for USD450, trying to abuse the hiring process/ talking a
big game about offers, to unfairly lowball the price. Like they are a well
funded, established, largish YC alum, at least they can afford to pay market.
Anyway, everytime I see "XXXX is hiring a YYYY in ZZZZ" on here from them, I'm
reminded. Feels good to vent.

Just sad. Anyway, I'm not going to name names because this is unwise to do
that in a public forum.

Anyway, I only added the above anecdote after writing the following comment.

This is why Triplebyte is so good. You do a single technical interview ( after
a quick online quiz ), then get fast tracked to final interviews at various YC
/ silicon valley startups.

Sure, maybe not everyone wants to work for a smallish startup. But some of
their clients are 200+ people. So...it's not all the same.

~~~
collyw
I have noticed a trend of not naming companies here. Is it against the rules?
I fell that we should name and shame companies that are disrespectful of
peoples time.

~~~
gravyboat
I've had multiple comments detached (as you can see by my comment history) for
calling out companies that suck to apply for in the monthly hiring threads.
Either the people who posted must have complained so they removed it, or HN
doesn't want people to know some companies suck to deal with when it comes to
hiring process.

~~~
meesterdude
Don't let the HN censorship discourage you. Name them and shame them!

~~~
dosycorp
I'm still undecided about the best option for these situations, but I made
this ( simple ) site as an experimental database of people's hiring
experiences, and posted to HN. I'm letting you know because you cared about
this:
[https://news.ycombinator.com/item?id=16895043](https://news.ycombinator.com/item?id=16895043)

------
makeupsomething
As part of an interview I did recently I was asked to build a web app that
mimicked a part of their service and deploy it somewhere for them to view. I
probably spent about 15-20 hours on it in total and I believe I did more than
what the brief asked for only to be told I did not get the job with no
explanation, even after I asked if they had any feedback.

I thing is I don't think anybody even looked at what I made because to view
everything in the app required the user to login (part of their brief) but no
new users showed up on the site at any point. Pretty frustrating experience
overall.

~~~
arwhatever
I've resolved to always get the company to agree to give me feedback prior to
working on a coding project. No guarantee of feedback, and I walk.

------
i_dont_know_
Despite some of the comments here, I'm actually a fan of the 'homework'
format. I'm not always great at hiring, but I know how to gauge the 'homework'
interviews, and I've usually been right about the interviewee's abilities
afterwards, WAY moreso than whiteboarding or riddles or any of the other
alternatives.

That being said, I would add some constraints:

1\. The 'task' shouldn't take more than 30 min to understand, and no more than
1-2 hours to do. If it takes more than that, you should make some obfuscated
dummy-code endpoints that do everything you're not directly evaluating on.

2\. You should let the interviewee know that it should only take 2 hours as
well. If it takes more, then there was either a misunderstanding in the task,
or the interviewee doesn't have the right knowledge/experience/qualifications
etc.

3\. You should do _your_ homework as well, meaning you've thoroughly reviewed
the code submitted and have detailed questions ready to go for the
interviewee. This should also be the 'control' to make sure the interviewee
really did write the code (you'll know really quickly if they didn't)

4\. No other coding tasks in the interview process. You can have them _review_
code or suggest approaches to solve problems (things that people do on the job
with others)

5\. The interview should be shorter than if they were asked to do
whiteboarding or live coding or anything like that.

Most employers have (on paper) "trial periods" of around 3 months. A 1-2 hour
coding assignment followed by 1 hour of well-crafted questions about the
assignment (plus the usual interview stuff) should be enough to know whether
or not you want to give the employee a chance for 3 months.

(I realize most employers don't use the evaluation period for evaluative
tasks, they just have the person start and hope for the best, but if done
right, it's actually a pretty good system)

~~~
animal531
I'm going to disagree with you on a number of reasons.

1\. A lot of studies out there show that the most important factor for a
hiring success is culture, not skill.

2\. Are you even measuring skill with your test? It doesn't matter if it
should only take 2h for a candidate to complete your test, if they're nervous
or want to impress they're going to spend 12h on it. How can you tell the
difference?

3\. What about the really good candidate that doesn't want to spend 2h on your
test, because he also went to 2 other interviews in the same day where no
tests were demanded of him. You're again not testing for skill, but
willingness (and against other employer's testing regimes).

4\. What is your test about? As a random example implement Web Service X that
does Y. Is it important for you that candidates know how to implement X from
the ground up? Will they be doing a lot of Y? Why spend 2 hours effectively
answering 2 questions, where you could have interviewed him in 2 hours
covering his whole history of computing experience, what he loves about it,
why, what code he's written, what he's into, what he thinks about X,Y,Z etc.

5\. Your job is not to impress him with your knowledge (not that I'm saying
this is what you're doing, just a general remark on interviewers out there),
but to determine what his is, where he's strong and where not, and how the
person will fit into your structure. Making your test the big determining
factor is a failure.

6\. There's a reason fizzbuzz for example makes a great starter test. It
immediately weeds out people that can't complete it, and allows you to see the
interviewee reason through it. He goes for example, well, I take this and
that, then I have to do this; oh yeah and so so etc. It allows you to see a
lot more than just the end answer. After that you can apply more
interesting/harder questions, but focus rather on how they do it than if they
can complete it in the allotted 5 minutes.

~~~
cimmanom
I find code review-style discussion of a simple coding challenge to be great
for assessing both culture fit and skill.

What we administer is like a web app equivalent to FizzBuzz in terms of
difficulty level. But unlike a whiteboard FizzBuzz, it doesn't put the
developer on the spot. As an extra bonus, unlike FizzBuzz, you'll get
different results from the kid who just successfully finished a CS degree and
the grizzled veteran who's built and maintained complex systems for decades.

The right challenge can be completed in an hour or so by a mid-level developer
in a brute-force manner. It will weed out those who simply can't code (or
don't grok how the web works or how apps are constructed); and also highlight
candidates who are capable of more sophisticated work. If being up to speed on
a specific technology from Day 1 isn't important, you can give the developer
their choice of framework or language or whatever.

Then when you do a code review in person you can get a feel for both how the
developer approaches code (a skill) -- but more importantly, how they take
critique. Do they get defensive? Do they riff off your ideas? Do they suggest
improvements/refactorings they would have made if they had more time?

You can also explore other cultural factors - for instance, how did they
approach constraint trade-offs in terms of time vs. completeness vs.
sophistication? How clearly are they able to communicate about technical
concepts?

~~~
hamidhasan
This actually sounds really interesting, putting some code up for one
interview or take home assignment, and then asking them to walk through a code
review. You could also throw refactoring into the mix, ask them to actually
refactor it if needed.

It also got me thinking: what if you just had some generic code, which was low
quality, or a prototype front end app, and then asked them to refactor or code
review it? That could spark some interesting discussion and be less stressful,
but may be easily bsed I think.

~~~
cimmanom
Yeah, that is an interesting idea. Might be harder to come up with, though.

------
tptacek
I believe the big problem people have with take-home work is trust. Nobody
wants to do a take-home assignment that doesn't really impact their prospects
of getting a job, or, worse, that has only downside for them. Hiring teams
give assignments and then run a standard interview gauntlet. On both sides,
most people believe it's the interviewers making the real decisions, without
much consultation to the tests.

The candidate thinks, what's the point? It's a seller's market. I'll work
somewhere that doesn't haze candidates.

We run work-sample challenges now, and (loudly) did so for about a decade at
Matasano. We had essentially no complaints once we locked our process in. Part
of the reason why, I think, is that we tried hard to be very clear about our
process. But more importantly, the work-sample challenges _made most of the
decision_. When we told candidates the time they spent on the challenge offset
time they'd have to spend in person performing for an interviewer, we weren't
lying. A 4-5 out of 5 average on our challenges more or less totally
technically qualified a candidate; the odds were overwhelming that such a
candidate would get an offer from us, _and we told them so_ before they came
in for interviews.

What I think I see now is hiring teams who want to have things both ways. They
don't want to take a leap of faith with a new process, so they're still
abdicating their responsibility to random subjective interviewers. But they
want to look on-trend and competent, so they assign homework.

When you get on the phone with one of these teams and want to check them out,
ask about the rubric for the challenge, and ask if they're going to do any on-
site technical qualification after you do them (if they're giving challenges
after the interviews instead of before, hang up the phone on them.) I'd be
surprised if even half the companies that assign homework even have a rubric.

~~~
Mc91
> Nobody wants to do a take-home assignment that doesn't really impact their
> prospects of getting a job

I spent a few days on one, sent it in and was told that day that they had made
an offer to someone who accepted the offer. Then I did another one over the
course of a few days which was looked at, but a new VP came in and eliminated
the opening.

It's similar to asking me to come down for an interview for a few hours during
the work day (and then to come back, because one person I had to talk to
wasn't there). Or before I've even talked to anyone, they hand me a form
asking for three references, preferably (or exclusively) former managers, and
their current phone numbers.

I don't mind them asking or me giving these things if they are near a decision
about me, but almost every short phone screen I have nowadays is followed by a
request for a work sample that in reality would take a couple of days to do
(between my schedule, and wanting to send it in near-perfect).

Something I think is preferable is for them to ask me for code I wrote. Then I
can write something once and be done with it.

As others on the thread have said, in a tight market, companies put up huge
new barriers to getting a job with them like several days worth of free work,
then bemoan the fact they can't hire qualified (young, SF local) people who
"care about their mission" of selling advertising junk from one corporation to
another corporation.

~~~
itronitron
I provided some sample code recently, it was experimental, rolled-at-home
'research' code that was, I think, in line with the mission of the team I had
applied to, complete with test data and instructions. Needless to say, the
interviewer didn't want to talk about what the code did but instead grilled me
on various technical language details. In some ways it is a great litmus test,
because the people I have enjoyed working with in the past would have had
endless questions for me about what the code was actually doing...

------
donretag
One big issue that a take home assignment is that it removes the ability for
the candidate to ask questions. Interviews are two-way streets.

Nowadays, I reject more companies than I interview for based on an initial
phone conversation. I have requirements in what I look for in a company and
position, and some simple questions help me resolve many of the primary
requirements. Given a take home exam, I will have invested hours of my time,
before given a chance to screen the company.

Just say no. End the madness.

EDIT: I should add that when I mention a phone conversation, I meant with
someone on the dev team, not HR. A take home exam after only an HR phone
screen is not acceptable.

~~~
cozicoolmail
At my current job (where i've interviewed candidates) we give out a take home
assignment, but we explicitly enforce a 2 hour time limit to do it. We felt
this was reasonable because combined with an onsite interview and a phone
screen, the total amount of time a candidate would spend interviewing for a
position is about 8 hours (a typical workday). I'm not against take home
assignments per se - but 3 day assignments seems crazy. I can't imagine myself
doing that as long as there are other decent jobs out there that don't require
it.

~~~
jasonvm
I think this is the best approach. You get the opportunity to do exactly what
the job expects, without the pressure of having to manage communication with
someone breathing down your neck. A savvy employer can extrapolate what you're
able to produce over 2 hours to whatever project they'd have you work on.

~~~
cozicoolmail
We've had pretty good luck with it thus far. Generally, it is a good way to
chop off the long tail of "not good enough" and bring the better ones onsite.

As you said, we calibrated our general expectation to what we think is
reasonable in 2 hours - so we expect "good enough" code, but not perfect code.

------
mazelife
A while back our company switched to the "homework" format for evaluating
software engineering proficiency because we felt like it took a lot of stress
off of candidates to write code in an unfamiliar environment (or worse, on a
whiteboard). And I feel like it's worked out really well. But we have pretty
strict criteria for any homework problem: it needs to be a simplified/toy
version of something we'd actually ask the person to do if hired, and it needs
to be something that a qualified candidate could easily complete in an hour.

The code itself is, of course, valuable for helping us to evaluate, but
honestly some of the best signals we get come from the open-ended discussion
that results from us talking with the candidate about their code when they are
done: "imagine this file you're processing is 100TB instead of 100kb? Would
you change anything?" or "What if this clean data you've been given to work
with had this or that issue, what might you do?"

My only point of disagreement with this article is that it seems to assume
that a homework problem has to be really time-consuming or burdensome, but
there's no reason that has to be the case. To me employers who are asking
candidates to spend hours or days on homework are of a piece with ones who ask
people to solve tricky, unrepresentative computer science problems on a
whiteboard: they reveal that they don't really know how to interview people
and they don't have much respect for their candidates.

------
gshulegaard
Interesting, I actually prefer take home challenges to the previous trend of
whiteboard interviews. And within the realm of take home challenges I prefer
ones without time limits which usually implies they are full or multi day
assignments.

Even with rehearsal I am not a great public speaker and impromptu formats
certainly don't make me better. I also prefer take home assignments since it
is more closely related to how I actually do work: Get a problem, think about
the requirements, do some research and design a solution, implement.

Since I personally prefer this style of interviews, I have given assignments
like this. Usually a simple microservice that is given AWS t2.micro size
constraints and no time limit for a response. When I get a submission I am
looking at code style, tests, and documentation as well as solution quality to
get a better picture of how a candidate approaches tasks and gauge how they
work independently. It's amazing how much you can tell about a single person
from a challenge like this.

I also think it is more flexible since there are no strict pass/fail criteria.
A solution in C/C++ that optimizes heavily but skimps on documentation could
be just as impressive as someone who uses Python to stitch together multiple
tools with more features and more thorough documentation.

Interviewing is difficult...for both interviewee and interviewer. Finding the
right compromise between depth for the interviewer and work required of the
interviewee is a challenge. I am sure there are take home challenges that are
ridiculous, but I wouldn't dismiss all take home assignments.

~~~
default-kramer
With no time limit, aren't you just optimizing for the candidate with the most
free time? (Granted, some people are more efficient than others, but you
probably only want to hire from the top band of efficiency anyways. Meaning
that "time spent" becomes the primary deciding factor.)

~~~
gshulegaard
This is definitely something to keep in mind, but we keep the requirements for
the microservice extremely light for this reason (2 API endpoints). A solution
submitted in a week with many extra features can be just as appealing as a
solution submitted in a day with the bare minimum. It's not primarily about
how much work is put in or how fast it is done; rather it is about the quality
of what is included.

------
0x445442
If the software development industry was the home building industry...

I'm a general contractor (Unicorn Homes Inc.) and I'm currently building a
number of spec homes. We're going to need electrical work done in all the
homes and I'm in the process of sub-contracting out that work. What I need
from you, Joe Electrician, is for you to go wire the kitchen of one of these
homes. Once you're done I will inspect your work and let you know if I'm
interested in hiring you for more work.

~~~
Epenthesis
I've never seen a take home coding challenge that had even the possibility of
being used as production code. For me it's always been a standardized, self-
contained example problem. Maybe there're scummy companies that try to push
job applicants' work to production, but I'd be shocked if that was anywhere
close to the modal case.

So it's more like "What I need from you, Joe Electrician, is for you to go
wire up this demo kitchen frame that I've set up. Once you're done I will
inspect your work and let you know if I'm interested in hiring you to wire up
the actual houses I'm building.".

~~~
Bartweiss
I've definitely seen some prompts that looked suspiciously production-ready.

It's usually not "build out our new feature", because spinup time is a thing,
but some companies do seem to throw around "build our new one-off
microservice". Stuff like "given this incoming data, set up intake, a storage
layer, and a backend layer that applies formula X". If the data and the
formula look business-relevant, it's at least enough to raise an eyebrow.

That said, I'm not sure how much I care, provided the interview is sincere and
not some kind of bizarre scam to farm out work. My main concern is much more
with scale. I totally see the merit of replacing 5 hours of whiteboard
interviews with 5 hours of "build a toy project at home". I don't so much
appreciate "take a week of your life and build a substantial project so we
don't have to do any legwork".

~~~
gowld
Wow the pendulum has swung from "employers don't ask any relevant questions"
to "I mistrust employers who ask relevant questions".

~~~
Bartweiss
I should maybe have gone into more detail on that example. If BigCompany or
even AcceleratorCompany asks for product-relevant work, I'm not going to
assume they're having random people off the street write production code.

What I've seen is something like "this is a startup in market X, whose product
doesn't do Y but could, and they're asking candidates to write code that does
Y, and sign over all rights to everything provided during the interview
process".

Optimistically, it means that if whoever they hire will be told "hey, first
assignment is Y, you've already made progress!" Pessimistically, it goes in
the same category as "Whartonite seeks codemonkey" and "sign this NDA to
interview" \- it's a matter of ego and ambition exceeding skill.

------
Grollicus
I don't understand this development. In Germany, we've basically got full
employment in the IT sector. Why would anyone waste their time with this? Just
apply for the Job next door.

As an employer, hiring someone new is always a leap of faith. Might work out,
might not. If it doesn't, dont feel bad to fire people again.

~~~
eksemplar
In Denmark we put a job offer out on the various job banks, receive the
applicants, go through them in the hiring team, starring them 1-5 and we call
the best 5 candidates in for a 30-60 minute interview.

Mainly it’s just a talk to see what personality is the best fit. Sometimes we
do a personality test, but only for people who will do any form of project
management. We’ve never done a technical test or had anyone program for us, I
know some companies do it for juniors that are fresh out of school, but
otherwise it’s faily uncommon.

That’s it really.

I’ve hired around 20 developers like this over the years, none have failed me.
One junior wasn’t really worth his salt at first, but he learned quickly.

When I read about these American hiring practices I wonder why anyone would
want to work there. I wonder if it’s because a lot of American developers are
autodidact? I mean, I’ve never interviewed someone who wasn’t educated in some
form of CS, academy level and up.

~~~
walshemj
That's just a bit elitist :-)

------
skate22
I gave a resume to IBM at a career fair.

They made me do coding problems on the spot right there. Given an integer n,
write a function to print ever digit backwards. It took me a minute to think
about it, but i solved it, and evan ran it at home to make sure.

Several months later they ask me to do an online 3 hour interview. I did it,
there was no human interaction whatsoever. I had to record video responses and
solve code problems.

More than a month later with no follow up, i got an automated response to take
another 3 hour interview with no human interaction.

I did not take the interview and will never re apply to IBM

------
mike-cardwell
Reminds me of an "interview" I had last year with a secure email company that
gets mentioned around here a lot. I was given a coding task to do. Think it
took around about a full work day spread over a few days. Wrote some code.
Submitted it. Got an email a few weeks later to say they'd hired somebody
else. No conversation about the work I'd done or the position it's self
whatsoever. My only communication with a human was basically somebody telling
me they were going to be sending me some work.

They originally sent me the work over Skype. I don't generally log into it
Skype but I did in order to accomodate the way they wanted to chat (send the
work). A few weeks after they told me they'd hired somebody else I logged into
Skype again to find a message asking how to access the work I'd done from
several weeks earlier. Even though I'd provided a link to it earlier on in the
conversation. To this day I don't know if anyone even looked at the work that
I spent a full work day doing.

Perhaps they just didn't like the work that I did. I wouldn't blame them, I
wasn't particularly proud of it. But I had a lot to say about the work, the
problems I had and the things I would do differently if I were to start it
again. I wasn't given the courtesy of expressing those thoughts though.

The funny thing is, they approached _me_ about the position in the first
place.

I wont be participating in any "interviews" structured anything like that ever
again.

------
azinman2
I actually prefer take home. I’m not great at white board solutions as they’re
basically asking you to solve something immediately with a gun to your head in
real time. It’s very unnatural. Take home gives me time to think about
something more like I would job.

That said there needs to be a balance where the tasks are enough of a test to
give indication of quality of work without taking 3 full time days to finish.

~~~
erichurkman
Whiteboard is cruel for actual coding. It's too far removed from your normal
tools. Whiteboarding can be good for architectural design or stepping through
something.

A sixty minute screen share is the sweet spot for me. They get to use whatever
environment they are familiar with. Seeing someone work in their environment
also tells you potentially a lot about the candidate.

Staff engineer that can't navigate their debugger?

Senior engineer that codes by copying every line off of stack overflow and
mashing the keyboard until the lines somehow work?

Bootcamp graduate that builds test cases in Excel?

Junior engineer that writes code in notepad.txt and pastes it into a Python
shell?

All things that I've seen happen that tell a different story than a whiteboard
or a 3 hour submitted homework test.

~~~
azinman2
That’s interesting. How does one build test cases in excel?

~~~
erichurkman
It was a combinatorial sum-of-n problem. Given a sequence of integers, can you
produce a target by summing N of the values. He used Excel to put the expected
return value in column A, the target in B, and C+ were values. Saved as a CSV
which his program ingested. It was neat, actually: the rationale he gave was
that it made it easy for anyone to add test cases.

(The candidate was previously in the accounting biz, so his navigation of
Excel to do this was wizardlike to produce a few dozen test cases in about a
minute.)

~~~
azinman2
Sounds like you appreciated his solution, but your first post made it sound
like he was incompetent. I’m confused.

~~~
erichurkman
Not incompetent. Doing it via screen share lets you see their approach to
problems — sometimes those approaches are poor. Had it been a take home test,
I would have never seen him do that, and never had the opportunity to discuss
the approach.

------
poulsbohemian
I've noticed an interesting trend in my career: the more grueling the
interview, the less-satisfying the job. The jobs I enjoyed the most, had the
most interesting challenges, etc, were the ones where it was little more than
a handshake. The ones where they put me through hoop after hoop turned out to
be the ones that were the least interesting and had the least upside.

~~~
the-pigeon
On the flip-side I've worked with engineers that were so entitled it
completely undermined their ability to work with others in the company.

Work is going to involve some bullshit. That's life. If you can't jump through
a few hoops for a job then you don't want the job very badly or have an ego
problem.

Don't get me wrong, expecting someone to spend three days on a technical
solution is nonsense. But the "special snowflake rockstar" mentality of some
developers is not helpful and important to check for during an interview
process.

------
xfz
It's not new - I did my first assignment around 18 years ago - and it's not
necessarily a bad thing if done right.

One problem is that assignments take too long and have over-optimistic
estimates, which apart from taking up time can lead to feelings of self-doubt.
Feedback on the assignments is lacking to no-existent, and this is an
unforgivable sin in my book. Also, anything that looks more like a
contribution to their own software than an academic test is a red flag.

The other options noted in the article are also pretty bad. Whiteboards and
fibonacci numbers, forget it. "Pair programming" in an interview is deceptive,
nothing like normal pairing; it's a test (fair enough, but don't call it
pairing). It's also problematic for me because I have a problem with my
eyesight and any "accommodation" requested up front tends to be inadequate,
leaving me at a disadvantage.

For people with busy lives or family commitments, a short "interview prep
homework" of 30-90 minutes would help the person to prepare, in their own
time, in a relatively less pressured environment. The interviewer could then
expand on this during the interview.

I've done assignments like this where it proved a useful opportunity to brush
up on a particular language (if cross-training or coming back from
management), sometimes even enjoyed it. Also, it means recruiting is less of a
numbers game - you might interview with a handful of companies instead of a
dozen, choosing the ones you really care about.

IMHO what we're seeing is a terrible implementation of a reasonable idea.

------
gelo
I went for a job interview with a company years ago just after I graduated.
This was for a graduate position so I kinda made a few assumptive expectations
on what they would want from me and my skills.

Went in there, did their pointless psychometric tests (which by the way
litterally have no relevance to my skills) went through their interview and
went out their door passing it with flying colours.

I got a call from the agent leasing with them. She was shocked to say that
neither me or another they interviewed made the cut.

I graduated with a first class degree with honours in Software Engineering.

The company said I wasn't qualified enough for the graduate position!

Told a good friend and lecturer at my university about this and he said, "pass
me the name of the company, i'll get them barred from exposing internships and
job oportunities to other students. Your time was utterly wasted and that was
a completely pathetic reason to not accept you".

The agent said to me that she would fight to testify my qualifications. I told
her that there was no point. This company is not only wasting my time but your
time as well. They dont deserve your time as an agent nor my skill set.

~~~
the-pigeon
Performing well at a job takes a lot more than just technical skills.

Social skills and having a compatible personality with others on the team is
very important to team success too.

May have been something else but you saying "did their pointless psychometric
tests (which by the way litterally have no relevance to my skills)" makes me
suspect you failed for non-technical reasons.

------
stratigos
Ive been grinding away at these for months... coming out of freelancing,
latest contract with a startup that hit its burn rate and failed, and having
beeen unemployed for almost 6 months (except not really unemployed, since
contractors dont actually collect unemployment...).

So many companies have these assignments. Theyre infinitely better than
whiteboarding, or hacker rank tests, or "pair programming" on some crazy
abstraction for only 30 minutes. Some of them are so much work - I am working
on my 20th hour of a really involved full stack project, and still think I
have about 6-8 more hours before its "pretty".

Then I have 2 more to do, and I told the HR people at those companies Id do
them 3 days ago.

One weekend I spent 12 hours doing a SOA example project, and on Monday, the
owner simply deleted the upstream - and my fork along with it - and said
nothing back to me. I hadnt even turned it in yet!

Hours, weeks... its been months of this. Im working for free and its just
energy that goes into thin air.

Not once has anyone actually done a real code review, I suppose they just get
buried in hundreds of these little projects to review and select someone and
just never get back to you :-/

Next week I have about 6 interviews lined up. Cant wait to see how many unpaid
hours of work I get to do!

...oh, and I have > 12 years of industry experience, have built multiple
applications that garner 50mm requests per month (I can almost guarantee you,
that youve been on at least 2 of my websites before), have worked with IoT,
HIPPA medical, ecommerce, greenfielded, been on big teams, worked
internationally, have buttloads of experience, and every person Ive ever
worked for praised and lauded me plenty. Yet I cant get a fucking job for the
life of me because of these paint-by-number processes.

Yah anyway, back to the grind, for free...

~~~
meesterdude
Man, thank you for writing this. I am totally in a similar boat - but at it
for 3x as long. It looks like you are far more agreeable than me when it comes
to these assignments.

Consider that for most jobs, there is naturally a 98-99% rejection rate. Only
one position and 300 candidates. So assume that most coding exercises you do
will go nowhere, will not be reviewed, and will leave you feeling
disappointed.

You are a thought-worker, and an obviously talented one. The most important
thing you can do is to stay nimble and don't get discouraged or bitter. Part
of that means saying "no" to unreasonable requests. You don't have to full out
say no - you can also just negotiate down the scope to something digestible.

Identify your limit and push back should anyone attempt to cross it. For me,
if it's a good company and the test is reasonable, i'll throw in 2 hours. If
it's a BS test, I'll push back or skip entirely.

These companies hiring with absurd processes do not know what they're doing -
and will walk all over you/us if given the opportunity.

------
dep_b
I had to do homework assignments twice. Once was for TopTal, which kind of
made it worth at the end since I've got a few years of work out if it already
and they're supposed to check their candidates thorough.

The other one was for Automattic that seemed to be intentionally vague to
check if I would spot some pitfalls and smells they left behind. Never heard
back. Mailed the guy that I had contact with twice. Not a fucking word. Thank
you for wasting my time.

I don't think I'll ever do it for free again unless it's a position for
Godmaster of Cooltech at AwesomeCo.

------
ldite
I've only been asked for this when I applied to a "digital design agency"
(i.e. web development) during a moment of madness.

They asked me to write a GAE based web app for something or the other. I told
them (politely!) that I'd do it, if they provided me with their diversity
policy, specifically with an explanation of how this sort of exercise didn't
discriminate against anyone (as noted in the article.) I got a "sure, we can
do that" and then never heard from them again. I didn't consider that a loss.

~~~
hnzix
_> "digital design agency" ... during a moment of madness_

Yeah you dodged a bullet there. Digital design agencies are almost always
sweatshops.

------
toblender
I spent days almost 10 hours, completing a massive interview project for one
company. I submitted a fully working solution, had unit test, was pixel
perfect, and they didn't even give me an interview.

Not only that they gave me a vague reason, only after pressing for feedback.

Left a very bad taste in my mouth, don't think I'll be doing code assignments
again, what a waste of time.

~~~
DenisM
Would you still do it if you were paid for your time? Say $500 for this 10
hour project.

~~~
toblender
That's a tough one... I guess it would make it better, but I'm not sure about
how the taxes and relationship with the company would play out...

------
pfarnsworth
I think the best way to hire is to hire optimistically, without forcing
ridiculous trivia questions or memorizing algorithms.

Then give up to 3 months of mentorship. If the person doesn’t look like they
meet expectations, then fire her quickly. Give them the remaining time up to 3
months as severance.

This is the best way to do it because you're not subjecting people to bad
interviews, and you're giving them up to 3 months severance if it doesn't work
out.

------
jt2190
Focusing on the “what to do instead”... There's an interesting suggestion at
the end:

> Holiday’s company [CallRail, Atlanta, GA] now uses a structured “live-coding
> interview” format that involves reviewing code together and talking about
> it. The process typically takes between 30 minutes to an hour.

I wish there were more details, like whose hands are on the keyboard, or
whether the interviewer has a rubric to structure their evaluation.

------
everdev
As a developer and a manger, there are many candidates who fake or bolster a
resume, especially in tech. Additionally, there are some developers who are
incredibly smart, but take 2-3x the time as someone who is not as senior to do
the same task.

The only way I've found to reliably hire is a coding test. I think it's
ethically irresponsible to choose actual client work for the task. But it is
important to create a test that is similar and with a time constraint,
especially if you work on any projects with deadlines so you can simulate how
they work under a time crunch.

~~~
mentos
What about paying someone to do client work as a test? That is what I've found
to be the best approach for me. It gives the candidate a very concrete
understanding of the kind of work they'll be doing as opposed to a more
contrived example. It comes at the cost of paying for poor work but usually
some portion of it is salvageable.

~~~
everdev
Yes, another option. I guess my primary concern is sharing client IP with a
non-employee.

My tests took about 10min to write and 1-2hrs to complete and then we could do
an apples to apples comparison across candidates.

My concern about client work would be the time it would take to bring someone
up to speed as well.

------
deepGem
I wrote a couple of these home works and finally gave up. In fact, I won't
interview for any dev positions that require me to do a recruiter phone
screen, another technical phone screen, a homework and then an onsite coding
test. Expecting production level code in a homework is just asinine.

I'd much rather do a timed coding test for 30 to 60 minutes and be done with
it.

------
Bokanovsky
I've had to do a lot of telephone interviews and then face to face interviews.
I'm regularly surprised how many developers can't do basic stuff. I'm not
talking code that requires specific knowledge of an obscure area of an API or
language, but things like picking the smallest values from an array (without
using a language's built in array min function).

At my current company we get them to write code (or something like pseudo
code) down on paper. We give them a print out that clearly explains the task,
which requires no domain knowledge and then leave them to it for an amount of
time to write stuff down on paper. We then go back in the room and get them to
talk us through their solution. But so many developers just can't do this.

Then some can, and they can't explain the edge cases, what if the array is
empty? What if it's null? What if it's all the same value? They'll just
freeze. They would have past the phone screen first to get this far.

We're not looking for more advanced stuff with red–black trees or an exotic
data structure. Just a demonstration of what you've written and simple
understanding.

In reality, this is an invitation to a conversation. Why did you do it that
way? Is there another way to do it? Which way do you prefer? Why? All if this
is basic first principle stuff.

Generally when you get to the good candidates you know. They can answer all
these questions and you go in different directions with the questions, they're
comfortable doing it too. You can get a technical conversation going. They
won't lie and I say they can do X, the good candidates will quickly admit they
haven't done X or Y, but their overall impression means not knowing X doesn't
matter. The trouble is I've had so many candidates who are on the wrong end of
the Dunning–Kruger effect.

[Minor grammar edits]

------
newshorts
Not going to name the company but I topped 38 hours over a weekend to land a
job at a tech company. It was brutal.

But I still prefer that to those stupid algo questions.

At a different company, I literally had an entire dev team sit on the opposite
side of a table in a board room and play stump the programmer. After the
interview I left hating the company regardless of the job offer.

------
jaimebuelta
This is nuts and doesn't make sense.

Testing coding abilities should only be done in the context of finding out if
I can trust the resume (which should be relatively minimal), or if the
candidate is looking for their first job (again, minimal as they won't have
experience).

The rest should be possible to analyse TALKING to the candidate or doing some
join design session...

Good luck making people spending more than 4 hours in a selection process if
you're not Google (and even so)... Seems like a great way of massively
discarding anyone that's busy...

------
mixmastamyk
Prefer homework to coderpad nonsense.

When a recent position asked for a 10-12 hour homework, said I cap them at 2-4
hours and quoted my daily rate. ;-)

------
matchagaucho
Isn't the obvious answer to simply pay candidates $50 hr?

This is still far cheaper than the costs of hiring the wrong candidate.

------
kjhkhhhhhb
I suspect any kind of process will disadvantage somebody. For example the on
site pair coding lauded in the article my disadvantage people with social
anxiety or who can better focus alone, and traveling also takes a lot of time.

That said, I would also reject homework assignments, as long as I can afford
it. If they want it, companies should pay for it.

Multistage interviews are also a big turn off. Imagine going on a date, and
your date saying "I just need to try yet another date to see if I like you"
multiple times.

------
ben_jones
I mean we can't have it both ways. You either hire with heavy credentialism or
cronyism or you hire based on whatever trivial markers you can come up with
for instant validation. Companies are trying tons of different validation
markers, from white boarding, take home, stuff like triplebyte.

The bottom line is you can't instantly validate how someone will perform over
the course of a year or more. It takes time. But nobody wants to spend time
so...

------
IshKebab
I think assignments are a really good way to test people, but they really do
have to be short. The second-last job I got took around 4 hours which I think
is fine. Helps if the task is a bit fun too.

Also it shouldn't be the _first_ stage of the interview process. That's just
taking a piss. For this job it was after a phone interview, and with a
scheduled face-to-face interview (where we discussed the code I'd written).

~~~
GeorgeSarkis
Getting a (new) job in IT is an exercise in futility if you don't know someone
inside, so essentially there's not an interview but an invitation.

It's lower-status, more degrading and has worser odds than cold-call
telemarketers trying to sell you some new credit card.

By the way, when a telemarketer tricks you into answering his call, you don't
subject him to a 3-days intensive rectal exam ("homework") just for the remote
possibility of getting a new credit card, which at the end you still don't buy
anyways on some made-up pretense.

So developers agree to way worse level of abuse and degradation than the
lowliest sales guy.

~~~
LeozMaxwell
Also there's one and only one defense with respect to abuse either at
interviews or after you get the job: having a large stash of money, which
allow you to say NO to abuse.

At least 5 years runway, entirely on your own "payroll". If you're actually
competent, that's more than enough time to pick any skill (possibly starting
your own business rather than look for a job using it), if you can do 12 hours
per day _by your own choosing_ on whatever you see fit.

Don't have those 5 years but only 1-2 or, god forbid, have nothing more than a
few months on top of a large amount of credit you have to pay back.... and
you're in the 99% percent of desperate and destitute crowd who will submit to
anything in exchange for a bowl of soup (guarantee they never break out of
slavery).

Problem with the solution is that it's a vicious circle: if you don't already
have those 5 years of savings, chances are you're never going to save them
anyways.

------
DoubleGlazing
I've been experience this more and more lately. Some employers make their own
tests, some rip off those from big players like Facebook/IBM/Google and some
are just blatant attempts to get some work done for free.

There is a lot of talk on HN about the best way to do IT recruitment, but
whatever method is close it really shouldn't take more than a few hours of
face-to-face contact and some legitimate tests. With HR doing due diligence by
checking references etc after an offer is made.

The first time I encountered this I was told that I would have to do a 8-12
hour coding challenger over the weekend. They wanted me to come up with a way
of proving someone has watched a training video without skipping or faking an
API call - something they admitted in their interview that their system was
lacking i.e. free work for them. I told the recruiter that I would not be
doing any such thing. I have young children and my weekend time is precious.
If you can't validate my skills from my a few interviews, my detailed CV, plus
HR doing due-diligence, then it makes me wonder about the organisational
culture of your company.

------
MattyRad
To be the devil's advocate here....

There is such a low barrier to entry for our industry that virtually anyone
can can call themselves a software developer after a week of coding exercises,
compared to other engineering disciplines which have a very high barrier
(mostly in requiring a degree and often difficult certification like the PE).
What's worse is that even computer science majors aren't guaranteed to be an
effective hire. This is NOT a new problem[1].

Ultimately I would argue that the risk is higher, for a company, of hiring an
incapable engineer, than it is for an applicant being hired (and paid) as an
incapable engineer.

Requiring an applicant to demonstrate capability is smart. Where we draw the
line for how much needs to be demonstrated is what is being questioned here.

[1] [https://blog.codinghorror.com/why-cant-programmers-
program/](https://blog.codinghorror.com/why-cant-programmers-program/)

------
anon1094
I actually gave a link to my Stripe account and told the company I was
applying to to pay me hourly as a contractor for doing this work. In the end,
they took it offensively and denied me.

------
zaidf
I wish companies would figure out a way to have their best engineers do blind-
shopper-style interviews where they regularly see if their best engineers
would be able to make it through their _current_ interview process. This can
be tricky to implement but if done right, would provide lots of insight into
the types of candidates their interview process is geared to pass.

I asked a friend who worked for years at a large public company if he would
have passed the company's interview process at the time that he left. His
answer: "most certainly not!". I think his answer is astonishing. Here is a
guy who is considered a top notch engineer at his company, assigned to help
the company screen for similar top notch engineers and yet, the established
process would specifically _fail_ a good percentage of high-quality engineers
of the type that already work at the company.

~~~
sudosteph
That reminds me. At Amazon there was a concept called "raising the bar" that
was applied during interviews and promotions. The idea was that you should
only hire or promote people with the capacity to do the job better than 50% of
the people already doing it.

I often wondered how sustainable that was. Will be interesting to see if they
keep that strategy long-term.

~~~
stevenwoo
Combine that with stack ranking for evaluations and firing the bottom 10%,
though I think they quit doing that recently.

------
anilgulecha
What you want from a hiring process is to be as objective as possible. For
fresh engineers, the industry has settled on basic coding right from fizz-buzz
to reverse a linked list and beyond.

For more senior roles, majority still rely on pedigree (previous employer,
resume etc), and those who try to be more objective hand out a take-home which
ends up frustrating both sides. Candidates have to spend a lot of time and
effort and the hiring manager/devs have to setup/run/review.

This is where the industry needs to make the process easy, and operationally
objective and efficient for the initial screen: An hour or 2 spent by
candidate on a take home problem, which can be reviewed in under 5-10 minutes
by the hiring manager. (This disparity in time is because the ratio of
interviewed to screened is roughly that.)

(Disclosure: I work at hackerrank, and build the take-home equivalent of a
coding test.)

~~~
organicDragon
An interesting thing happened to me when I applied for an internship with your
employer. I got a call, saying that I would have an interview few days days
later even gave me the exact time. Come the day of interview, I got no call.
And no response to follow up emails/calls. So, when somebody from hackerrank
comments on the hiring process, it is a bit vexing.

------
yosito
As a senior level engineer who's currently job searching, this trend is really
burning me out right now. I'm at the point in my career where I don't want to
work for just any random company (I've already worked for NASA and done client
work for Young Money, Carrie Underwood, UFC and more). All of the positions
I'm applying for are highly competitive. My resume performs well; I get
interviews at 70-80% of the companies I apply to. And I usually have 3-4
interviews that go well. Then I get to the final stage, unpaid take home
projects that last anywhere from a few hours to a week. Many of those go well
too; I get feedback from companies saying they're really impressed by my work
and that they'll talk to their team and get back to me with an offer. And then
a few days later I get an email saying that they think I'm a really talented
candidate but that they're moving forward with someone else. Over the last
month, I've been working 50 hour weeks, unpaid, on code that's looked at once
and then never seen again. It's really demotivating and exhausting.

I think there are a couple of solutions to this; 1) Instead of unpaid sample
projects, companies could offer paid trial periods where you work on real
problems and get paid, with the option to continue as a full time employee
after the trial. 2) Instead of doing tens of sample projects or exercises, one
for each company you apply to, you could be evaluated one time by a third
party and that evaluation could substitute for technical interviews done by
the companies themselves. It would save companies the cost of evaluating
candidates projects, and it would save candidates the cost of proving
themselves time and time again. The evaluation could potentially be handled by
different entities; tech recruiters, certification bodies, or standardized
tests. This may be what companies like HackerRank are attempting, but they
don't really seem to follow through with job offers.

As it is, I'm trying to find a solution to my own job search. If anyone has
any recommendations on how to find a job without burning out, I'd love to hear
them.

------
NTDF9
Can someone explain why we need: 1\. A phone coding interview (Potentially
another phone coding interview)

2\. A homework

3\. An onsite with 5 rounds

Seriously, what does that additional tree traversal round prove?

~~~
RickS
I don't necessarily buy fully into all of these (I am in favor of homeworks,
but they should be paid and later staged [this usually means more phone
screens. can't win em all]). But since you asked:

1) because meaningfully large swaths of applicants are too stupid to fizzbuzz

2) because passing fizzbuzz is not the same as writing software

3) because teams are comprised of many people in many roles and just because
team/person X likes you, doesn't mean you'll be a good aggregate fit.

~~~
NTDF9
I will respond:

1) I don't get fizzbuzz in first coding rounds anymore. It's usually graphs

2) Ok. I write software. Your whole team can go take a look at my software.
Done coding?

3) Ok, so the last on-site round should be meet and greet then? Because I
already submitted my code before. You guys all should have looked at the code
and agreed to bring me on-site and just meet me. Why ask me to traverse trees
again?

------
jasonvm
I was recently on the job market and decided to apply to TripleByte as a way
to hopefully skip some of the annoyances. The process involves a short online
quiz, and then a 2-hour live-coding session with one of their engineers. It's
supposed to demonstrate to potential employers that this candidate is actually
skilled at coding.

The problem were the employers, who just treated TripleByte approval as an
add-on, instead of a substitute.

One employer said I'd also have to do a take-home assignment (which was, I kid
you not, _recreate React_ ) and then a 4-5 hour onsite. A few weeks later,
that employer contacted me to see if I had completed the assignment. I told
him I accepted another job. (Of course I didn't even bother starting the
assignment)

------
PeterStuer
The main reason 'take home' assignments are BS: It will take the qualified
person that needs to evaluate the submitted result more time to go through the
code you produced and make a basic judgement, than it would take that same
person sitting in a room together with the candidate to produce a far better
judgement of the skills by just talking with the candidate if front of an IDE.

Truth is many of these 'assignments' will never be looked at by a person that
can actually derive good insights from the submitted work. Most of the
superficial treatment they will receive could have been established in a five
minute phone interview.

------
systematical
Sounds like it might be the same company the author is referring to. I'll
throw them under the bus. Hello Fresh interviewed me and that interview went
fine. They then wanted me to deliver an application using no framework. I was
allowed to use libraries, but no framework.

After reviewing the requirements I quickly determined that it would take me
days to build an application that I felt would effectively WOW them. I told
that to the recruiter and bowed out. Being already employed I looked at the
opportunity cost and it was too high. Sure they paid more and are a bigger
company, but that is A LOT of work. I rather do almost anything than unpaid
programming on an application that will be thrown away in a few days.

If I were hiring, the "lab" portion of the interview would be simple. For mid
to senior level developer:

1\. Write a small inefficient application that requires some indexes, SQL
optimization, and some code optimization. Very, very small. Very, very easy
for a reasonable developer to fix. Afterwards I'd have a conversation about
other general optimization things like Redis, Beanstalkd etc... because we'll
optimization is important, especially to me.

2\. A small class with very poor cyclomatic complexity. Clean it up. Because
clean code is important.

3\. A small insecure application. Things like not using bind parameters in
queries, not cleaning inputs, XSS etc.. Secure it.

4\. Show me something you've done. Personal or professional. I would then ask
questions and it would be an open conversation. Because I want to see how much
you enjoy programming and how we'll you can speak about it.

I think these 3 items, combined with the other interview questions, their
experience, and resume would give me enough to go off. Ideally, 1, 2 & 3 can
be done in under an hour. I think that is reasonable for pre-screaning an
applicant. I certainly wouldn't send me running, but I am also writing the
test so who knows...

Why would I want to look through 10 or so applicants full fledged application?
God no. I have better things to do. I guess for some companies its a way of
weeding people out, but these large tests are akin to using DDT to remove
weeds.

------
dilap
I worked for a company that did a 3-day, challenging take-home problem. It was
a big ask, but it was amazing for getting a sense of people's problem solving
and coding abilities. (The final result would only be 1000 lines of code or
so, but 3 days provided time to think and experiment with different
approaches.)

Maybe paying people for the time if you end up saying "no" is a nice
compromise?

If you are potentially serious about hiring the candidate and they are serious
about potentially working for you, doing some "homework" does not seem like a
major amount of effort in the grand scheme of things to me.

~~~
mieseratte
I've met folks, recruiters and (tech) leadership, at companies that do these
type of long, arduous interview processes.

They make it a point of ego / pride that they have such a long, arduous
process. They've even mentioned the occasional candidate who asks about
billable hours for an 8+ hour code problem, saying "Who the hell does that guy
think he is?" Of course, they were shocked and appalled when I asked who they
thought they were to ask someone to spend 8+ hours of their time on an code
test before even having so much as a phone screen to determine compatibility.

As it were, this was a company in the contracting space so I assume this is an
intentional filter to weed out all but the most compliant individuals. If
you're not willing to bend over backwards, they're not interested.

~~~
bg4
What a great way for me to filter them out of consideration.

------
peteretep
In the UK I see a lot of tests that say "don't spend more than 2 hours on
this", and that I'd generally expect to take 3-5 hours tops. Americans seem to
have a strange employee / employer dynamic

------
stuaxo
I refuse outright to do this kind of thing... mostly, it's because I fail
badly ! - I do fine in real jobs, but when it comes to tests and homework, I
tend to not get the job. Also... I could be applying for other jobs in that
time or doing.. you know, anything else - lying in bed doing nothing for
instance.

Seriously, this trend can go fuck itself... the last time an recruiter told me
I had to write 100 words about 12 different skills I told her no, and hung up
- only to send her a ranty email about how stupid the idea is.

------
tinix
Latest trend? Haven't we all gone through this before? Back in 2012 I did a
"homework assignment" too, just to get an interview with a game studio. The
task? Build a SOAP client and server that do some silly little thing, I don't
even remember what. Basically, just something to prove competency. It took
about an hour, mostly time I spent reading the manual for the programming
language they requested. Trivial shit... WTF are people being assigned that
takes multiple days? Ridiculous...

------
reefoctopus
I had this happen recently. Not only did they want me to do unpaid work as
part of the interview, but that work was directly related to their business.
They requested that I do this before they had even spoken to me. I responded
that I was unwilling to do it in exchange for a chance at an interview. I was
an excellent candidate, and I’d done years of work on similar systems that I
could point to. This sort of thing is a huge red flag that the employer
doesn’t respect the time of those who work for them.

------
radicalbyte
This practice is a massive warning sign. I've only seen it applied by
companies who lacked the technical competency to actually interview a
candidate in depth. If the tech lead / architect is competent then he/she can
just a candidate from the CV and an hour of chatting.

The two worst developers I've ever had in my team both aced those interviews.
How? Their colleagues did most of the work for them. It worked out great for
me, it cleared out the dead wood and punished some faker at another company :)

------
slezakattack
I did a take home assignment a couple years ago where it was time constrained
to 2 hours. At first, the idea sounded cool: nobody looking over your
shoulder, I can listen to my music, use my tools, and work in my most
productive environment. Selfishly, it can be nice (assuming that you didn't
just have a long day of work and aren't dirt tired).

I'm very quickly realizing it wasn't that great and in fact, comes off as lazy
from the interviewer perspective. There are several "coding interview
websites" where you can ask the question, give the candidate a terminal to
code, provide a list of available languages, and even have unit tests from the
interviewer side to make sure the solution is correct. We've completely
abstracted out the interaction and questioning of candidates and it has become
a binary "did they get it right or not" type of process. Sure, there's follow
up interviews and such but what have you gained from this process that
couldn't be asked over the phone?

Worse yet, if you aren't using these tools and simply letting the candidate
email you a project (or however they choose to deliver it to you), it can
quickly become a time sink as you spend it trying to understand the solution
(i.e. why did they interpret the meaning of "API" as a REST service? What is
this obscure library they're using? etc.)

No thanks.

------
booleandilemma
I say this every time this sort of article comes up here, but seeing how
ridiculous this field has become, I wish I had chosen a different career path.

I don’t know if I’m going to be able to go through all this bullshit the next
time I’m looking for a job.

I have a friend who is a pharmacist and I asked her what a typical interview
is like for her. She said interviewers will typically ask something like “tell
me about a difficult situation you had to deal with at work?”.

No homework, whiteboards, group interviews, or stupid puzzles. No bullshit.

~~~
orangecat
_She said interviewers will typically ask something like “tell me about a
difficult situation you had to deal with at work?”._

That's just another form of bullshit. I'd much rather write a for loop than
try to figure out if the interviewer wants an honest answer or a humblebrag.

------
vinceguidry
Obviously if you don't have a job, do whatever you need to in order to get
one. But code challenges are making it increasingly difficult to move forward
in your career without subjecting yourself to inordinate risk.

I've been going back and forth on how best to handle this newest trend. I
don't think it's a good approach to just refuse to do them. But accepting the
challenge and then not following through is even worse. So a mutual
understanding of the deliverables and level of effort is required, otherwise
job hunting approaches second job status.

My current stance is to first negotiate the code challenge until it's fully in
your wheelhouse. If you are unable to do this, politely decline to do the code
challenge, citing time constraints. One job wanted me to do a basic web app
wholly in vanilla Javascript with no libraries. That's a challenge I should
have just declined.

Second, refuse to accept time limitations. No company that's not paying you
should be allowed to monopolize your time. One job had a hard 20 day limit and
the task was both outside my wheelhouse and expected to make a measurable
delta in their production application. I _definitely_ should have declined
that one.

But if a company is willing to work with you rather than subject you to an
inhumanly-inflexible meat grinder, that's the one I want to work for.

------
mindhash
Given that interviews now a days are not one way street.

Candidate is also evaluating whether company is worth working for.

if you give a take home assignment and don't give me a feedback on stuff I
could do better, i would consider the whole thing a waste of time. So if you
don't offer that's ok. But let the guy learn a thing or two.

Many people will agree that yc interview process is rigorous but it's also
satisfying as you get to learn a lot through out.

------
TomAAnderson
The whiteboard interview is coding's version of being asked to do a freestyle
rap. It's largely orthogonal to creating good code, but it's also a trainable
skill that arguably does show _something_ , even if that something is just a
willingness to play the game. Whether it's worth seriously training for is
another matter.

Cue the music (sung to the tune of Eminem's 8 Mile Road, see
[https://bit.ly/2iI8tG2](https://bit.ly/2iI8tG2)) and the scene of a nerd
sweating in front of the whiteboard as the interviewers nod their heads to the
beat:

Sometimes I just feel like, quittin', I still might Why do I put up this
fight, why do I still write (code)?

Sometimes I just hate life, something ain't right Hit the brake lights, case
of the stage fright Drawing a blank like

It ain't my fault, great big eyeballs My insides crawl, FizzBuzz I scrawl And
I clam up, I just slam shut

Gonna travel this road Gonna show 'em my code My Big-O is so dope the universe
will implode (as our hero proudly writes = O(1/n^2) on the board)

etc. Someone can make a viral video; I'm too busy practicing code problems.

------
DataWorker
I think the tech interview process is an area ripe for workers’ rights
legislation.

~~~
jpttsn
Might make it more expensive to hire, favoring insiders who are employed at
the expense of outsiders. Also makes it harder to switch jobs if you’re
unhappy/deserve better.

~~~
amorphid
I'd wager that insiders are given preferential treatment in most companies
anyway.

~~~
vidarh
Some places, yes. But surprisingly many places do a really bad job at
encouraging internal recruitment, even when it would often let them identify
talent they are unaware of and let them shift people into positions where they
might be a better fit or be stay for longer. It's especially bad as it can
often help shift more of their external hiring to lower risk entry level hires
where a bad hiring choice has a lot less impact.

------
jillesvangurp
I tend to regard interviews as an opportunity to determine if the company
doing the interviewing is worth working for.

People like me are highly employable/contractable. So, the way an interview
works is very simple: you got lucky enough for me to take note of the fact
that you are looking for somebody with my skill-set and I've determined that
you are worth talking to. Fantastic, because I ignore all HR & recruiter spam
so you probably found me through some mutual contact that I trust. Great, now
we meet and my expectation is for you to decide to hire me on the spot and be
extremely enthusiastic about it and get in a mode where you are selling the
job to me as something that I want. We both know that nobody else similarly
qualified is just going to walk in and take the job. If so anyway, my advice
is to hire them on the spot. No hard feelings. So, very simple choice. Either
we like each other or we move on. You don't get to ponder your decision for
very long. The words "home work exercise" typically don't feature in such a
conversation.

------
plinkplink
How many of these homework assignments become actual code that companies use?
If you ran a business and your programmers were all busy but you wanted some
new widget, what is preventing you from posting a job and tricking 100+
programmers into bootstrap your project for free?

A friend recently spent a week building a chat server with a very specific
telnet/ssh interface as a "code test" for a video game company. I helped him
test it and it was really solid. He walked the interviewer through the setup
and how everything worked and then never heard back from them. I suspect that
they simply used his code in their next project. What's stopping them from
handing over his code to the next round of applicants with the assignment of
adding all the features they need? They could post a high salary and offer
great benefits, but never actually hire anyone.

We, as developers, simply can't be giving away free work. We're all aware of
how opportunistic business folks can be. I refuse to do "homework assignments"
(unpaid work) and so should you.

------
thegigaraptor
I've been on both sides of this, having completed and reviewed interview
tasks, and I have to say it's absolutely crucial.

As an interviewee, it gives you a glimpse of what work will be like once you
come on board. Are the requirements solid or loose? Is testing required or
even mentioned. These and other things can give you insight into how the
business interacts with engineering.

As an interviewer it's hard to take resumes and unfortunately even GitHub
profiles at face value. I've personally had code taken from work put up on a
colleagues personal GitHub. The coding task is also not a sure-fire way, as
I've seen those taken from online tutorials. The real magic is in the tasks
code review.

I'm not saying this is how it works everywhere, but given the task matches the
deliverables asked and we make it to the code review stage, it becomes less of
a tech interview and more of a glimpse into the social skills of the
developer. PR criticism is a daily occurrence and getting an opportunity to
test these waters prior to fully committing to onboarding a developer.

------
fxj
Are there any web sites where you can read what each of the companies do for
their recruiting process? At least you would know what to expect...

~~~
collyw
Glassdoor has a section on interviews. I always leave a negative one when I
have put up this this crap.

------
monksy
Things I've had to do lately for the "homework":

\- Write a game engine

\- Answer 5 programming problems in the span of 2h20m

\- Do 3 coding exercises over the span of 12 hours. (That was pretty brutal ..
as that you have to build everything up yourself)

\- Write a REST service and business logic

\- Write a rules engine with depending rules and type.

Worst part of all of this:

When you're getting evaluated you're up for being nitpicked over spelling
mistakes in the documentation.

------
adamredwoods
While we're on the subject, I am tired of taking coding tests or homework,
then getting ZERO FEEDBACK on the coding "test" I spent 2 hours (sometimes
much more) on.

If I am going to get rejected, then have the professional courtesy to give me
constructive feedback on the code. In the future, I will ask for this up front
before taking any more tests.

------
imglorp
Hiring story.

It looked like it might be a devops position with a smallish family video
site. After an NDA (?!!) and some emails, no interview, they set me up with an
obscure chat client with a guy for a tryout. He had me set up some services on
a new instance, apache, php, firewall, etc, following a checklist they gave.
He wouldn't answer any questions about the company. I did this quickly and
that was the end of it. In the end, they said no thanks, but did not give any
feedback. They paid me for a day of work on Elance :-/ $50 for a couple hours
of work.

The bad news is the whole process was super shrouded and I to this day do not
know why. The NDA forbade revealing any contacts, who the site was, or
anything else. Nobody was willing to give me their name or answer any
questions. I guess the vanilla family site could have been a sister/front for
a gambling or pr0n or offshore hedgefund company, or ... what? Why bother?

------
uhhhhmmmwhat
This absolute trend of garbage behavior has happened to me for a while now.
That mixed with this so called _absolute inability_ for employers to provide
feedback because their so called policies do not let them. It is absolutely
ridiculous how someone in a position of power is treating others who merely
are trying to be part of the ingroup and get a job. This ingroup/outgroup "do
these steps, and you will join us" mentality is complete shit. Especially on
the part of these so called HR departments which claim they are having such a
hard time finding right candidates. Before this jumping through all these
hoops crap was invented, companies were hiring fine, people were working fine.

Then these geniuses decided to profit and "flatten the world" as they call it.
Just so they can make a buck.
[https://www.hackerrank.com/aboutus](https://www.hackerrank.com/aboutus)

The only thing that is going to get you is a flat brain. Unable to think
creatively, unable to respond to uncertainty but very very good at memorizing
solutions to problems.

And all this is based on this ignorant viewpoint that you can observe
something or a product for a short period of time and somehow ascertain
anything which you can use to form a useful opinion of a candidates future
performance. Bullshit!!!

No HR department should be allowed to ask any candidate to do any work without
compensation. It is work since the HR department is using this information to
make their decision therefore protecting or enhancing their companies
performance. I don't know how this is not related work to a companies success.

I am sick of it. I have started to send templated emails in response to them
which explain i do not work for free. I can share personal code from past
projects or my github.

A few months ago a PR representative even called me once to make sure i was
not going to sue when part of the application process was a take home
assignment which was asking me to implement a new feature for their actual
product. What a sham.

When i said to her "You mean to tell me that over the past years when you
claimed that this process has been the same, no interviewer has derived any
benefit from free work of past candidates." She didn't even know how to
respond.

The more candidates that do not put up with this behavior, the faster it will
stop and hopefully move onto something more agreeable by both parties.

------
shoguning
Work sample tests have been researched extensively in applied psychology [1].
There is a healthy debate on how effective they are.

For me personally, I am not so much for or against 'homework', as would rather
see it used correctly. For example, I want to know how it will be evaluated,
no tricks. I also think there is a quickly diminishing return on the length of
the assignment. Up to ~10 hours may be OK, beyond that seems like a waste.

On the other hand, it's also a way to see who's serious. If you are serious
about getting a job, it may take some investment. Also, the most glamorous
companies can always demand a higher investment from applicants.

1\.
[https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=work...](https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=work+sample+test+validity&btnG=&oq=work+sample)

------
DataDisciple
Its not just devs that get these assignments. I've seen startups use BD folks
for interviews to research and structure potential deals as a "homework"
assignment. They dangle a fancy job in hopes of generating leads, which they
do, and then they never end up filling the position. Candidates put in weeks
of work setting up an actual deal for a fictitious job.

I think these tactics are a lazy way to interview. If you truly want to build
a great company, you need to spend significant time with someone to understand
1) can they do this job, and 2) are they a fit with the culture, and 3) are
they passionate about the mission. There are no shortcuts, yet so many
companies try to hack the process.

Everyone is busy and looking for ways to steal time but this seems to be one
of the most short-sighted. The best companies don't look at labor as just an
input.

------
andyidsinga
My response - and even how I got jobs early in my career as a developer was
this:

"let me know show you some of the software I've developed on my own - I'd me
happy to tell you how it works, the tech involved and the patterns used ..yada
yada"

Corollary : if you apply for a job or walk into a tech interview and can't
talk about these things, its a red flag for the interviewer.

PS. Now well on in my career, and having started my own software business
after leaving a bigco. I'm back to a very similar thing: if I go to a meetup /
conference / sales call I better have some code I'm will to show and discuss
..and highlight what I consider my talents. (fwiw : it feels great to be back
in this mode)

edit: also : asking an candidate to do the kind of homework/coding is total
BS. I wouldn't do it. But I would give them the line above :)

~~~
h1srf
I have been developing software for 12 years. I can't show you anything that
I've worked on because it has all been internal tools. I can talk about what
I've built, but I can't show it to you or show you any code.

~~~
exDM69
Unfortunately that puts you at a disadvantage and is a small downside of your
previous job. Many professions these days require a portfolio to show, and
programming is quickly becoming one of them (and it's better for your career
if you can accumulate it while working). I guess your options are either
making a portfolio weekend project or submitting to interview homework or
other coding test.

------
bryanrasmussen
I have had the 'normal' 3 hour assignments, but recently I had one that was
timed - they gave you 1 day to do it. I got at 10 at night, I thought I would
have time but the kids had tired me out and I could not stay up 1-2 hours
getting environment set up that I needed.

So then actually I could not do anything about it until 5 in the afternoon the
next day, I had problems with environment setup (hadn't needed this particular
stack on my new machine so I hadn't set up and things had changed) so it took
3 hours. then I had to do dinner with the family, get kids to sleep. I didn't
have the time available in my weekend to do a 3-4 hour assignment that also
required 3 hours to get ready to work.

I sent off an email to the recruiter and said what happened and I wasn't
really interested in anything similar in the future.

------
Dowwie
What do folks here think about an interview assignment where an applicant is
asked to address an existing issue from an open source project, presented as a
variety of cherry picked issues from popular projects but selected by the
employer? The code would then be discussed during interview.

------
hellbanner
I just got one of these for a mobile position. "Please complete this coding
interview. It should take about 3 hours." Me: "I've been writing mobile apps
since the app store launched in 2008. Can we skip this step?" Corp: "Sorry,
good luck on your search!"

~~~
cableshaft
> I've been writing mobile apps since the app store launched in 2008

That's your main mistake, admitting that. I'm sure they were looking for
someone with five more years of experience than that.

You should have been writing apps in a text editor waiting for the app store
to come into existence, you know. I'm sure they picked someone with at least
that much experience.

------
lmilcin
I work as a contractor which means I change projects fairly frequently.

In the past, I would sometimes accept the homework assignment. Now I refuse to
do those (as opposed timed assignments where you get set amount of time, for
example 2h, in which you are supposed to complete the assignment).

If the homework deadline is few days you can practically be sure that there
will be candidates that will spend all that time perfecting their solution.
What is supposed to take "2-4 hours of effort" in the words of the interviewer
is actually the entire duration from the moment your competitors got their
assignments until the deadline.

There are other reasons to refuse:

\- you can spend that time working to find other offers,

\- the interview is supposed to be both ways, they want to learn who you are
but you are also supposed to learn who they are. Home assignments give
practically nil information about your prospective employer.

\- there are other ways to test the candidate with probably same or better
results. Choosing a way which minimizes the effort on the part of employer but
maximizes the effort of candidate shows lack of respect for the candidate.

\- Home assignments are invitation for cheating. I can assume there are
candidates that will take advice from others on how to solve the problems they
are given so this is hardly proof of their ability. I refuse to take a test
where I will be punished for my honesty.

\- Even though the assignments tend to be time consuming they are typically
hardly challenging. This is typically some mundane application and not really
a test of your ability. My ability is to solve complex problems, organize
complex logic, produce extremely reliable or efficient applications, draw from
wealth of my experience to propose and discuss solutions. Yet another Pet shop
written in technology of employer's choice is hardly any test. If you want to
know if I can program at all you can learn it in few seconds of conversation
with me, you don't need to force me to do 2 days of programming for this.

------
jigarkb
I worked for a company where I devised a 3 hour assignment given to candidates
(full-stack developers) during on-site round. They were free to bring their
own machine and work on their choice of stack. They were free to use any web
resources available. I even broke down the assignment into small steps to
follow.

Assignment was to develop an application which tested the candidate's skills
to access web api, browse through documentation, do proper authentication,
parse api results and finally the most important skill to design an algorithm
to compute desired output depending on the question asked. It was doable
within 3 hrs for experienced engineer. Not many succeeded in this task but
those who were able to do it were real good developers.

------
brandonmenc
When I first heard about this trend, I balked.

But then I realized that the occasional two days of unpaid homework is still
way better than being required to have a college degree or professional
certification - unlike nearly every other profession that pays over $100k.

I'll take it. For now.

------
naskwo
My dad was first concertmaster of the VSO, and my wife is a pianist. What's
interesting is to look at the parallels between auditioning for a position in
a prime orchestra and "auditioning" for the position of software
engineer/developer.

When you get called to audition, you spend a ton of time preparing at in your
"free time".

If you want to apply as a (senior) sw engineer, it makes sense to have an idea
about the depth of knowledge and/or the engineering approach of the candidate.

Oldie but goodie: [https://blog.codinghorror.com/why-cant-programmers-
program/](https://blog.codinghorror.com/why-cant-programmers-program/)

~~~
stevenwoo
The other side of that coin is many orchestras now do auditions blind - nobody
evaluating the artist can see the artist, they can only hear them. The numbers
indicate that this eliminates most of the bias orchestras traditionally had
for visual attractiveness/friends of the orchestra/white males.

------
sb8244
For me, I think that unpaid homework is an acceptable, if not important, part
of an interview. However, it must pass some qualifications:

* It can't be taken as real work at all. Something like building a game is clearly not "real" for a SaaS company and still conveys lots of value

* It should be discussed in person as the in-person technical interview (don't come in an whiteboard in addition to the outside work)

* It is provided only after a phone screen and it is discussed on the screen.

* There are clear objectives and "bonuses"

* The language of choice can be used

With this, I think that it provides value to the interview process and helps
the interviewee as well by simplifying the in-person technical components of
the interview.

------
ovulator
I've had this happen once. I applied for a standard Rails dev position. After
the first interview they wanted me build some sort of standard CRUD app in
Rails on my own time at home with specific requirements. I already had a full
time job, as well as a family, so even though there was nothing difficult
about it, I told them I probably wouldn't have time. They gave me extra time,
I said okay. A few days before it was "due" I told them I wouldn't have time.
They said they understood, and asked when I could come in for the next
interview. I guess it was just optional homework?

------
xab9
For my previous job I spent around a week to build a simple microservice from
scratch, with login, jwt, email notification etc. - it's still not a full app
of course, but it was very very time consuming (it was a bit out of my field),
considering that I did not spend that time with family, hobby et cetera.

I know noone forces me to do it, but back then I really had high hopes for
that job. Nowdays the maximum I'm willing to spend with an entry app is two to
four days but deep in my heart I still find it a bit too much, but I still
prefer this over live coding sessions in front of a bunch of random devs.

~~~
gaius
Where do you get 2-4 days to work on an entirely speculative project?

~~~
xab9
Weekends, plus you may want to take a day off. Otherwise chip it off from your
evenings. 7pm to 11pm through a week does the job.

~~~
gaius
These are unreasonable asks for just an interview. A read of their CV and a
30-minute phone screen, maybe two with different people, is all that is needed
to determine if a candidate is worth bringing onsite, unless the interviewer
and their company is utterly incompetent.

~~~
xab9
I share your opinion Gaius, but the recruiters seem to think otherwise. In the
last year I've been to seven interviews (by all standards I am considered a
senior dev with an appealing pedigree) and this is not something unexpected at
all.

The worst is that when they ask you to take three or five days off and work
with the future team to see how you fit in (not for free though). I'm not too
enthusiastic about it, but I already did it twice.

~~~
gaius
Can I ask which geography you are in? I am in England and that would be
excessive here. I think the most I have experienced is 3 phone screens and two
full but non-contiguous day’s of on-sites.

~~~
xab9
Hungary, but I had two non-hungarian interview processes as well, one for
Berlin and another for a multinational company (for the Hungarian office
though, but six interviews out of seven was held in English), same experience.

The latter one had a fairly complex entry task which took me three full days
and I had six interviews after that - during the seventh I gave up and told
them we are not looking for each other here.

~~~
gaius
I think a company that did that here would have a very hard time getting
candidates... at least for now

------
zaidf
My favorite interview anti-pattern: a position that stresses analytical
decision-making, and then an interviewer that pushes you to give a quick, off-
the-cuff answer to “do you think we should build X or not?”.

------
bewe42
The best hiring process in my opinion:

\- select your candidates that maintain a public profile. You should get a
broad idea about the skills and interest of that person

\- have an informal interview to see if they fit your company culture and
values

\- then have them work on a small real project for less money

I'm a dev and happy to work on such a "taster" project. It's a win-win: I'm
not working for free, the company gets a realistic picture of how I work and
how I fit into the company.

This works naturally best for hiring contractors, but even then I don't
understand why not more companies do this.

------
longsigh
As a potential employee, I am just not going to do this. Nor am I going to
complete puzzles (and I LIKE puzzles) or dance around in front of a
whiteboard. What I would be willing to do is work as a well paid contractor
for 1-3 months to make sure that we are a good fit for each other. I'm not
sure how anything other than actually working together could suffice. I am
fortunate enough not be concerned about immediate benefits like medical due to
my wife's employment--I recognize that this might not be a good option for
everyone.

------
pleasecalllater
I'm waiting for recruiting doctors like that. Oh, you are a surgeon with 17
years of experience? Cool. How about you make a surgery at home, record it,
and send us a video?

More pathetic are only companies which have only one question at the beginning
(without any further contact) "how much do you want to earn" and after that
there is no negotiation. Sometimes I feel like I'm drowning in an endless sea
of candidates. Oh, wait, no, the same companies publish the same job ads over
and over for years. So maybe not.

~~~
dagw
_I 'm waiting for recruiting doctors like that._

Would you want to work in a world where all programmers have to be trained,
licensed and regulated with the same rigor as surgeons?

Surely the fact that programming is one of the few 'white collar' jobs where
someone with no education and no relevant work experience can get hired based
purely on the fact that they've taught themselves the necessary skills and can
do the job is a net good thing.

~~~
LandR
> Would you want to work in a world where all programmers have to be trained,
> licensed and regulated with the same rigor as surgeons?

Uh, yes...

There are too many terrible developers out there.

~~~
natalyarostova
The ability to build a top tier career from self-teaching is remarkable. It
would be such a tragedy if this was stripped and replaced with a signalling
based equilibrium run and/or structured by the government :(

~~~
CamperBob2
Very true. These people literally don't know what they're talking about, or
what they're asking for. They've thought their plan through about five seconds
into the future.

Either that, or they're blatant featherbedders who aren't bothering to
disclose their interests.

In neither case do they belong on this site.

------
meaydinli
Anybody know the names of the companies that do homework/project type
interviews? I am looking around as an Android dev, and I would prefer those
over whiteboard/coderpad.

------
recharged95
Yes, I had that offered too, a 6hr Xp assigment with a selected employee.

But intentionally blew off the 2nd quiz (thus the interview) before starting
the assignment due to a family medical emergency (I thought, WTF am I doing,
there's more to life!)

Nowadays I find it's either: a. multiple quizzes or HW assigment b. a
presentation (that takes 4-8hrs to make anyway). c. and that github was a
source of code eval, not anymore! ...cause everyone forks everyone's stuff
(that's ok, by design).

------
hitekker
I once interviewed with a startup that disguised their technical-onboarding as
the last step of the interview process.

Learn three different frameworks, unlisted in the job description, and extend
a github repo, suspiciously similar to their production codebase.

That they insisted what was easily a 10-hour assignment could be done in 30
minutes "if you're already familiar", made it seem like they were offloading
the learning-on-the-job to before-getting-an-offer.

Cunning, but ultimately a sign not to sign.

------
ospohngellert
I dislike tech interviews as much as the next guy, but let me pitch my two
cents from the startup world.

The company I currently work at has very few employees, and each new hire is a
big cost. If a new employee does not work out, that can be a major dent in the
budget. We still haven't figured out a great way to interview, and we are open
to new ideas, as we have had hires who looked great on resume/interview who
did not end up working out.

------
cornholio
3 days of work is certainly an excessive commitment. But limiting the verbal
interview to one hour then leaving the candidate with a a computer and a well
defined problem that takes half a day to solve it's not unreasonable, in my
opinion.

They have already committed that day to this interview, there is no abuse -
show us you can solve problems and not give up after a couple of Google
searches, like a good half of programmer-aspirants do.

------
xivzgrev
Yea I think live exercises are the way to go. Or hire person as contractor for
a day or two. One small startup did that with me and I thought it was
brilliant.

------
brentonator
We had someone come in and spend 45 minutes on interview problems e.g. produce
an array of 100 to 1. You had free access to Google and were encouraged to use
them for reference.

After each attempt we said no that's not right, and then at the end we were
asked if we were going to use his code in our platform and if he should be
paid.

Felt kinda bad for him, clearly didn't pass the interview test.

------
jpalomaki
”..format that involves reviewing code together and talking about it. The
process typically takes between 30 minutes to an hour.”

This sounds like a good idea to try. Much less time consuming than having to
create meaningfully things from scratch. Also more relevant as you don't
usually need to write code in front of others, but you might need to review
existing code.

------
mnky9800n
I did this for a data scientist position. It was obnoxious. Compounded by the
interviewer believing his dissertation work was representative of the entire
machine learning field. I didn't get that job. I'm more interested in the
people I'll work with in hiring than what they know. to a certain degree
anyways.

------
tastyfreeze
It really seems like the best way to avoid the need for interview coding is to
have a professional license for software engineers. Prove to a licensing board
that you are competent in your desired specialties.

I don't ever hear of structural engineers or medical doctors having to solve
puzzle to get past their interview.

------
hammerbrostime
You could pay serious candidates for their time spent on long assignments, and
a think that would fix quite a lot.

------
wintorez
Why do you want to work for a company that gives you days of unpaid homework?
If you're given such a task, simply state that you don't think you'll be a
good fit for them, and move on. It is an employees job market at the moment,
and every tech company has several unoccupied openings.

------
expertentipp
In my last job search I solved a dozen or take home assignment. For the one I
finally got, I... refused to solve the assignemnt... and it is a very good
one. Do everyone a favor and refuse, fixing the situation is in hands of
anyone looking for a job.

------
AmrMostafa
I find it best to be asked that kind of homework after I have done my first
interview. It usually means we both got a chance to pass and that the effort I
will put will be seriously considered.

------
johnvega
It's probably not for most people, but this is how I got my job as a C++
programmer in CAD software industry (Computer Aided Design) that lasted over
10 years.

------
GordonS
I've never been asked to do a take home exercise (I'm in the UK, I guess it's
just uncommon here).

Out if curiosity, what kind of assignments are you being given?

------
mtnGoat
i think its utterly insane for companies that are well funded to ask skilled
labor to work for free. if you want the best talent, attract it. i have
refused every interview asking me to work for free, and ive never had problems
finding gainful employment. these companies are just losing out on great
talent by being shortsighted. only desperate people will work for free, and
they are usually desperate for a reason. my $0.02.

------
purplezooey
I had one recently. Two nights it took me. Just a couple of questions will do.
You don't have to come up with pages of shit from a committee.

------
voodootrucker
Spend 20 minutes pairing with someone on real code, and you will know more
about them than 2 hours of code review or any amount of interviewing.

------
gd2
Hasn't this been going on for like 2 years? Or has the trend line made this
more common.

------
mildweed
I touch on this point in my talk, “The Two Question Code Quiz”

Slides
[https://slides.com/scottconnerly/2questioncodequiz](https://slides.com/scottconnerly/2questioncodequiz)

Video: [https://youtu.be/x8vmNXULbp4](https://youtu.be/x8vmNXULbp4)

------
epx
I would love to have the chance of doing some work for a company being paid a
low rate or nothing at all to prove myself. That's how I incidentally got my
best jobs. In the last interview the RH recruiter labeled me "arrogant" \- God
knows where she took that one from. Most probably concealed ageism.

------
tgvarik
Here’s how we did interviews at the startup I used to work for.

Phase 1: 30min non-technical phone screen. Usually we would have three people
on our side of the call, but only one of us would do most of the talking. The
goals were simply to describe the company and role to the candidate, gauge the
candidate’s interest and ability to communicate, and allow the candidate to
ask basic questions. We would discuss and decide whether to advance the
candidate immediately afterward and usually notify the candidate on the same
day.

Phase 2: in-person code review, typically 1hr. We’d give the candidate the
option of providing us with a (working) code sample from something the
candidate had written previously, or of coding up a solution to a problem
faced by our actual product. While I was there we had a good mix of candidates
picking each option. The “homework” problems were always drawn from the
product, so they were real, but we also kept them small, so that an average
candidate could have a working solution in an hour or two. And we’d give the
candidate a week with no other constraints. Candidates could use any language,
could research solutions however they liked, and could email us any clarifying
questions they had. Whichever option the candidate picked, the goal was just
to get our eyes on a dozen sample the candidate had written and could be
expected to understand. At the on-site, we’d have the candidate explain the
problem in the candidate’s own words, then walk us through the solution. The
goals were to determine how well the candidate understood the problem and how
comfortable the candidate was with explaining technical decisions. It didn't
matter to us if the code ran or even attempted an optimal solution; we
advanced some candidates whose code samples did neither, but who were able to
talk us through what they had tried to do and explain why it didn't work.

Phase 3: working interview. We hired candidates for a day, paying
$1,200–$1,600 for the day, depending on the role. The candidate would pair
with a member of the team and work on some aspect of the product. The task was
selected to be representative of the work the role required, but also simple
enough that could be completed in a single day. At the end of the interview,
the candidate got to ship the code to production (for which purpose we had a
big, important-looking, physical red button). The goals were to determine how
well the candidate functions in a working environment, how readily the
candidate asks for help or conducts research, how the candidate responds to
frustration, and how interesting and engaging the candidate finds the work.

It was quite a time consuming process on our end, but it was _extremely_
effective at finding excellent candidates. We never made a bad hire the whole
time I was there.

------
your-nanny
A take-home seems most suited for new grads or otherwise entry level
candidates.

------
some_account
I prefer homework to timed, monitored programming sessions. But the task
should never be a big task.

When i interviewed for a backend development position, the task was to set up
flask with two routes, connect to a dB and retrieve a couple of rows when user
visited the routes.

It was not hard even though I never worked with flask before, and it wasn't
supposed to be hard either. It was a test if I had basic programming skills
and that was it.

~~~
derwiki
> the task was to set up flask with two routes

Setting up a web server is also a different skill than backend programming --
many experienced back-end programmers have never needed to spin up a new
server for their job.

~~~
apetresc
I'm very comfortable with never hiring a backend programmer for whom this is
an insurmountable obstacle.

~~~
gowld
There are plenty of surmountable obstacles that are a waste of interview time.
Why give a jr admin task that any teenager can do as a test for a professional
programmer?

~~~
walshemj
It does test for having a wider skill set that just memorising a text book.

Ill give you an example of a task that I got in my 2nd/3rd year of employment.

Came into work and was presented with an exotic (for the time) piece of
hardware (A0 digitizer) that cost 3x my salary to be asked to.

1 Connect it up and get it talking to a PDP

2 Research and Write the code to allow the kit to interrupt the os and
transfer the data to a second program (this is on RT11 which only allowed 1
process normally).

3 Work how to initialize and calibrate the kit

4 Was asked to go and chat with one of the senior engineers and work out how
to use the kit to digitize the results from his experiment to produce a 3d
representation of the droplet cloud - z axis was done by using a camera with a
very tight focal plane

------
jlebrech
I had to choose on starting "homework" and accepting another job offer. I
chose the job.

------
fwdpropaganda
Come on. People complain about problems they have to solve on the whiteboard.
People complain about problems that they can take home.

This is approaching simple lazyness / entitlement.

~~~
meesterdude
It's lazyness / entitlemant because we want competent hiring practices that
are respectful of our time? Because when you have to work 40 hours in a week
doing nothing but take home tests & quiz projects for companies you are
interviewing for - for the 38th week in a row - and still not taking in any
money for all this - it's more about affording food and paying rent.

The free work is the issue. Pay me for a day to do a project to validate my
abilities and we're golden. I've done that at a few places, and it's a great
way to go.

~~~
fwdpropaganda
Ok, so what do you do when you're asked to do homework?

