
Companies shouldn't give candidates homework assignments - le-mark
http://www.gayle.com/blog/2013/09/18/companies-who-give-candidates-homework-assignments-knock-it-off
======
daenz
The title is a little click-baitey, because it suggests full on assault on
homework. The actual post is more measured in its criticisms, but basically
the idea is: don't use homework as a cop-out for not having a good interview
process, and thereby likely wasting everyones time.

Personally I like homework, because it allows me to show my strengths
(actually writing code that is solid) without being "under the gun" in terms
of interview pressures. I love being able to discuss the code I wrote and talk
about areas it can be expanded and improved, and why certain decisions were
made. In my mind, that process is closer to the job I'm often interviewing for
than questions like "explain why Bellman-Ford uses v-1 relaxations"

~~~
chaddattilio
Ya I agree, I was all ready to blast it but then realized it had a couple good
pointers in the case that you are gonna offer homework.

I do disagree with the premise though. As someone who's worked in the tech
field for 15+ years and has been on plenty of interviews, I for one would much
rather do a short homework assignment than have to stand in front of a few
people and white board shit in real-time.

Here are reasons why it's a solid idea:

1\. Way more closely simulates a real work environment. I've never had a job
where I get the assignment moments before I have to code it, use a whiteboard
to write code, am severely time-boxed and have a group of people watch me
work.

2\. More inclusive. Your work stands out more when someone isn't making
judgements based on looks, age, gender, etc.

3\. If you complain about having to spend a couple hours outside the interview
on work, maybe you don't really want the job that bad enough. People will
spend weeks on an unpaid open-source project but then complain about 2 hours
for something that might get them a great job.

Having said that, I agree with you also that the interview can be a great time
to go over the homework and ask questions, talk about why a particular
strategy was chosen, etc.

I would personally love to see more interviews go away from whiteboards and
more towards homework.

~~~
ryandrake
Unfortunately the homework project is usually followed by an in-person
whiteboard hazing, which nullifies those benefits. I recently interviewed with
a well known tech mini-giant who admitted that my homework assignment was
among the best they’ve received, but still flunked me based on the in-person
slugfest so YMMV.

~~~
pmiller2
Name and shame. This is disgusting.

~~~
yen223
Bearing in mind we're only hearing one side of the story, of course

~~~
pmiller2
Assuming, in good faith, that the poster is telling the truth, either the
company lied and the homework project wasn’t that good, or they were more
interested in whiteboard hazing and the project didn’t mean very much. What’s
more like real software engineering: writing code on the spot, on a
whiteboard, in front of people; or writing code at home, which you can then
have a conversation about? A sane hiring process would put more weight on one
than the other, and this company obviously wasn’t sane.

~~~
ryandrake
I won’t name but it wasn’t shameful. They ended up looking for someone with
experience more balanced between web services and embedded devices, and my
background was very focused on embedded. Had they gone purely by the homework,
they might not have gotten the best candidate.

------
strict9
>Candidates spend many hours completing homework assignments for companies
which are not sufficiently interested in them. This is abusing candidates.
Knock it off.

Given a choice of spending 2-3 hours on a take home assignment vs. 1 hour of
writing code on a whiteboard in front of 3 or more developers, I'd choose the
former any day. It's not even close.

The author is taking one end of the spectrum and painting it as an endemic
problem. It's not that.

edit: my strong preference for homework vs whiteboarding is because that's how
I've solved problems and accomplished tasks for years. Whiteboarding to "see
how I think" is not at all how I would think if I were work for them.

~~~
sytelus
The problem is effort asymmetry. For in-person interviews, company must spend
as much time as candidate is spending in evaluation. If homework thing becomes
standard, I'm afraid that will become first line of filtering because it
doesn't cost much to company. So you will still get in-person interview and
may be even bit of whiteboarding (to make sure your friend didn't took test
for you or somehow you cheated out by getting questions before hand).

You can look at the test process in terms of game theory where one player
deals with several players who are not all equally qualified and have
incentive to get the prize. So the adversarial mechanics will eventually ensue
and folks will try to "game" the system. If you invent whiteboarding as
defence, folks will try to invent leetcode as counter measure. If you invent
homework as defence, you can bet sites pop all over that allows you to search
problem and frameworks and download the common solutions with a click.

~~~
wilsonnb3
> you can bet sites pop all over that allows you to search problem and
> frameworks and download the common solutions with a click.

Sounds like a lucrative business, especially if you can introduce enough
variety into the downloaded code to prevent detection by a system like the
anti-plagiarizing systems colleges use.

~~~
Thiez
There are techniques to identify a person based on their writing. Perhaps
software could be developed that takes a sample of a persons writing, and then
modifies text to match their style. Of course one would have to be careful
that the introduced variations are not just in identifiers and whitespace, or
plagiarism could still trivially be detected by comparing the compiled
binaries (after stripping debug symbols, and assuming a compiler supporting
repeatable builds).

------
mysterypie
If there really is a developer shortage that we're always hearing about, then
why don't more developers say "f. you" and walk away from homework
assignments, 5-round interviews, cubicle farms instead of offices, etc. It
would be a self-correcting problem -- companies that wanted good candidates
would stop doing this shit. The only explanation I can come up with is that
the developer shortage doesn't exist. Maybe it's a developer glut but we don't
realize it yet?

~~~
nybblesio
Excellent observation. We should all be asking ourselves why the laws of
supply and demand aren't asserting themselves in the software development
space. From what I see, companies are aggressively forcing rates lower and
doubling down on insanely obtuse hazing rituals.

If demand were truly inelastic (what a "shortage" would imply) then
rates/salaries would be rising and the on-boarding processes would streamline.

Their behavior appears to telegraph that there is no actual shortage.

~~~
tomp
My interpretation (which I think is "more correct" than yours and your
parent's because it subsumes both) is that there's a _good developer_ shortage
and _bad developer_ glut, and the interview process is an attempt to sort an
individual developer into one of the two buckets.

Although from personal experience, I've been rejected from several companies
_and_ gotten more than one _amazing_ offers _at the same time_ (i.e. in the
same "I want a new job" career cycle). So maybe the categories aren't that
clear cut.

~~~
Apocryphon
The thing about separating developers into good and bad is that it also flies
in the face of software engineering culture- the idea that you have to have a
Growth Mindset and you can Level Up through Best Practices and Deep Work and
Teach Yourself to Code after 10,000-Hours (or 24, but who's counting?). The
idea is that if you work on passion projects and read up on the hottest
frameworks and design patterns in your spare time, you can become good. On the
other hand hacker culture also praises 10x developers who are seemingly born,
not self-taught into it.

What this translates into industry is that managers and companies continuously
grind through candidates in an attempt to find 10x rockstar ninja cliches,
rather than finding diamonds in the rough and trying to just _train people_.

Of course, there are going to people who are worse than that level of quality.
Maybe there should be some sort of guidance for them, that even if they fail
interviews, they should at least get feedback as to how they can improve
themselves, instead of being stuck in bad interview death spirals.

------
overgard
As a candidate, I kind of like the homework thing. It's a lot closer to what
you'd actually be doing on the job compared to white board tests, and
interviewing is a pretty big time commitment on both sides so I don't feel
taken advantage of if it's not crazy (the interviewer has to spend a lot of
time on it too)

Also, I think it's really a stretch to call it _abusive_. If you feel like the
test is too much of a time commitment or you don't think the company is
serious, you can just not do it you know?

~~~
ghaff
Something that takes an evening (whether code, writing, etc.) seems generally
reasonable assuming that it serves as a reasonably effective screener. After
all, _some_ homework for an interview such as researching the company and
their market space is typical. Something that takes 40 hours? Not so much.

~~~
maltalex
If a homework assignment takes 40 hours then it’s likely that either you’re
under-qualified or the company has some unrealistic expectations regrading the
position.

Either way, you should reconsider your application.

~~~
ghaff
Depends on the assignment. I don't regularly program these days but for work I
often do (preparing/giving presentations, writing of various types) I could
trivially come up with an assignment that would take a person skilled in the
topic the better part of a week to do--especially given that they'd want to
make their "homework" particularly impressive.

------
netaustin
My company, a fully remote consulting firm, uses test projects for some
candidates whose skills we cannot fully evaluate using the code samples they
have submitted. We only use test projects under conditions designed to skirt
the clear pitfalls mentioned in this article.

\- We pay all candidates for the test project, whether they pass or fail. The
original article did not mention paying for homework, which we concluded was
the only fair way to treat the excess time candidates spend helping us make a
decision.

\- Passing the test project means you move on in the process; we don't
knowingly pit two candidates against each other.

\- The project and a corresponding scoring rubric are predefined by position.

~~~
poulsbohemian
This lines up with what I finally concluded (and blogged about) as well - find
some small, somewhat isolated project and pay them for it. My preference would
be for it to be a project you actually intend to use, especially if they also
need to interact with your team in some small way. This way you get to
actually seem them in action - how do they communicate, what questions do they
ask, how do they approach the project. You see the candidate in the context in
which you intend for them work.

~~~
netaustin
We've toyed with either paying them for something we can use or asking them to
add a feature to one of the open source projects under our care — but in
either case, you lose the reliability and objectivity of the task, because
different candidates will have better or worse prompts. Using the same prompt
for everyone means we can use the same rubric for everyone.

------
jasonincanada
I recently did a homework assignment as the first step in an application
process, and I didn't find it inappropriate or untoward at all. It's certainly
not abusive, I don't understand why the author used this term. It got me
through to the second step, a video conference interview with a couple
developers and a chance to go over my code, explain my reasoning, answer some
challenge questions, etc, in other words, tech screening. I didn't end up
getting the position, but at no point did I feel my time was being wasted or
that I was being disrespected.

~~~
nybblesio
You received a response. What if you hadn't? How many hours did you invest?
How many of these would you be willing to attempt if you never got a response
of any kind? When the author speaks of abuse, I believe -- in part -- this is
what they're trying to highlight. For every submissions, like yours, that gets
a response, there are probably hundreds that hear absolutely nothing.

I'm OK, within reason, with these homework assignments if they come after some
kind of initial screening where the company has shown at least the minimum
level of interest.

However, as the very first step, it feels off to me.

~~~
astura
And there's the asymmetrical relationship if you're asked for a several hours
time commitment before they put any time into you.

------
fartbucket
I've been burned in too many interviews for this kind of homework garbage. I
simply won't do it, if someone wants to waste 1-4 hours on something, go for
it. My apps/etc I build should be enough to see my work.

I don't want to waste 1-4 hours for every job I apply to, especially when I'm
not one of the final candidates. I'm often blown off for not being caucasian
enough or liking whatever sportsball team or brand of music the rest of the
crew is into. To often I feel I'm thrown into interviews because they need to
interview 7-10 people. Same with interviews without phone screening, there are
lots of questions I can ask that can save both me and the employer time and
money by doing a phone screen first instead of driving to their place (mostly
talking about 30+ minute or hour long drives). I've been burned too many times
with things they forget to mention that make it a job not even worth my time
(required 60hr work week, mandatory overtime, etc) things I can find out with
a screening.

------
pmiller2
Let’s compare this to a recent experience I had with a recruiter from Square.
Among other things, I was told that in a 1 hour phone screen, I was to write
code as if it were production code, and not “just interview hacking.” She also
suggested I read _Cracking the Coding Interview_! For a phone screen!

I think this is frankly ridiculous. What do you think the rest of the
interview process is like? At best, it’s cargo culting. At worst, it’s abuse.

Edit: I should add that no guidance was given as to what the standards for
“production code” in this phone screen were.

~~~
rabidrat
The Facebook recruiter told me that "successful candidates spend an average of
10-15 hours preparing"...for a 45 minute phone screen. For the onsite you're
expected to spend a few weeks reading CtCI and working through the practice
problems.

~~~
Shish2k
I wonder where they're getting those numbers from, as I didn't spend anywhere
near that long preparing o_O If it was rephrased as "successful candidates
will have spent an average of 10-15 hours writing code over the past week"
(ie, just working your regular day job counts as "preparation", since you're
flexing your coding muscles), that sounds more believable...

~~~
bradlys
Nope. It's definitely interview specific prep.

Read /r/cscareerquestions/ to get an idea of the levels of prep that many
candidates go through. Not everyone needs that level of prep - especially if
they're interviewing candidates at their current job. (No better way to see
how not to fail by seeing endless failures)

People prepping 100+ hours for interviews at these big tech companies is
completely common. After all, spending 100 hours to get 100k+ more comp is a
pretty good tradeoff. It's just sad when you do all that prep but fail anyway.

~~~
pmiller2
It might be a good tradeoff for the individual candidate, but it’s a terrible
practice for the hiring company. It’s like hiring a physicist by giving them a
pop quiz on quantum electrodynamics.

------
sqs
At Sourcegraph we faced the same decision and decided to go with an option for
paid “homework” projects. It’s working very well so far. The payment is an
important element of respect. More info at
[https://about.sourcegraph.com/blog/our-project-based-
intervi...](https://about.sourcegraph.com/blog/our-project-based-interview-
experiment-for-hiring-engineers).

~~~
deskamess
Paying shows you have skin in the game, value the candidates time, and you are
not just handing out a ton of take home projects to every one who applied. I
am sure some companies have the money to do so, but generally I think that is
not the norm.

Off topic/curious: Is the take home project expected to be honor system
'closed book/closed internet'?

------
tombert
I had a friend that did an interview homework assignment, was declined for the
job, then found out that they used his assignment's code in production through
a friend that already worked at the company.

While I'll admit that this is atypical, it seems like something a more
nefarious company could abuse.

~~~
FlipperBucket
So it was an accurate depiction of what they would be working on? It seems
like this is akin to the goal.

~~~
homonculus1
That is... missing the point.

~~~
FlipperBucket
That's missing a tangential point, but not THE ostensible point (topical)?

------
jkingsbery
I've only interviewed with one company that did a homework assignment "right"
\- it was time constrained, it was after an initial phonescreen, I was still
more on the junior side, and the onsite interview included a follow up on the
homework assignment to extend it in some way.

But generally, I agree. There are enough other ways to screen people for basic
competency (20 min tech conversation, looking at StackOverflow profiles, etc.)
that it seems unnecessary for anyone except junior developers.

------
pnathan
I'd rather have a weekend homework assignment than a loop of coding
interviews. Much more reasonable assessment of skills & potential, in my
opinion. Happy to do whatever algorithm madness thrown my way in that case.
It's a real tough gig to solve some ambiguous algorithm problem in 40 minutes
on a whiteboard, analyse it, and then discuss optimality. Gets worse if the
loop is serial algorithms interviews.

~~~
pmiller2
This is one of the exact points made in the article: if it’s a screening
question, it should be really basic, like FizzBuzz level. If it’s an
evaluation that goes into whether to make an offer, then the scope needs to be
clear, and it should replace a portion of the actual onsite interview.

The problem is that it typically doesn’t replace part of the interview.
Incidentally, this is my criticism of TripleByte... pass their interview, and
you skip to the onsite with a bunch of companies. But you still have to do the
full dog and pony show with the client company, except the phone screens, and
there’s a substantial time investment involved up front. You need to do a lot
of onsites before you recoup that.

------
rootusrootus
I have had great success giving out a very simple homework assignment before
in-person interviews. I ask you to write me a class which will score a bowling
game. Just hard enough to show me that you know how to code, but not so
difficult that you will spend hours working on it. Gives us something to talk
about while I figure out if your personality will be a good fit with the rest
of our team (because really, that's number one on my priority list).

I'm biased, I suppose, because I loathe algorithm questions in interviews or
whiteboard coding. Neither does a good job of letting me show you I'm not an
idiot.

------
madisonmay
Or as an alternative -- give them something legitimately beneficial to your
company to work on, pay them standard consulting rates. This puts the impetus
on the company to have fairly high certainty the candidate is qualified before
the take-home, and the candidate gets compensated for their time even if a
hire isn't made.

~~~
heyyyouu
We only do homework as a final screening test for candidates. If it's any more
than a simple project, we pay them standard rates. That way no one feels taken
advantage of. Plus, if they bomb it and they're getting paid for it then you
KNOW they're not the right candidate, even if every other test was aced, so it
really is an excellent tool for us. But again, ONLY at the end of the process
when we're making a final decision between, most likely, two candidates.

------
AndrewKemendo
This is really not that hard to do right in my opinion. This is what we did
for a "homework assignment" after a 30 minute phone and 30 minute team screen
were already done:

1\. Supervisor had candidate sign mutual-NDA, work for hire at $X/hr rate & IP
assignment agreement on docusign.

2\. Potential future supervisor gave the candidate access to a repo and
assigned a production task that should take less than 10 hours to complete.

3\. Candidate does the assignment, communicates with supervisor via email/repo
where necessary and a check gets cut for the hours worked.

4\. Hire/No Hire decision made

Should take less than a week in total.

~~~
ghaff
>Supervisor had candidate sign mutual-NDA, work for hire at $X/hr rate & IP
assignment agreement on docusign.

This would be a violation of the candidate's employment contract at many
companies.

~~~
icedchai
Perhaps, but totally unenforceable... It's a non-issue.

Developers are not slaves, and you cannot prevent them from doing outside work
while looking for other jobs.

~~~
wirrbel
Don't know about the US but here it would be reason enough for my employer to
terminate me right away. This is the default for employments. For work on the
side, I need to ask my employer for permission. My employer will then be
concerned that all labour protection laws will not be violated, which for
example means that my employer must accept that I might not be able to work
overtime for them, with a side contract, because I would surpass a 10h per day
threshold

To cut the long story short: I would politely declined that model, even though
I think it is sensible.

~~~
sixstringtheory
In terms of hours per day worked, what is the actual difference between doing
an unpaid take-home test? Signing a contract is just being honest about the
time it's going to take to learn about the candidate.

What about onsite interviews, do you have to tell your supervisor that is the
reason you are taking off work? Would they be OK with that?

~~~
wirrbel
One major problem is that the employer has the right to monetize work products
that are close enough to your jobs work. So if my side project is writing a
novel, there isn't much danger and the employer must basically grant the
permission.if it is related to coding things get trickier. With a contract for
coding I would basically transfer monetization rights to you. for that I would
have to ask my employer.

So I would not do take home exams that could appear to be consulting work etc.
Of it appears to be a test, it is fine(r).

Note that this is tricky for open source work too, you are doing on the side.
(At least in my legislation). Contributions that are closely related to my
work topics I usually ask for a permit according to company policy. For
completely unrelated stuff (writing that interpreter in prolog, etc.) I assume
my employer is disinterested (they really are). I cannot assume that for
contract work with another company.

Anyway, a judge, given a CONTRACT with another company that was not approved
has the easiest case in front of them. This is the situation I want to avoid.
I wouldn't bet on a judge to dive into the details on what stuff was coded in
this situation.

You mention the on-site interview. This is easy. It is not a consulting or
contracting gig. So I don't need to get a side-work permit, and they are
treated as any of my other absences when I don't supply a reason.

~~~
icedchai
"Side-work permit"? Is that actually a thing?

~~~
perl4ever
I don't know if that's a thing per se, but my former employer (Fortune 500
non-FAANG) quite definitely claimed any code I wrote during my employment
whatsoever.

I did one time formally ask (just to see what would happen) whether I could
volunteer at the Red Cross, which demanded I sign a boilerplate intellectual
property agreement saying anything I made as a volunteer belonged to them. The
letter of my employment agreement was incompatible as all IP I made belonged
to my employer. The question went all the way up to the corporate counsel, who
presumably said "obviously a troublemaker wasting my time with this sh*t" and
responded "NO".

I thought about writing the CEO because it was absurd you technically couldn't
volunteer with the Red Cross, but of course, there were most likely lots of
people who simply ignored the issue and/or ignore what they sign. There were
certainly people in my division who appeared to be flagrantly violating the
prohibition on an outside business while employed, but no doubt had the sense
not to specifically ask permission.

~~~
icedchai
I find it is often better to seek forgiveness than ask permission in these
matters.

~~~
perl4ever
Yes. Like I wrote, it was something of an experiment.

------
teh_infallible
A lot of the commenters here are saying they like homework assignments because
it’s a test that’s similar to the job, more so than solving a problem on a
white board with no resources and a group of people watching you. I agree with
all of that. Here’s where homework assignments are abusive- when they reject
you with no critique of your work whatsoever. That’s been my experience, and
it’s total, disrespectful bullshit.

If I spend time on your made up assignment, you owe me a detailed response
that shows that you actually looked at my work and can critique it credibly.

------
agentultra
The article has a bit of a weird structure. The premise is that _homework is
abusive_. And the final bullet points elide the exceptions to this rule...
which seem broad enough to make the premise weak.

Using homework as a first-pass screen in your interview process, unchecked, is
definitely a smell that you're not using this tool appropriately.

However I use it in the process I've set up successfully as a way to give me
something to code-review with a candidate when we do a final interview. It's a
small, limited scope project. We give plenty of time to finish it. And the
chances are good if you complete it we'll give you a final interview.

I find it helpful because I don't have to make assumptions about the
candidates knowledge and skill level from a cold-start. I can ask them
questions based on their code. _How did you interpret this requirement?_ _What
trade-offs did you consider when you chose this data-structure?_ Etc. If their
code shows an aptitude for computer science fundamentals I can ask them
questions about algorithmic complexity in Big-O notation or I can scale it
back and see if they understand the same concept using more descriptive means.
Conducting an interview is hard if you don't have a starting point for a
conversation.

For more experienced candidates with a history of open-source work do consider
allowing them to present their own code in place of your standard homework
assignment. It has the same effect and shows that you respect them enough to
review their work. I think that's rather nice.

------
curtis
I probably won't ever do a homework assignment again, unless I feel like I
absolutely have to. It's not just the time involved, it's the time involved
plus the apparent necessity for me to _ask_ what the results were.

It seems obvious to me that if a company asks an applicant to do a homework
project, then when the applicant submits the project, the evaluation and
feedback (positive or negative) should happen automatically and quickly.

------
logfromblammo
Last time I went around interviewing, companies that passed out homework
basically front-loaded the candidate time commitment so that they only had to
put in their own time if the candidate looked promising.

The homework was not bad in itself, but was a symptom of a larger problem.
Namely, the company was unwilling to invest equal energy into the interviewing
process as their candidates. So my first impression of the company was that it
was full of brusque assholes that valued their time more than mine. It seemed
patronizing.

This was certainly also possible without homework, as in the case of the
companies that wanted me to disclose my salary expectations up front, without
disclosing what they expected to pay the median candidate. They wanted to
reject quickly, so they could reject more candidates, and finally get to the
good one that they'd want to hire. As the article author might put it, don't
use an evaluation tool as a screening tool.

------
Apocryphon
Link is (2013). As a further note, the writer is Gayle Laakmann McDowell,
author of CTCI (Cracking the Coding Interview), which might explain its
covertly pro-whiteboard interview stance. Though I agree with the comments
saying that her actual criticisms aren't as clickbait or absolutist as the
title makes it sound.

------
burger_moon
Of course she doesn't like homework assignments. She can't sell books and make
money off that idea.

------
jawns
By assigning homework, companies are also missing out on an opportunity to get
to know how the candidate thinks and works on a problem.

It's as if you're scheduling a full-day interview but only being present for
half of it.

And the thing is, especially if the opening is for a less senior position,
observing the person while they are completing the assignment can help you get
a much better measure of the person's skills, and can also prevent them having
to toil for hours with no feedback about whether they're on the right track.
If you're observing, the candidate might come up with a suboptimal solution,
but with a few gentle nudges, they might correct course and prove themselves
to be a very capable person. And you would have totally missed that if it were
homework.

~~~
lawnchair_larry
The opposite is more likely true. You are very unlikely to get a sense of how
someone approaches a problem in an interview setting.

If given an assignment, it is very easy to have them explain after the fact.

------
bumholio
"Thank you for the interest you have shown in me. With regard to the current
home project you have assigned me to progress in the interviewing process, I
cannot but feel it's somewhat excessive with regards to my time, while being
completely free and noncommittal for you.

On the other hand, I certainly understand that you must vet that candidates
can actually produce quality software before extending an offer. I believe we
can reach a reasonable understanding where I am compensated at my standard
consulting rate of $150/hour for any project or homework you might have. This
way, the process becomes fair with my time and it also gives you the
possibility to thoroughly test my abilities by risking no more than a token
amount.

Thank you, eagerly awaiting your response."

------
tomp
Ok, so let's summarize: no homework assignments, no whiteboard interviews, no
"trick" questions, no "personality fit" questions, no work-sample exercises
(could be interpreted as unpaid work), no relying on credentials.

How are we supposed to hire _anyone_?

~~~
ManlyBread
How about "here is our simple sample project and documentation, please study
them and during interview we will fix a few simple books and discuss how to
implement a bigger change or how to refactor the existing code". Fair for both
sides and much more closer to actual work setting.

------
vinceguidry
Homework assignments are fun and interesting to do, so long as I don't already
have a job or I've golfed my day job down to a few hours a week. But if I have
a non-trivial day job the intrusion into my personal life quickly becomes
unmanageable. I declined to continue three opportunities because the homework
was ridiculous in various ways.

If your organization assigns me homework, I'm going to look very carefully at
it to ascertain whether the filtering function it serves is meat-grinder'y or
not. If it feels like a meat grinder, you won't even get an answer from me as
to why I didn't complete it. There just won't be any further contact.

------
woah
We used timed (8 hours), paid homework to hire recently. It definitely was
really helpful for one position in particular. Dunno why the article doesn’t
mention this option.

Looking at the profile of the blog post’s author, she went to a prestigious
school, worked at prestigious companies, and wrote a book on whiteboard
challenges.

It is clearly very heavily in this person’s favor that interviews stay focused
on these criteria over actual performance.

Our homework ended up weeding out some impressive looking candidates in favor
of people who were less impressive and good at interviews but better at
getting things done.

------
asadlionpk
I am a founder of a company that helps companies with screening candidates[0].
I understand the negative response towards this. But I feel there is a balance
here: We advice our customers to send smaller problems (~30mins) meant to pre-
screen the candidates (without wasting too much of their time). This followed
by a phone interview is currently the best funnel we know so far (and we have
been working on this problem for years now).

[0] [https://remoteinterview.io](https://remoteinterview.io)

~~~
kowdermeister
These candidate screening portals that make me code in a web IDE are nightmare
to me.

My main problem with all of them is that I don't have a chance to talk to an
interviewer real time to ask questions, express my thought process or go
though the problem together. The time limit is usually harsh, once I needed 5
more extra minutes and the system shut down the test, zero points, but I had
it coded 90% and the rest was in my head.

Test cases are revealed _after_ the code is submitted (or not even at all)
which is the reverse of what happens in real life.

They are solving the wrong problem. Yes, hiring is broken, you made it better
for employers, but not for candidates. I guess this is what we get for bashing
the whiteboard interviewing for years, finally the disruption has arrived.

~~~
asadlionpk
I think some simple tweaks by your employer could have made this system more
comfortable for you too:

\- Like giving ample time to the test and then deciding from the report if to
approve or reject.

\- Hidden test cases are to avoid hardcoded solutions. I think these cases
don't map to unit tests, they serve a different purpose here.

~~~
kowdermeister
> Hidden test cases are to avoid hardcoded solutions

If that's the case, it's another way to prove my point that these portals are
bad. I would expect that the company checks the submitted solutions so there
is little reason to write "if (x == 5) return 1" style code if that's what you
meant by hardcoded solutions.

------
gedy
This is terrible advice for companies and interviewees. We've successfully
used "homework" project when a candidate had potential, but didn't do well in-
person for various reasons (nervousness, time pressure). In many cases we've
hired them when they can produce something and talk over it afterwards.

Without this, they wouldn't have received an offer.

While I get they said "which are not sufficiently interested in them." but
it's still an interview process to see if we should/should not hire.

~~~
pmiller2
This is something I support, provided the time involved for the candidate is
roughly the same as the onsite interview. I can talk about code I’ve written
in advance much more easily than anything can pull DP algorithms out of my ass
on the spot.

------
harlanji
As a destitute person who used to be on the A-List for hiring, homework is a
deal breaker. I did one assignment out of desperation recently when a hiring
manager approached me via twitter DM, and despite the good unofficial coffee
meeting I didn't even make it to a phone screen after spending 2 days' free
time working on the homework (about 4 hours). I knew I should have declined to
begin with, perhaps could have spent more days, but it did take a number of
them. They just said my submission wasn't up to par, which I knew would be the
case given my time constraints... but the coffee meeting went well enough that
it seemed okay and like it'd just be a formality, so I went ahead.

Opportunity cost, wise: the bottom of the barrel for pay is Day Labor, where I
can make about $100/day after a 1-2 day wait list... putting the fair market
price of the homework assignment at about $250. The hiring manager did Venmo
me $100 for food, which was well appreciated... but unfortunately still didn't
give me what I needed to do a good job on the homework. Not that it's directly
productive work, but considering head fees are in the 5 figures, a little
compensation to grease the gears doesn't seem like it'd hurt even in
aggregate. Perhaps it'd incentivize the hiring company only giving homework to
better screened candidates, as well.

Makes different points than ones in the article, which are the original
reasons I opposed homework. But in addition to all that, it also blocks people
without good living situations from the job (perhaps desirable for some
reasons, calling it out).

------
triplee
I like this idea. I HATE the homework, because one look at my resume tells you
that even if half of it's true, I've been around the block and can pick up
anything that doesn't actually require specializing in it since 2000 when I
started working professionally out of college (as opposed to co-op jobs where
I also did IT).

And very little in IT fits that bill.

IT hiring processes are part of the reason I took a sabbatical in 2016. I
didn't have time for all this homework plus networking and interviews all for
finding a new job on top of a new baby and a stressful day job.

Of my four eventual job offers, yes four, so I was clearly in demand, only one
even had homework, and it was a short online logic quiz, not even coding.
Everything else was phone/in person interviews.

Screw your homework. If you really need it, this article has a good mitigation
plan.

~~~
deskamess
Did you have to whiteboard code/algorithm during the phone/person interview?

------
jonjonBoy
Great article. Agree with most of the points there. It's an idea that looks
great on paper but it's actually not that great.

It's like the 'unlimited vacation' rubbish. Like yeah, people area actually
afraid of taking vacations or they take too much and got terminated. It's best
to know how many days you've actually earned and take what you have than
playing those games.

Plus no actual way to verify who's doing the homework assignments anyway as
opposed to a live interview however imperfect that is.

On top of that, some sketchier startup founders opened up that they're
starting to abuse this to get tasks done for free. So as a candidate why'd you
allow yourself for this?

------
xenihn
My biggest problem with homework assignments is that they measure how much
time you're able to dedicate to the assignment more than anything else.

Even with disclaimers like "limit yourself to x hours" or "this shouldn't take
more than x hours", some candidates will still spend way more time than that
on it.

Employers end up comparing assignments with 200%+ of the "expected" time spent
versus people who actually adhered to the timebox. If you're someone who can
only dedicate 3 hours in total, whether or not you're in the running might
depend on how many other candidates can spend 10+ for perfection or going the
extra mile.

------
rm_-rf_slash
I’m fine with homework assignments as long as they’re paid. I was given a task
to implement a new feature for an interview to the tune of $300. I didn’t get
the job, but I got paid, and I have held the company in high esteem ever
since.

------
gamache
Candidates who don't have a portfolio with code I can look at: hope you like
homework.

I don't like asking for unpaid labor, but I also won't risk my team blowing
hours interviewing someone who has no chance of being successful here.

~~~
jdlyga
Portfolio of what? I'm not a graphic designer. I'd love to be able to save
great code that I've written over the years. But it's all covered by license
agreements, NDA's, security clearance, etc. So unless you're exclusively
looking at amateur projects or open source developers, that's not a good
business plan.

~~~
logfromblammo
Right. All my best code (and worst, of course) is owned by my employer and
former employers, and is usually exclusively licensed to their customers.

I tend not to code in my off hours any more, because I have different
interests now, that require the expenditure of leisure time to enjoy. If I do,
it is to solve an immediate need or as a side-project to make money directly,
rather than to maybe get another job in the future.

If you want my portfolio, I want to browse your defects database and code
repository before we interview.

------
grigjd3
I have a four year old. I research a company, try to determine what tools they
use and architecture they might be employing. I research what people are paid
there. If requiring a move, I do research on the location. This takes lots of
time. The more of my time that is taken up, the less likely I am to bother.
The position of the company may well be they aren't interested in someone
who'll not keep jumping through more hoops and if that's their position, then
it's fine. I likely don't want to work there. I'd likely be happier at a
company that respects my time.

------
newnewpdro
I can totally envision recruiters spamming candidates with the take-home
exercises regardless of what the employers intended when drafting them up, in
the interests of keeping a full queue of candidates+results on hand.

That's unfortunate, because I strongly prefer the take-home approach over in-
person whiteboard exercises. Recruiters are often sleazy, that's really the
crux of the problem IMHO.

Personally, I've always had a portfolio free software projects to point them
at for review before any in-person interactions take place. I think that's
probably the best defense for candidates in this scenario.

------
llimllib
> And that's the problem: it's free. Interviews have a nice limiting function;
> if it's a waste of the candidate's time, then it's a waste of the company's
> time. Not so with homework assignments.

The company I work at's primary screener is homework, and it takes us a ton of
time and effort to grade homeworks, this argument does not make any sense.

[https://adhocteam.us/2018/02/26/why-we-use-homework-to-
recru...](https://adhocteam.us/2018/02/26/why-we-use-homework-to-recruit-
engineers/)

------
xexers
> "Have a high "pass" rate. If 90+% of candidates aren't progressing past the
> homework, you might have an abuse problem. In an abuse problem, companies
> are giving homework to candidates who are unlikely to make it. Figure out
> why you're giving too many poor fits homework, or figure out why good fits
> aren't doing well."

That sounds crazy to me. Both from an employer and employee standpoint... If
that's the case, it's a waste of time. Why have a test that almost everyone
passes?

I've given homework like this before and our pass rate was about 40%.

------
ares2012
While the post makes a few good points, the underlying proposition is flawed:
homework assignments are not abuse. The candidate can refuse to do the
assignment if they would like. There is no forcing function on the work that
would make it abuse.

In general, I'm in favor of any interview process that ensures the best people
get the job. From what I've seen, homework assignments increase the likelihood
the best person gets the job instead of the person who is best at
interviewing. While it does take time on behalf of candidates, I'm not sure
there is a better way.

------
caboteria
I was recruited by a large travel company that advises people on trips. They
sent me a questionnaire beforehand and asked that I send them my answers.
Being earnest and naive, I spent a couple of hours writing a bunch of short
essays as answers. When I arrived for the interview, nobody had even read
them. Needless to say, each time I've been re-recruited by that company in the
intervening years, I've declined. Fool me once, shame on you. Fool me twice,
shame on me.

~~~
astura
What sort of questions were they? Technical? Personal?

------
perl4ever
I have only had one experience with a "homework assignment" and I liked it.
But the end of the process was coding during the in-person interview, so doing
well on the homework availed nothing. It seemed like a disconnect in the
process.

In looking for an employer, I think I might like to filter out anyone who
particularly dislikes or does not value homework and/or Fermi problems.

------
jvagner
The only rule I think I have is I won't do any test or assignment off an email
invite.

Qualify me a little, then a personal invitation for a test/homework can be
considered.

But sometimes it seems like you send in a resumes and the _response_ to a
resume submission (even if a human looks at it) is to send you to a test.

To that, "no".

------
GoToRO
One thing I don't like about homeworks is the useless things they ask
sometimes. For example, I had to sort a table after all the columns in the
table. Guess what? if I can sort it after one column, I know how to sort it
after any other column. It just showed that they didn't care about my time.

------
arwhatever
Recently had a place give me a "homework assignment," but with a 2 hour
deadline and a "feature request," but with no really concrete completion
criteria, just to see how far you get in that much time. I preferred this to
most other types of interviews that I've done.

------
jdlyga
I'm 100% more willing to do a take-home coding exercise than sit for grueling
whiteboard problems. Preparing for whiteboard problems takes prep time at home
too, in fact a lot more so. So homework assignments actually save time.

------
lawn
Both extremes are bad. Yes the interview process is broken but no I won't
spend a lot of time if I'm not paid. If it's just a small assignment then
that's fine.

------
rickmacleod250
this whole homework / whiteboard discussion is disturbing, how about do a good
interview, prepare, and target the individual your speaking with, ask specific
questions, not blanket questions pulled out of a "how to interview" book, and
then send em home with an assignment or two....

if you cannot work through asking someone detailed questions, to find out how
they operate and solve problems, if you cannot formulate words to ask me the
right questions... do i want to work for you???

------
rootsudo
I had homework for two engineer roles, ironically both in Santa Monica.

Then I was ghosted afterwards. They weren't complicated. Simple Git/bash, and
I did it in about 3-4 hours.

I won't do it again.

------
darksider611
If they pay you for that task I don't see why this could be bad

------
User23
This is a great example of skin in the game affecting behavior.

------
darksider611
If the task is paid, I don't see anything bad about it.

------
modells
There was a SF startup with an obnoxious, high-density open-floorplan (ie a
narrow table down a long corridor like a bad library with the panopticon of
managers watching) that tried this bullshit with me. They didn't even look at
it, a sample of how Puppet works. They asked about MySQL architectures for
scaling reads, writes, HA and backups, I provided clear answers to all of them
while they made all sorts of snide, unprofessional remarks that weren't based
in facts. Overall, they were incredibly rude, dismissive, inhuman, self-
aggrandizing and arrogant muppets. They even had the gaul to call me up for
another interview later after previously literally pushing me out the door. As
I went independent, I let their HR and recruiter know with some actionable
feedback: "Please convey this message to (shall go nameless hiring manager):
'Fuck you. (middle finger). Please go fuck yourself.'" They deserved chewing-
out but they weren't worth my time.

------
hwork4u
Why not?

I love _paid_ homework in the range of $300-$500 USD/ for a day or 2 of work.

I'm all against _unpaid_ homework (f u pay me) - but paid home work is great.

What's not to like?

