
Guide to Take-home Coding Challenges - janephilipps
https://www.fullstackinterviewing.com/2018/02/02/the-ultimate-guide-to-kicking-ass-on-take-home-coding-challenges.html
======
songzme
Good article, but I personally hate Take-Home coding challenge:

Some people spend an entire week on the coding challenge, which really set the
bar unrealistically high. Once I turned in a project 2 hours after receiving
the assignment and got rejected because they think I didn't even try. Every
functionality worked flawlessly, but they want tests and for me to treat it
like a real production work. But how do you motivate yourself to treat such a
trivial take-home (which will never be maintained because its a take-home
assignment) with the perfection you would in production code? In production
you have much better insights like how your component impact others, how users
will be using your app, etc.

~~~
stevenjohns
> but they want tests and for me to treat it like a real production work

When hiring back-end devs, I explicitly make it clear I want at least some
tests -- and the more the better. Of course I don't expect 100% test coverage
but writing tests shows that 1) you understand best practices 2) you know how
to write testable (readable, maintainable, clean and clear) code. If
candidates chose just one module and tested that well it would be enough to
get the point across.

Seeing how people write their tests is also quite important for me -- if
instead of using factories (or Faker or some other strong dummy data app) they
instead decide to hardcode usernames and passwords, it's a huge red flag.

~~~
nmjohn
> they instead decide to hardcode usernames and passwords, it's a huge red
> flag.

Why? That seems like an oddly specific thing to be a _huge_ red flag. Are you
expecting random data to actually surface bugs - ie: making your tests non-
deterministic? Or is there some other benefit you are looking for to make it
such an important criteria?

~~~
stevenjohns
It shows a lack of familiarity with the tooling. It's actually quicker to
write something that prototypes the model you're testing -- as little as a few
lines of code -- that eventually saves a lot of repetition.

That prototyping later becomes extremely useful for building local databases
-- a strong factory suite means I can rebuild my local database and have it
completely populated with randomized dummy data. Changing locales on some
packages means I can have a local environment populated with data tailored for
a completely different country and language, for example.

Re: surfacing bugs -- in a lot of cases yes. And a lot of bugs have been found
this way.

~~~
simplify
It looks like you're mistaking "good programming" for "programming the same
way I do".

If a developer can test, that's a good fundamental skill. If they don't use a
factory, that says nothing about their fundamentals – it's something you can
easily tell someone _who knows how to test_ to go learn & do once they're on
the job.

------
busterarm
In general I find them to be completely unrealistic if you're trying to find
anyone good.

Someone really good will already have a full time job, a limited number of
"sick days" to use for interviewing and is probably somewhere in the interview
process with at least 3 other companies -- probably closer to 5 or 7. Then
they might have family responsibilities on top of that.

I budget 4, maybe 6 hours if I really like you, on any company in the
application process -- tops. Almost 100% of this time is taken up by
interviews. You want me to give your company a whole weekend of my time on
some chance that you might hire me? In addition to hours of interviews? That
already tells me that I don't want to work for you -- especially if I haven't
at least interviewed with 3 technical people already. It's one of the clearest
negative signals I know of.

If I work 13 days in a row without a day off, I'm going to be a crabby asshole
at my job and probably blow all of my other interviews until I recuperate.
Your company isn't worth that.

~~~
djtriptych
This is a little silly. If you’re “really good” you may be talking about
signing on for a 4-5 expected commitment at $1M+ Total compensation over the
course of your tenure. Million dollar contracts require due diligence. 4-5 man
days between both companies is totally reasonable.

If you’re interviewing with 7 (!) companies at a time, maybe be a little more
discerning about where you’re trying to work. Everybody wins if you’re
specific on that point.

~~~
busterarm
> If you’re interviewing with 7 (!) companies at a time, maybe be a little
> more discerning about where you’re trying to work. Everybody wins if you’re
> specific on that point.

Oh please, honestly. I'm not a terribly good programmer, but after 2+ years at
one company, my inbox was flooded with emails from recruiters representing
companies with very interesting opportunities (and like 10x as much crap).
Then there's the friends and former coworkers who have been regularly asking
me to come work with them.

Once I made the decision to leave my last company, I talked to about 9
companies over a 1.5 month period, and it only lasted that long because of
Christmas & New Years. At least 5 of those went through multiple interview
rounds and I was juggling 4 different, attractive offers before picking one.
This is just from the 9 that I decided to talk to...

I live in a major market and there's a lot of demand here and in my focus
areas. A lot of companies that you don't think have interesting problems
actually have fascinating problems to work on once you talk to them. Probably
much more so than GOOG, FB, AMZN, etc.

You talk about being discerning, but how do you know if you can deliver value
until you have a conversation with them? You end up talking to that many
companies at once because once you start the process with them you have to be
respectful and see their process through to the end, but you don't stop
interviewing while you wait for a decision.

------
davidbanham
I think the only humane way to administer a take-home coding challenge is with
an enforced time limit. That way everyone is on a level playing field and
you're not advantaging those candidates that happen to have oodles of free
time.

The author raises that point:

[https://www.fullstackinterviewing.com/2018/02/02/the-
ultimat...](https://www.fullstackinterviewing.com/2018/02/02/the-ultimate-
guide-to-kicking-ass-on-take-home-coding-challenges.html#2-take-home-
challenges-can-be-very-time-consuming)

But takes the wrong view on it in my opinion. The answer simply can't be "Just
spend as long as it takes".

That point is the thing that led me to build a tool that enforces time limits
on challenges via a custom git server. I used it to interview around 100
candidates at a company I was running at the time and it was an excellent
approach.

I've since productised it as [https://takehome.io/](https://takehome.io/)

If you want the benefits that take-home coding challenges can bring to your
hiring process, but are concerned about treating your candidates humanely, I
urge you to check it out.

~~~
deepaksurti
>> enforces time limits on challenges

But there is a problem. In the past when I have worked on take home challenges
which usually take 3 hours or thereabouts, I have done them in 3 sessions of 1
hour each in the evening after work:

1 hour: think about the problem in detail making some notes for scenarios,
edge cases, high level API design etc 2nd hour: first rough cut working draft
with some test coverage 3rd hour: polish it up with more tests, docs, a README
and sending it out.

I think `takehome.io` should account for time boxing across multiple sessions,
else it might be pretty much useless for working engineers who do it across
time boxed sessions :-)

PS: I apologize in advance if you already have this feature, but is not so
readily evident on your product home page.

~~~
davidbanham
Thanks for the feedback! I appreciate your perspective on time limits.

I don't have that feature, because I would argue that a problem you attack in
three 1 hour stints across a span of 72 hours is a 72 hour problem.

Writing software is so much more than just typing. I tend to do my best coding
in the shower or driving, nowhere near a computer. As soon as the challenge
has been loaded into your brain, the work starts and the timer starts. How
much of that time you spend in front of the keyboard is up to you.

Incidentally, I also think that a time limit of longer than an hour is
probably too long. Uninterrupted multi-hour blocks are pretty hard to come by.
Even if the time limit given isn't enough time for you to complete a perfect,
working solution to the problem, it should be enough time for you to do enough
work that we've got something meaty to talk about in the next interview.

------
vanderreeah
As someone who's just entered the employment market after being a freelancer
for a while, I have no idea how people who already have jobs can physically
participate in the lengthy interview processes most jobs require these days.
Ok they can take sick days, but if you're a bad liar or have an overactive
conscience that's not ideal.

I totally understand the value of coding challenges, but I've been endlessly
surprised at companies that expect me to expend more than 4 hours of my life
interviewing for a chance at a job that will be paying maximum 50k. These are
small companies, hardly the big 4. It's really an imposition.

------
PacifyFish
There's a lot of contempt for take-home coding tests in this thread. They
aren't a perfect solution by any means (1), but much closer to an engineer's
real responsibilities than a whiteboard session.

For this reason and this reason alone, I will continue to advocate for take-
home coding challenges.

(1) IMHO, the best practical solution is a pair-programming interview using a
real bug or feature the company has already solved.

~~~
Clubber
No. Talk to them, get a feel, if they are a fit, hire them for a 3 month
contract. If they aren't any good, don't hire them. If they are really bad,
cut the contract short.

You will never know what someone is capable of in a few hours. People who
think that works are just fooling themselves and turning away good talent.

~~~
Arainach
And what of people who don't want to leave existing full time employment for a
three month contract that might turn into a job if things work out well?
Similarly, I doubt most people would be willing to move to a city for a
temporary contract.

~~~
Clubber
Usually if people are looking around, they are pretty fed up with the place
they are at for one reason or another. If it's a good developer, they would
know they would make the cut, so they would take it if it were a decent
company with decent pay.

Also, there is no distinction between a full time job and a temporary contract
in terms of survivability in the US. Most states are at-will states, and not
being able to do the work is a fire-able offense. Just about everyone knows
this.

Most people who have been in the business a while aren't willing to move to a
new city, again unless you are willing to pay ungodly sums of money. If you
are hiring right out of college and junior-mid's, this technique is better
suited, but still, you're competing poorly in a seller's market.

I mean, just because you are hiring and paying a salary doesn't mean 20 other
companies in your area aren't doing the same. What makes you better? Certainly
not market rates and certainly not your exhaustive and mostly ineffective
interview process.

You want a good person, ask your tops if they know anyone good, then try to
lure them away with an offer of 1.2x - 1.5x market rates, depending on the
return you expect from them. Trust your tops and don't put them through this
silly process. Have the top interview them with you in the room. Spend about
an hour with them to make sure they aren't some weirdo. Trust me, they'll
respect you more if you respect their time.

------
gtirloni
Time suggestions are usualy completely off the mark. Four hours turn into 20
for anything resembling production code (which is what companies are
interested in).

I like the idea of take home projects but it shouldn't come right after a
phone screen or even the first real interview. More like as close to the last
step as possible.

And then there's the question of what other professions ask candidates to work
X hours for free. Let's assume our field is too special and not get too deep
into the rabbit role ;)

~~~
janephilipps
I agree that time suggestions are usually way off, which can be a positive and
a negative. On one hand, you can spend a ton of time to go above and beyond,
while on the other hand, you may have other priorities. I think if you're
really excited about a job or company, you can show that by putting a lot of
effort in.

~~~
davidbanham
That inherently biases the interview process away from people that have
additional commitments in life. You're going to end up preferentially treating
those that are currently unemployed versus people that have jobs, for example.

Also, you're likely to be homogenizing your applicant base. Expect to see many
more young people and recent grads versus people with kids.

~~~
janephilipps
I agree with your point. I'll add that for new developers and those without
the traditional CS background (both of whom are my target audience for this
guide) spending the extra time is a learning experience for them that can make
them better developers.

Ultimately, it's not the applicant's choice what the parameters of the
interview process are (I commend you for working to make it better) - given
that, why shouldn't an applicant try to give themselves an edge in the
process?

~~~
davidbanham
You're completely right, Jane. I was viewing this pretty myopically from the
hiring side as that's where I usually sit. Very true that candidates usually
need to play somebody else's game, regardless of whether it's rigged.

------
fizx
A take-home coding challenge will bias your pool of applicants towards
relatively junior overachievers who are barely skilled enough to complete your
challenge.

More senior and skilled people will decline to participate, because they know
they can get a better return using the time otherwise spent on the coding
challenge to network for more and easier opportunities.

So now you've done a great job of raising the floor of your interviews for
little direct cost, but you've also lowered the ceiling.

~~~
on_and_off
I got my last 2 jobs after take home challenges (and I am a senior engineer).

The last take home was a 6 hours exercise. I did not really see it at an huge
opportunity cost.

I would have trained 10 times that for a whiteboard entirely unrelated to my
day to day job.

~~~
watwut
6 hours seems too much to me. I would put maximum I am willing to spend at 4
hours.

~~~
on_and_off
ah, funnily enough we have recently reduced the scope of the exercise and
asked only for 4 hours. At the same time we are now looking for less senior
engineers.

We are also looking at ways we can make this exercise easier for everybody by
providing a template with some of the basic boilerplate already written for
the applicant.

It is difficult to find a happy medium between a trivial exercise and
something you can work on for hundred of hours but I think we have something
fair.

I find it interesting that you put your limit at 4 hours. It is not that much
smaller than 6. Anecdotally 6 hours is the maximum timing .. if an aplicant
said they only had 4 hours that day but delivered something solid in that time
frame I would vote for them.

~~~
watwut
> I find it interesting that you put your limit at 4 hours. It is not that
> much smaller than 6.

4 hours means that I can do it within half a day on weekend and then spend the
rest of the day with something else - going somewhere. At minimum, I can send
kids outside with someone and be 4 hours alone at home to focus. Or I can do
it 2 hours one evening and 2 hours another one - two days.

6 hours means that I spent most day with it. Expecting them to be outside 6
hours in row is a lot and so is blocking one room for that long. It would also
mean 3 evenings not just 2.

> At the same time we are now looking for less senior engineers.

That is cool. (Really, all too many companies like to pretend that juniors
don't exists or cant learn or just should magically become seniors instantly.
Meanwhile, junior can the right person for the job.)

~~~
on_and_off
>4 hours means that I can do it within half a day on weekend and then spend
the rest of the day with something else - going somewhere. At minimum, I can
send kids outside with someone and be 4 hours alone at home to focus. Or I can
do it 2 hours one evening and 2 hours another one - two days.

Makes sense, thanks for answering. Yeah for 6 hours you probably need to leave
work a little bit early or work until late at night (if you want to fit it in
your work week).

The way we setup the take home, you can't work on it for several evenings in a
row : we have a 24h timer starting when you first see what you have to do. So
you can do it over a full day if you want (like 2 hours in the morning and 2
more in the evening) but you can't work on it 60 hours over a week (that's the
main potential issue I can see with take homes .. it encourages you to spend a
lot of time in order to show off)

>Really, all too many companies like to pretend that juniors don't exists or
cant learn .

Yeah, in all fairness a junior is a lot of work. You basically make the bet to
lose 20% (made up number) of productivity for a year in order to mentor
somebody in the hope that this person will stay for a while and grow.

However, we already spend so much time interviewing people that honestly I
can't be convinced that only hiring 'best in the world' engineers is such a
great idea.

------
thesmallestcat
There's a good chance somebody on the other end's going to run your program
without really vetting it. Especially if you include a convenient "test
runner." Not a bad attack vector, just get your alias past the first screen
and you might get arbitrary execution inside their firewall on a host with
access keys and the like.

------
kirbypineapple
I failed a take home challenge from Netflix and was a sad with how it went
down. I was given 24 hours to complete the assignment and told that I 1.)
shouldn't try to be clever, and 2.) that it shouldn't take more me more than 2
to 3 hours to complete. Early on I realized that the only way to get optimal
performance was to use an interval tree to store my data, but thought that an
interval tree was being "too clever". I implemented a more naive solution and
submitted it with comments indicating that I believed that an interval tree
was the optimal data structure to use in this case.

The response was that an interval tree was the correct data structure to use
in order to pass the performance component of the assignment and that they
were not interested in continuing with me any further. In hindsight I should
have reached out to validate whether an interval tree was "too clever", and I
think that was a more important factor in my failure than anything.

~~~
220V_USKettle
Once you are hired, will your coworkers at Netflix communicate better with
you?

------
q3k
If you're thinking of interviewing your candidates with such methods, it's
worth noting that their current employment agreement might prohibit them from
participating due to IP clauses (more exactly, that they require permission to
release any code that they've written, even in their spare time).

This can restrict your pool of candidates quite dramatically, as most US
companies seem to have their engineers sign such agreements.

------
mighty_bander
I was hired by my current employer following a take-home coding challenge. I
consider the experience very positive, and it has allowed us to hire skilled
developers. However, I notice a few differences between my experience and
those of people who complain about these challenges:

1\. The challenge was timed, and limited to 3 hours. This is critical, because
it respects the applicant's time. 2\. Similarly, other interviewing was
limited to a very brief "are you actually a developer at all" phone screen and
a 30 minute personality interview. 3\. I was allowed to choose the framework
and language I was most comfortable with, demonstrating that the employer was
interested in how well I worked, not in how well I knew the particular
framework chosen for their product. I think it probably helped that I used the
same stack advertised in the job description, but I appreciate the flexibility
being there.

Interviewing is a separate skill from software development. A take-home
challenge isn't necessarily going to tell you everything about a person, but
it seems to sort candidates into high/medium/low skill levels with a higher
degree of accuracy than a paper or whiteboard tech interview.

------
booleandilemma
Please don’t normalize “take-home coding challenges”.

~~~
closeparen
At least make them instead of, rather than in addition to, whiteboard coding
sessions.

~~~
Clubber
How about neither. Should be: talk to the guy, talk about prior work. If you
think they fit, hire them. If they end up being bad, fire them.

~~~
toast0
I've worked with and interviewed enough people who can talk a good talk and
can't code their way out of a paper bag. I would rather figure that out before
investing in onboarding someone.

~~~
Clubber
This is going to sound worse than I intend it to, but If you can't tell when a
programmer is blowing smoke up your ass during an interview, you are either
asking the wrong questions or you need to have someone else in there with you.

Interviews, technical or not is a psychological process. Ask them about their
previous projects, ask them what their favorite projects were and why. Ask
them what their least favorite projects were and why. Ask them what the
hardest thing they've worked on to date was and why.

~~~
pkaye
I would have to agree with parent post though. There are always people who can
talk they way through well but can't code. Its actually amazing to watch the
real pros at this deflect all the tough questions. It is a great talent for
something likes PR, sales or marketing but it might not be programming.

~~~
Clubber
I've only hired one person like that and it was my fault. I was bedazzled by
his color printouts of his UI. It was probably fine, but he was super lazy.

------
megous
So if take-homes are to become the best practice for interviewing, perhaps
there should be some service for re-using old ones that companies can trust
that the candidate did for some other company and under which constraints.
Otherwise it's just too much of a time waste for candidates, repeating take-
homes for every single candidate company.

It would be like personal projects but with a few guarantees on top, like that
the candidate got the work assigned by someone else, and s/he did it under
some time/other constraints. And there would perhaps be a public free-form
feedback from the other comapny (no rating). So the next comapny has to judge
the work itself without ability to pre-filter based on whatever rating the
previous one would have given (to avoid bias, because the scoring would be
non-objective/non-standardized).

But I would still not like the take-homes. :)

------
dawnerd
I dunno, I’ve just walked from interviews that want take home tests. Plenty of
other offers out there that don’t require it. If my Github and past experience
are not enough then we’re not a match. Companies are going to put you on a
probationary period anyways so if they don’t like your work they’d fire you
then.

~~~
janephilipps
Sure - as someone who transitioned into development from another industry, I
could only rely on my past experience for transferrable skills, and not my
actual technical skills. While Github is great, my Github is not always an
accurate reflection of where I am technically as I continue learning.

~~~
Clubber
Jane,

It's not a reflection on your or the article, it's more of a reflection of how
silly tech interviews are these days.

------
eee6e6e666ee
If there is a guide or a book for taking an interview, the interview has
already failed and is measuring the wrong thing.

~~~
Arainach
I disagree. Even if you had a hypothetical perfect interview that tested
exactly the right combination of raw logical ability/concept
familiarity/effective communication, there would always be room for a guide
that reminded people exactly how to best prepare.

Interviews need to test for certain characteristics in a limited amount of
time. Even if there was some way to say "here's the code I've wrote, e-mails
I've sent, and videos of me leading team meetings in the last year", no
company would have the time to look at those materials.

As such, we're stuck with the common question formats trying to determine if
you communicate with your interviewer well (understanding what's being asked,
explaining your thought process and what you're trying to accomplish, talking
about anecdotes and building rapport) while also showing off some technical
understanding of implementation and design. No one's yet found a better system
that's more respectful of both the company's time and the candidate's time.

These "take home projects" are an interesting variation that I'm not going to
dismiss out of hand, but it's hard enough to take 8 hours out of my day for an
interview loop - adding more hours in the evenings isn't something that scales
well to interviewing with a bunch of places.

------
rboyd
don't do unpaid take-home coding challenges. particularly if you have no idea
what the company is offering for compensation should you get the offer.

~~~
janephilipps
For a well established developer, this may be true, but for newer developers,
they often have to jump through a lot of hoops to prove their skills first,
especially if they come from a non-traditional background and do not have a CS
degree.

Do you think this issue is less pronounced with the move toward companies
providing compensation guidelines up front?

------
PeptoKelpto
I think it's awesome that you can become a software engineer without any
engineering schooling. Where I live you have to take software engineering in
University to be a software engineer. Bootcamps and certs usually only get you
dev jobs.

~~~
caymanjim
Many companies use the terms "software engineer" and "developer" and
"programmer" and "analyst" interchangeably. There's no universal agreement on
the difference. I'm not saying it should be this way, but it is.

~~~
indemnity
In my market, a title of software engineer generally means higher
compensation.

Even though most of us are not real engineers who studied BEs, the title has
more prestige than developer (or god forbid, programmer).

~~~
booleandilemma
It’s a title arms race and that’s how it’s going to be until it’s illegal to
call yourself an engineer w/o an engineering degree, such as in Texas and
Canada.

------
grim-jokes
Instead of take home challenges, I think I would rather give a potential
employee some code snippet and ask them to explain to me in their own way as
to what is going on and if they see/find any particular issues.

This will theoretically let me gauge their attention to detail. Ideally I
think it would show me how familiar they are with the programming language the
snippet was written in as well.

------
zerr
Flagged. Since no payment is mentioned, so it is assumed that a candidate will
do it for free... Lets not encourage such deeds.

