
Every software company giving a take home coding test - ritchiea
https://twitter.com/realKetamine/status/1230127442733535232
======
onion2k
Last summer I had an interview with a large insurance company for a front end
role. The take home test was to produce 5 HTML+CSS+JS components to their
existing designs (basically to take a PNG and turn it in to a web component).
The components were a hero element, an information card, a notification
banner, a list and a working calendar with selectable date ranges across
different months. They were all to be made without any CSS or JS frameworks
and they needed to be responsive. They suggested this would take about 4
hours.

I read through it and spent a couple of hours on it before abandoning the test
and sending them an email to withdraw my application. If I was making those
components properly I'd spend about week of solid dev time on them - 1/2 a day
on each of the first four components and 3 days on the calendar..

Maybe I'm a terrible dev and very slow, but all that test really told me was
that the developers there hack things together without spending time thinking
about what they're doing, and on reflection I didn't want to do that.

~~~
simplecto
I can respect that position. However, as a hiring manager I can see their
perspective.

That perspective being, "How fast are you at good enough?"

Food for thought in the future. As a hiring manager I would have accepted all
5 done for 50% of all screen types, with developer comments on where you got
stuck or what compromises you chose to make.

In my opinion that should be made clear in take-home tests. "You are allowed
to cut corners, but please comment on WHY you did so"

~~~
ryandrake
Are you really looking for “engineers” who can half-ass a project from start
to finish in 4 hours? That’s the quality bar?

If I were in OP’s shoes and feeling frisky I would point out to the hiring
manager that for a real world project like that it would take at least 4 hours
to just gather the requirements, write a design doc, and provide an estimate.
4 hours to do design, implement, and test something properly is ridiculous.
And if you aren’t focused on doing it properly, then——wait, what company are
you hiring manager at?

OP’s hiring company was an insurance company, not an Uber-For-Cats startup.
The software he or she is eventually expected to write on the job is supposed
to work.

~~~
megous
An engineer should be capable of selecting the best solution for given
constraints. That's a core of the job.

If the constraint is 4 hours, the manager can't expect nothing but 4h quality.

~~~
subhobroto
> If the constraint is 4 hours, the manager can't expect nothing but 4h
> quality.

Yes, but how does the manager estimate that it indeed typically should take 4
hours when the numbers he gets are:

1\. From his employees who work at that company, intimately familiar with the
problem domain, tools and possibly came up with the problem statement in the
first place and solved it in 4 hours after thinking about and working on it
with no other distractions

2\. From candidates self reporting the time it took to complete the problem
severely undereporting the time it took them because they want to impress the
employer?

Is 4 hours of focused, uninterrrupted time equal to 8 4 hours of unfocused,
interrrupted time?

How does the manager discover the truth?

~~~
megous
By looking at the result of 4 hours of such work, obviously. :)

------
aphextron
Take home coding tests are fine. I really prefer them, honestly. The problem
is when you do the test, write great code, they love it and ask you to come
onsite, and then you get white-boarded by a group of people who never even
looked at your code or resume and get rejected for not being able to cough up
a DFS algorithm on the spot.

~~~
nostrebored
I'm always shocked by the examples that come up when people talk about when
talking about whiteboarding. DFS is a simple, understandable, easily
modifiable, and useful algorithm with many real world applications.

~~~
cblades
That there is a plethora of implementations of and documentation available
for. There is no need to memorize an implementation.

------
seanhandley
A coding exercise should, ideally, be a starting point for a follow up
discussion.

Then discuss things you would do next given unlimited time etc and justify
those points. That's plenty insightful into what a developer prioritises and
how they approach software development.

Expecting a perfect solution isn't reasonable.

~~~
ritchiea
That sounds ideal but oftentimes, intentionally or not, that isn't the
expectation communicated. And in some cases I have seen take home coding
exercises that are literally projects big enough for weeks of work. That
particular small startup had copied their take home exam from a bigger
company. They told me "work as far as I can," but it is intimidating and
accomplishing a reasonable amount in say half a day still would feel like I
had barely scratched the surface of the work they "suggested" I do "as much of
as I feel like." It made me feel like an inadequate candidate by feeling like
I can't possibly put enough of a dent in that task for it to be meaningful.

Oh and by the way the people asking me for that work were significantly more
junior than me. I politely passed on the company and even beginning the take
home test.

I have also had a company give me an API to build with sparse instructions. I
did my best, I probably should have asked them more questions. I got no
response. I followed up weeks later to see if there was more I could do with
the coding exam or if I misunderstood it somehow. They didn't have anything
else to say about the coding exam but they did invite me in for an in person
interview. They then ghosted me and my further follow ups about the status of
the hiring process.

The thing about the hiring issues in the industry is they aren't as simple as
the solution you offered up. They are systemic and it doesn't feel like anyone
is even trying to improve them. And that the changes that do happen are
equally but differently absurd in their expectations as the mistakes of the
past.

------
undebuggable
"Afterwards we'll ghost you or send a rejection email without providing any
meaningful feedback."

~~~
seanmcdirmid
Ya, ghosting...what is up with that? I get a kid doing that, they might not
know better, but a CEO of a startup?

~~~
subhobroto
How can candidates attack the ghosting problem?

Leave a comment on "Who's hiring" that "Hey, I got ghosted by this company?"

~~~
Apocryphon
Perhaps there can be a list on GitHub that updates with self-reported
incidents. And there's always Glassdoor.

~~~
heavyset_go
Glassdoor deletes reviews and has no problem with employers and marketing
agencies writing dozens of fake, glowing reviews to drown out the real ones.

------
thebigspacefuck
I spent a good amount of time on one of these after a phone interview for a
company I was mildly interested in. I’m confident it met all requirements, yet
I submitted it and got no response one way or another. I could have put more
time into it, but it met requirements and I thought, in the real world, would
this company need me to make everything perfect or just good enough? However,
in these take home tests where the implementation is so simple that
submissions are judged on factors other than code correctness, it becomes an
arms race between applicants and who has the most time to jump through hoops.
I’d rather do a white board test and if I fail, have a chance to redeem myself
with a take home test. In my experience it wasn’t worth the time I put into
it. At least it helped me make a decision.

------
mooreds
At a previous job we assigned folks a take home test, but paid an honorarium
for their time. Then part of the interview was a code review session.
Different projects for different experience levels, but never more than 15
hours total (usually more like 2-10).

Seems like the best compromise between flexibility for the employee, seeing
actual real world work for the employer and allowing the candidate to shine.

------
dyeje
I'll do the take home over a whiteboard firing squad any day of the week.

~~~
crimsonalucard
What does this say about people who survive and dominate the firing squad?

~~~
52-6F-62
Only that they've practiced dodging bullets in order to get a job designing
kitchen appliances?

~~~
crimsonalucard
Well it identified a person who has the super human ability to dodge bullets.
What other way to identify and hire a super human?

The kitchen appliance is a separate issue separate from identifying super
humans.

~~~
52-6F-62
The point being the ability to dodge bullets is irrelevant and costly when the
task is to design a kitchen appliance.

~~~
crimsonalucard
The cost of the super human is irrelevant to the cost of identifying the super
human which is the main point of the overall topic.

You are making the divergent point of "is it worth it to hire super human
coders?"

------
eplanit
These stories are why I'm so glad to have spent my career as a consultant,
where this kind of BS doesn't happen (in my experience, anyway). I wouldn't
dream of applying at any of these places for this reason alone.

~~~
subhobroto
Fair enough but a consultant is an orthogonal role to an employee.

This thread is about employees being frustrated with the current state of
affairs.

A consultant is a business owner.

As a consultant, my deliverables are rarely code. They are strategy documents,
reports, plans, proposals, market surveys and video conferences.

As a contractor, my deliverables are mostly code.

Neither of them is similar to an employee's role, although, as a contractor,
my deliverables match up with those expected of an employee.

Which is why it seems to me that you are conflating consulting with
contracting.

Perhaps you meant to imply you love working as a contractor not having to deal
with what employees have to but employees have to "deal" with it because their
role requires it.

~~~
eplanit
As a consultant, I deliver lots of code in addition to documents. The two are
not mutually exclusive.

It's sad that employees have to deal with BS like "whiteboard exams". Their
role "requires" it only because their employers have followed that trend like
lemmings.

So, yes I'm very happy to get hired (as a consultant) based on merit and
proven experience.

I would not call it "orthogonal" (though that does sound smart in those
documents), because it's more akin to a parallel than an orthogonal role.

~~~
subhobroto
> Their role "requires" it only because their employers have followed that
> trend like lemmings

You speak as if the employer is a different entity than employees.

it's a collective. a composition of employees.

The employees bring it upon themselves.

Employees have to deal with BS like whiteboard exams because the employees
themselves conduct the whiteboard exams!

The reason, why you are not subject to the rules of the employee is because
you are playing a completely different game.

As the employees waste each other's time doing pointless pushups to prove how
good of a fit they are at a moving company (after all a physically fit person
can move more stuff per hour than one less fit!) you help sign up more
customers who need to be moved and make it a more pleasant experience so the
same customers remember your client.

It might be that you end up doing a few pushups or even move boxes but the
real value is in the above.

While every employee at a moving company is focused on maximizing their push
up count, you make your client come ahead in the market by retaining more
customers and sign more up. then repeat

------
mns
Here we go again. As this seems to pop up quite often, it's seems like it's an
important point for companies and candidates. I think that there are good take
at home coding tests, and there are bad ones, there are good in person coding
interviews and there are bad ones.

For me personally, I always liked take at home tests, and we are doing this at
my current company. We have a small task with two levels, one to complete
quite fast and the second level is if you really want to show off and spend
more time on it. We use it as the main technical skills evaluation for the in
person interview, because it opens up an actual discussion and we can talk
about reasoning, see how candidates take feedback and how they can explain
what they did. In the same time, there is another department with us that has
a take at home task that I wanted just for laughter to try, it took me a whole
weekend and it was insane. So I can see where some people are coming from. In
the same time, we also had some cases where we paid the people (who requested
it) for the time spent on the task, and we hired one, which was one of the
most worst decisions that we made, because he turned out to be one of the most
selfish and self absorbed people we hired.

In the same time I've been through some ridiculous situations where I was left
alone in a room with no phone allowed, no pc, just write on a paper some SQL
queries based on some printed tables from a spreadsheet, or, in another case
to write on a flip-chart some code to print the first X prime numbers in front
of 3 people. This I also find quite ridiculous, but it was 8-9 years ago.

In the end it's also how you feel as a developer about these things, if you
are ok with take at home tests, maybe the company is a good fit for you and
that is it, if not, you dodged a bullet.

~~~
subhobroto
> We have a small task with two levels, one to complete quite fast and the
> second level is if you really want to show off and spend more time on it. We
> use it as the main technical skills evaluation for the in person interview,
> because it opens up an actual discussion and we can talk about reasoning,
> see how candidates take feedback and how they can explain what they did

This sounds so refreshing and atypical of the standard take home test that I
would love to give this a go.

My email is my handle at gmail.com

No obligation. You don't even have to have a current opening. I want to
honestly give this a go only to experience what this feels like. That would be
value enough for me.

------
fjfaase
I guess this has to do a lot with hiring culture. Here in the Netherlands,
these kind of tests are rare. You get two or three interviews, which are more
about your programming experience and your personal strengths and weaknesses.
It is as much about whether you fit the company and its culture. Sometimes you
are given a tour through the company or are given the opportunity to talk to
some of your potential new colleagues, and usually includes meeting your first
responsible manager. If everything goes well, you get a job offer and a one or
two month trial period, during which both parties can terminate the contract
on any moment. In my life, I only once had to do some coding before a white
board. I felt like not doing very well, but they thought it was great. The
idea that you can select the best person for a job position by testing is not
really based science, because there are so many factors involved that you
cannot assess with some tests either performed on spot or at home.

------
s5ma6n
It is usually very easy to criticize and not provide any alternative
solutions. How should a company process 1000 resumes with a meaningful effort
and find candidates to be called for an on-site interview?

If you think a single on-site interview of a candidate, it usually takes a few
hours of multiple people from every segment of the company (mgmt, development,
HR). Everybody has to prepare beforehand for some time for these interviews as
well. So in the end, every on-site interview is costly for a company.

As an engineer, I dislike attending on-site interviews to hire people because
I see it as a time that is not used optimally.

Finally, IMHO, the solution is to provide both standard-track and fast-track
interviews where exceptional (on paper) candidates can be called directly for
an interview and regular candidates would have multi-stage interviews.

~~~
varjag
If you send out a 1000 4-hour take home tests as a screen and 100 people took
upon it, you just wasted 396 man-hours of other people time. Creating a bunch
of frustration, disappointment, and unnecessarily accelerating heat death of
the Universe.

~~~
fennecfoxen
Outside of _Madoka Magica_ (2011), it's pretty safe to say that no human
activities we undertake today will have any impact on the heat death of the
universe whatsoever.

------
scarface74
I live in a major metropolitan area not on the west coast. When I’m looking
for a job, I change my LinkedIn status to “actively looking”, call a list of
recruiters that I have worked with in the past, and usually have two or three
offers in hand within two weeks to a month.

For whatever level I’m at at the time, usually companies pay within a narrow
band ($10-$20K). Why should I waste time doing an at home coding test when I
can get a job that pays just as much just by answering a few behavioral
interview questions and describe my past career history?

I’m no special snowflake. I’m just your average everyday CRUD/SAAS software
developer/architect/team lead depending on how the wind is blowing.

~~~
spuz
Would you work for a company that spent 0 time vetting you? There has to be a
"right" amount of time for evaluating candidates and the amount of time the
candidate is required to give up when looking for a new job is only one side
of the equation. Unless you want to work somewhere that has no hiring
standards, you should not dismiss the time taken for take-home tests as wasted
without evaluating their effectiveness vs other methods.

~~~
scarface74
I had two offers before I took my previous job.

At Job 1: the interview was a white board interview where he wanted me to
write a merge sort. He asked a few questions about my background and had to
model a database schema and write a few queries to answer done questions.

The 2nd offer was after an interview where I had no language questions, he
asked me about how I would design a development process “if I ruled the
world”, methodology about dealing with legacy code and integrating it with a
mobile app via APIs, my feelings about testing, what trade offs I had to make
based on “business realities”, etc.

The first offer was full time, a shorter commute and paid more.

The second offer was contract to perm, if I translated it to an FTE salary it
paid less (I ended up making more since I got paid by the hour), and it was a
longer commute. They were both supposedly “senior software engineer” jobs.

I took the second offer. I didn’t realize at the time that they were actually
looking for a dev lead. When they bought me on perm they matched the offer of
the first job.

Are you really not able to tell someone’s experience if they have been in the
industry for awhile just by talking to them? The truth is that most developers
are “dark matter developers” doing yet another software as a service CRUD app
or a bespoke internal app that will never see the outside of the company.

------
BlargMcLarg
To attack the take-home test instead of defending it:

The entire problem with these things: the more complex the problem, the longer
the assignment generally takes. Half-an-hour assignments are trivial. 1 day
assignments can range from trivial to absolute bogus. Now do these 10 times
and congrats, you just worked a week or more for free, on things you're not
even sure can be carried over to the future. Most of these companies don't
compensate you for taking the tests and they don't assist you either (time is
money, friend). Most of these take-home tests aren't universally applied,
either: test A at company X is irrelevant to company Y, so they will give you
test B.

If anything, this take-home test is a symptom. A symptom that companies have
gotten lackadaisical thanks to their power, and a symptom that the industry as
a whole needs to evolve. Don't punish capable devs because the guy you hired
yesterday couldn't do FizzBuzz, and pay your talent accordingly (both in money
and respect) so you don't have to resort to these practices in the first
place. Most of all companies are there to take risk, not to give it back to
(potential) employees. They get paid accordingly. If they dump the risk on us,
then they should also lose a piece of the pie. Unfortunately, the reverse is
happening: they have their cake and they are eating it too.

And a personal anecdote: most of the tests I've come across have nothing to do
with the job at hand, and frequently the people able to clear these tests are
overqualified for the job. Don't give a junior a take-home test made for a
senior.

------
remmargorp64
As time goes by, I'm slowly becoming more and more convinced that the only way
to accurately and fairly interview programmers is to give them some real world
contract work assignments that match the kind of work you want them to do at
the job position, and then see how they perform.

This approach has a lot of upsides:

1\. Real work gets done

2\. The developer gets fairly compensated for their efforts

3\. You get to gauge them objectively, based on their actual results

~~~
subhobroto
Bang on!

I have been doing software for a while and there has been no realistic
situation when a business needed a problem solved from scratch without an
collaboration, discussion or feedback within an hour.

I have been in situations where companies needed a solution within an hour and
it almost exclusively involved extreme team work and cooperation.

I made a comment on how it would be cheaper for the employer to hire a
candidate on an at-will basis once the candidate passes a basic skill and
design test:
[https://news.ycombinator.com/item?id=22368572](https://news.ycombinator.com/item?id=22368572)

------
mnm1
A coding exorcise should only be done after the main interview and given only
to finalists. Doing well should give one a huge chance of getting the job. The
last few hires, we gave the test to three people and chose the best candidate
after talking to all of them. Giving a test before even conducting an
interview, a real one not a ten minute phone screen, is just an asshole move
by shitty companies.

~~~
fhbdukfrh
Your approach doesn't save any time or effort for the hiring company though.
It's the in depth interviews that take up so much time. You may not like it
but this is their motivator, not being nice

~~~
mnm1
Sure if they don't know how to interview, they're going to waste their own
time and still not get good candidates (or if they don't have much to offer as
most companies don't). Our interviews were an initial phone screen of I think
half an hour followed by an interview with the team for one hour followed by
the take home test followed by another half hour to an hour interview to
discuss the code and allow for further candidate questions. All interviews are
conducted over a conference call with screen sharing for the code review.

You're right. I don't like it because what you're describing is generally how
shitty companies operate. Often they won't even look at the coding test. Those
companies deserve to get the worst of the worst candidates that they're
optimizing for. They fit into the category above: nothing of value to offer
the candidate.

------
MrQuincle
I don't like them.

However my brother is paying applicants for the home coding test. I think
that's different.

Nothing beats (for me) an online track record.

------
Zekio
Even better when they give you a project with tests in it, where the tests
aren't even able to compile, and they then also want you to write tests which
does the exact same thing as the non working ones

~~~
bberenberg
What if it's paid? We pay people for interview projects where they have full
access to our team and resources. This helps us gain an understanding of how
they interact as well as their technical chops, produces value from most
candidates, and makes sure the candidate is being compensated for using up
their free time to work on this.

~~~
ghaff
You're still using their free time even if they're being compensated. In
addition, there's a decent chance that, assuming you're in the same field as a
current employer, they may be violating their employment contract by taking
money from you. I know people will argue this isn't a big deal and "who will
know?" but it seems an awkward situation to create.

------
msluyter
I worked at a company for a while that had a pretty good take on this, IMHO.

First, the take home test was relevant to the sort of work one might actually
do in a reasonable time frame. (Read a csv input file, hit an api per record
in the csv, then merge that info and output an updated csv.) Could
theoretically be done via shell script; my submission was ~100 lines of
python.

Second, submissions were graded mostly for correctness. Style/structure and
all the other stuff were nice but _usually_ weren't deal breakers, iirc.

Third, the on-site included a session where we reviewed the submission with
the candidate, asked questions, etc... This was pretty instructive, because we
could point out a missed edge case or whatnot and see how they'd modify the
code to handle it.

Fourth, names & personal identifiers were scrubbed from submissions, to avoid
potential bias.

So, it was a better than average process, IMHO. That said, after reviewing a
fair number of coding tests myself, I'd note that there are still a number of
downsides to take home tests:

a) even if the instructions specify that one should take less than N hours,
it's obvious that some applicants spend considerable time on them. So we'd see
solutions of wildly differing quality; some with detailed unit tests,
documentation, Dockerfiles, vagrant files (!), build scripts, etc... etc...
some with just a single file.

b) Even though the primary criteria was correctness, there was some room for
assessment of code quality, and subjective bias of the reviewer was
undoubtedly a factor. A solution that missed an edge case but which was well
structured would generally be fine, but "well structured" is obviously
subjective. Or, although applicants could use "the language of their choice,"
those selecting, say, Java, were at a disadvantage. Most of us were
Python/PHP/Ruby folks and the Java submissions looked generally bloated in
comparison, and seemed to be graded more harshly.

Not sure what a perfect solution would look like. To resolve issue a) one
could impose a strict time limit or use HackerRank or whatnot, but that would
create time pressure and up the anxiety. For b) one could rely on purely
objective criteria, though that may create other unforseen problems.

~~~
subhobroto
> use HackerRank

Please dont. It's a terrible IDE.

My suspicion is that those who do well on HackerRank have invested a lot of
time to be familiar with HackerRank.

Is that a valuable and meaningful longterm strategy and worth the investment
for a developer?

If you were given an hour of paid time and asked to choose between:

1\. learning something valuable and meaningful that would continue to pay
dividends for the rest of your life, or

2\. learning to use a website that you will probably use a dozen times or less
your entire career

... which choice would you, personally, make?

The actual IDE I use at work is nothing like the HackerRank IDE. I have built
up muscle memory that just does not work on the HackerRank IDE.

Have you tried to run a Javascript program within HackerRank? Try out how the
Map implementation works and how it's different from the Map implementation
available when you use Babel.

Timed tests on HackerRank select for candidates who have trained on HackerRank
problems inside the HackerRank IDE.

If that's your perfect candidate don't come complaining that your employees
solve HackerRank problems all day at work.

I have been doing software for a while and there has been no realistic
situation when a business needed a problem solved from scratch without an
collaboration, discussion or feedback within an hour.

I have been in situations where companies needed a solution within an hour and
it almost exclusively involved extreme team work and cooperation.

~~~
msluyter
Yeah, I agree about HackerRank's UI. When I was using it I think I just used
PyCharm and then cut & pasted my solutions.

~~~
subhobroto
Exactly... but even then, how do you run the test cases?

Download them and run them from PyCharm?

I just don't like HackerRank.

Let me rephrase that.

As I grow in my role, I see writing the type of code that can be tested on
HackerRank less and less, so investing time in learning how HackerRank works
is absolutely not on my radar.

... and getting a HackerRank solution in within that ticking timer absolutely
requires me to be comfortable with the HackerRank UI.

So stalemate there.

------
RutZap
This has really annoyed me recently, I've had interviews with companies that
expected me to do a test before even seeing them in person (following a phone
interview) and their tasks mentioned I should spend around 8 hours on the
test. I think this is definitely unreasonable.

I prefer longer interviews where you're working alongside your potential
future colleagues. This way you can also find out if you actually want to work
with them or not. Interviews should be a two way affair.

~~~
b0rsuk
The phone and online coding exercises are called pre-screening. Their goal is
not to find out if you're a good programmer, but if you can write code at all.
This is to save time and see as many candidates as possible.

~~~
RutZap
Yes, i get the screening process; but asking a candidate to spend 8 hours on
implementing a client library for their API at the beginning of the
application process is definitely unreasonable. It's not a coding exercise,
but a full blown deliverable.

------
thih9
Alternatively: ”How desperate are you to work here?” or ”Are you willing to
spend your free time doing unpaid work?”

------
Ocerge
I've given up fighting white boarding. If anything, at least it can be gamed,
plus if their expectations are bad or you got bad luck on a topic you didn't
study for, at least you get to waste their time as well. I'll take that over
wasting only my own time on a take-home test.

------
altmind
i would rather take a coding test than suffer an unknown number of office-
visits, talking with people who are not interested in hiring, who are stuck
talking with me and who are adamant to stop getting interrupted and get back
to work.

as bad as coding interviews can go, they are better than spending more than a
day to talk to people who evaluate you on how good you are at talking.

------
crimsonalucard
>Or: here's a whiteboard, implement some algorithm you last studied 10 years
ago and that you will never use in front of 4 people staring at/judging you.
You have 5 minutes. :D

I agree with the fact that this style of testing is not directly measuring the
skills relevant to the job.

Say what you want, however, a candidate that does solid on the whiteboard
despite the irrelevancy of it, is a good candidate because a whiteboard is one
hundred times harder than the take home test and most of the problems that
will be thrown at the candidate on the job.

