
Why Developers Hate Coding Skills Tests and What Hiring Managers Can Do - rmason
https://hackernoon.com/why-developers-hate-coding-skills-8m6u3za1
======
fsniper
I am happy for people who has energy, free time, and will to prove their
abilities.

I don't have the will, or energy for that.

I have been working in the area like for the approx. last 2 decades. If anyone
wants to validate if I can really code, can put 1 hour of their time and go
over my github/gitlab profiles instead of asking 1 hour of my free time to
redo what I was doing professionally.

This will be counted against their paid time, but not mine. This is asking
some sacrifice on my part for a process everyone knows is highly subjective,
and not really doing the expected.

And more to that, while working I solve real problems of the business. I do
work with colleagues. I do use as much time as needed to do research. I don't
get on a dumb IDE which is not integrated to my workflow, or tooling and do
puzzles with illogical limits.

~~~
ChrisMarshallNY
I have over 100,000 LoC in the public domain, in over 30 repos (the ones I'm
highlighting). I have architected, deployed and shepherded large-scale hybrid
systems that are currently in use daily by thousands of people around the
world.

I have a decade of checkins and mineable Git data.

I have solved every single problem that I've ever encountered; whether
technical or human (I was a manager for a long time).

I have convinced skeptical, conservative, empirical Japanese managers to fund
and support development strategies, based mostly on their confidence in me,
and my Integrity.

I have hundreds of pages of articles I've written in Medium and my own
(multiple) sites.

I have multiple sites that I have designed, and hundreds of pages of
instruction, along with entire courses and presentations.

I can trot out a long line of personal and professional references that
include CEOs, VPs, lawyers, doctors, consultants, politicians and other
engineers, all of whom will sing my praises to the stars -on the record.

A big reason for that, is the reason I'm not extremely well-known (outside of
a certain narrow demographic). I have done most of my engineering over the
last decade in Service and open-source projects that aren't about making
money; but helping others.

However, when I say this, the answer I get is "IF what you say is true..."

Why the hell would anyone make a claim like that without backup? That's crazy.

OF COURSE I can back it up. My SO Story is a mile long, with links to
everything.

Damn straight, there's folks better than me at pretty much any branch of what
I do, but I know few that can offer the depth of knowledge across the
spectrum, like I can.

AND I CAN PROVE IT.

But you'll make your decision based on a half-hour, fifty-line binary tree
test that a college student would ace better than me.

I want to see lawyers do legal briefs as part of their interview. I want to
see doctors do "trial surgeries." I want to see HR managers do "how would you
handle this?" tests as the ONLY variable in their hiring process, while having
their portfolios ignored.

After a few months of this crap, I just gave up, and started my own gig. It's
working well, but slowly.

I would have been fine, taking a low salary, if the work interested me. My
retirement's set. I have a couple of companies that I can use to keep my toys
going.

But good luck with the "Draw Spunky" matchbook tests. I'm sure that you'll get
top-shelf talent, that way.

~~~
baron_harkonnen
While I completely agree (and sympathize) with this frustration, there are two
things I want to point out.

1\. If you can do all these things, then it really won't take that long to
freshen up your your white board skills so that you can solve a variety of
algorithmic problems easily in code on a whiteboard. If you really want to
work for at a FAANG-style place, then just divert your energy for a bit into
practicing white boarding. I agree it's stupid and it's probably not worth
your time... which leads me to point two.

2\. Given your experience and interests, it sounds like working as a cog in a
mega-corp is probably _not_ the right career path for you. You're going to do
much better in a place where your role is a much bigger part of the company,
and your individual skills will have a much bigger impact.

And as far as the "other professions don't have to do this" line. I worked for
a while in a career that was not software engineers and to be honest, I wish
there was basic, practical proficiency tests as part of interviews in more
fields. The number of professionals in my last field of work that could not do
the basic requirements of the field correctly was horrifying.

~~~
inimino
The takeaway here isn't for people looking for these jobs (yes you can bite
the bullet and conform to any stupid process) but rather for hiring managers
that if you actually want what you say you want (diversity, hiring for ability
to do the work, etc) then you should not be hiring in this stupid broken way
that we all know doesn't work and doesn't give you those things.

~~~
dragonwriter
> for hiring managers that if you actually want what you say you want

They don't; what they say they want is pure PR aimed at creating an image of
both fit to the perceived culture of the developer community and compliance
with the law.

> then you should not be hiring in this stupid broken way that we all know
> doesn't work and doesn't give you those things

Of course. The fact that they continue to do despite the fact that it is well-
known not to produce what their PR says they want is pretty compelling
evidence that their PR is nothing more than that. But as long as enough people
are willing to both play the game and believe the PR, there's no incentive to
change.

------
dmoy
Sure, yes. I don't like coding interviews that focus on super complicated
algorithms where you get a massive advantage from already knowing the subject
area, or require a sort of "a ha" realization to solve properly.

When I ask coding questions they tend to be fairly straightforward. Do some
basic transformations on data, handle a couple of edge cases (bonus points if
you can see the edges of your own code), that's about it.

Don't have candidates redo kmp or a read black tree from scratch on a
whiteboard in 30 minutes. That doesn't tell you much about the candidate.

On the extreme other end though, don't use interview questions that rely on a
huge amount of background or domain knowledge for your specific team (unless
you _really_ need an expert asap and can't ramp them up). You can't expect
candidates to burn more than a day on your company.

~~~
lhnz
The problem with this approach is that interview nerves don't just stop you
from writing complicated algorithms, they often cause people to forget or
stutter over trivial/simplistic stuff like syntax or popular APIs.

For example, recently I did a simple algorithmic coding test in JavaScript.
The overall logic I wrote was completely correct, but there were two issues:
(1) I accidentally missed a function call (which I caught later on after re-
reading the code), and (2) I completely forgot about the API
`Array#slice(start, end)` so had to recreate it using a loop.

When I'm doing this kind of test, often I get very dizzy and can't think
straight. Once I wrote the correct code, but couldn't read the output that was
printed. The interviewer remarked that the code was correct, and that's how I
knew I'd not made a mistake. That is ridiculous: I was capable of writing the
logic, but lost the ability to read. The only other thing which makes me feel
this way, is public speaking.

I have a decade of experience, and can generally solve the problem. It's
demoralising that despite years of experience, I forget the most trivial
APIs/syntax. I've literally done this every day for years of my life. I have
no idea how I'm meant to prepare for this, because I've clearly practiced
enough, and I am losing fundamental cognitive abilities.

In general, I am much better if: nobody is watching me while I do the test,
and I have enough time for my most intense nerves to pass. Or if it is a take-
home coding test.

I know sometimes people wish to argue that "you should be able to code under
pressure', but I was able to meet crazy deadlines at startups by working fast.
There's something different about having only 30 minutes, and somebody
watching you screen, judging your every move.

~~~
nobodyandproud
Do more interviews. It’s the only way to get used to it.

~~~
hobs
This "might" work - but there's plenty of people that no matter the number of
interviews they flop sweat, forget, and generally feel fight or flight
reactions, though medication might help.

I have trained a lot of people and you see similar stuff with test anxiety -
even if it is the easiest, open book, no pressure situation, I have had people
openly weep and quit their job - the internal pressure is just too great.

------
blunte
I float between three or four programming languages on a fairly regular basis
(within a year), and I use or explore as many different frameworks depending
on whether I'm fixing something from the past or researching something to use
for the future. Similarly, I may have to go back and do some OOPish work, or I
may be doing FP, or I may be doing plain integration glue scripting.

There is no coding test which will identify the value I provide to clients. A
coding test will make me look bad. However, if you instead give me a "how
would you solve this" question, and I will ask questions and discuss
strategies, then I will look very skilled and experienced.

So yeah, I tend to take offense when companies want to test me with a "google
style" (as far as they believe) interview. Frankly, if I'm going to have to
pass a Google Style interview, then I'll just interview at Google and get a
better job with a higher salary than what company Y is offering.

I understand that for companies, hiring good developers is difficult. But I
would argue that the supposed quality of the developers is only part of the
equation, and that growing good managers is an even bigger problem. A great
developer can die (or quit) under a bad manager, and a mediocre or weak
developer can grow into a decent developer under a good manager.

~~~
sieabahlpark
Yes but at the same time those are also the easier questions to answer in the
first place. The goal isn't always to finish solving the problem but to
evaluate how you solve the problem.

~~~
blunte
I agree, but I'm sure you can identify with having high standards for oneself.
It's rather demoralizing to be very experienced and feel like a moron for a
good part of an hour in front of someone who _must_ be skilled or else they
wouldn't be giving you the coding test.

------
neilwilson
As usual the managers and marketeers forget the obvious point.

You are asking a professional developer to write code for you. But apparently
you expect them to do that for free.

In a supply constrained market, that's not how it works. If you want to do
code assessments with professional developers, and therefore you don't want to
do the leg work looking at and assessing their Github contributions, then you
have to expect to pay them for their time.

Otherwise why should anybody who has fifteen options on their table waste
their free time on you?

There was a time during the full employment era that people were paid expenses
to attend interviews. It was considered good manners to respect people's time.
That common courtesy seems to have gone by the wayside during the shortage of
jobs era. Now the tables have turned again.

------
empath75
Our coding test actually revolves around things you would actually do as a
coder — we give you a simple python script and ask you to identify a couple of
problems with it, then read a PR that attempts to fix one of them and leave
comments on it, then add a simple new feature, write a test and documentation
for it and submit a PR. The actual coding part is fairly trivial and related
to the work we do (interacting with the aws api)

We have a bunch of notes for the interviewer with common questions the
applicants ask.

The whole thing takes like an hour, and it’s not a gotcha or brain teaser.
It’s more about whether you understand requirements and can work with other
people.

~~~
blunte
This sounds like a coding test I would actually enjoy - solving problems.

I haven't done a lot of tests, but I have yet to do one like what you're
describing.

------
Youden
Something that bothers me: Most of these tests are looking for the same
things, so why do I have to take the test for each company I apply for? It
strikes me that this is a similar problem to getting into university and for
that there are standardized tests.

Why can't we introduce a standardized coding skills exam where everyone is
given the same problems and the same amount of time to solve them? The whole
industry can collaborate and learn from each other to make a single test that
gives the best hiring signal.

I'm under no illusion that such a thing would be trivial to do but it seems
feasible. You get rid of the resume check, replace the recruiter call with a
first-stage multiple choice exam that gates the rest of the process and
replace the coding interview with a long-form exam with open questions. The
long-form exams can be graded by the engineers that would otherwise be doing
interviews but distributed amongst all the companies involved.

Once you've got all the technical stuff out of the way, you can do a single
onsite interview for culture fit.

Seems like this is a win for everyone. Candidates can do the coding portion
once instead of say 5 times, those 5 companies only have to evaluate a fifth
of the candidates' exams (since the grading is shared amongst them) and you no
longer have the problem of each candidate getting a completely different
process to every other candidate.

~~~
ldoughty
Because then people will cheat and have a friend do the test for you. I have
people pretending to have certifications they don't have.. or actually having
the certification but somehow can't answer non obscure facts... (How can you
be AWS solution architect, verified certificate, but not know that the general
purpose EC2 instances is... Or what AWS service provides script execution in
languages like python or node? And what service they provide for logging? I
had a candidate that failed all 3 questions... My only thought is someone took
the exam for him...

Other than that, there is teaching to the test. You can easily argue that's
what happens now, and most systems suffer for it. People learn concepts they
are required to, but rarely good application of those concepts because they
only have 90 minutes to cover a large topic like composition vs inheritance.
You get one or two examples about animals barking and cars. Good luck figuring
out how to apply that to business topics!

~~~
Youden
> Because then people will cheat and have a friend do the test for you.

Isn't this problem shared by all exams? How often do people successfully cheat
on their university/college entrance exams?

And hell, can't you potentially do this with a technical interview? Nobody who
interviewed you is actually going to see you again and I don't recall an ID
check at any of my interviews.

> Other than that, there is teaching to the test. You can easily argue that's
> what happens now, and most systems suffer for it.

Exactly, we already have "cracking the coding interview" and similar tools. I
think this approach would be better because the questions can always be
prepared for a particular year's interview from scratch, so there's no chance
they've been seen before.

~~~
Izkata
> And hell, can't you potentially do this with a technical interview? Nobody
> who interviewed you is actually going to see you again and I don't recall an
> ID check at any of my interviews.

I don't think this is reliable. One of my interviewers became my manager; we
make it a point to hire for a specific team and get someone on that team in
the interview.

------
calpaterson
Developers hate them because coding tests have devolved into multi-hour,
unpaid assignments from which no feedback is given. It's especially bad if
they are unwilling (as some seem to be) to have a short phone conversation
prior to the coding test to tell you about the project.

~~~
freddie_mercury
> It's especially bad if they are unwilling (as some seem to be) to have a
> short phone conversation prior to the coding test to tell you about the
> project.

I'm sympathetic. But the last time I put out a job ad I got over 100
applications for a single opening. How should companies try to narrow that to
a manageable number?

Just look at the CV and throw it out based on that? What do you look for on
the CV that cuts out 80%+ on candidates to get the number down to an amount
that I could have a short phone conversation with?

~~~
rleigh
Yes, that's the exact reason you got those CVs in the first place. To assess
their suitability. To require every applicant to waste their time to do some
test to save you reading them is phenomenally lazy. Do that after you've done
an initial pass over the CVs, and had a conversation with them.

Your job opening should have had some sort of "job specification" with
"essential" and "desirable" criteria. Take a quick pass over all the CVs. If
they don't meet all essential criteria, then put them in a reject pile. For
the remaining CVs, order them by how many of the desirable criteria they meet,
then take a cut of the top _n_ to contact. If you don't have the time to do
this, then HR should. This won't take long; it's a quick yes/no. You look at
the remaining CVs in detail after that initial pass and take into account
their education, experience etc.

Any candidate worth their salt will already have tailored their CV for the job
specification to make sure that the essential and desirable criteria are
clearly shown on the CV to make your job easier. If it's not clear, then
reject it. You'll have that number down to a shortlist of 5 in 60 minutes.

------
dep_b
I work mainly with Swift nowadays and some stuff like String handling is way
more complex than in other languages like C#. Since I'm not doing a lot of
series-of-characters manipulations in my daily work I need to rely on Google
and a friendly IDE that highlights errors and warnings and provides
autocompletion through get through tests like simple anagrams. Codility and co
have neither.

Nothing worse than unsuccessfully sweating through an exercise for 20 minutes
on something super simple, only to fix it within a Playground in a minute just
because you get errors, warnings and autocomplete.

------
jimbob45
I don't hate them. What I hate is getting filtered out by HR people with no
comprehension of the CS field. Rejecting me on sight for a Java position when
I have 5+ years of C# experience is just silly.

------
odyssey7
An illustration:

Recently a recruiter asked me to do a coding challenge. He told me to spend
twenty minutes on it. Three hours after starting, I had completed the six
questions that covered things like dynamic programming, converting between
integers and Roman numerals, and processing data from csv files; and I had
figured out why their submission script was causing a stack overflow and let
them know about the bug.

The code was clean, it passed all but a few of the hidden test cases, and the
recruiter told me I scored well; and I could get back on the work I was now
behind on for the day because of my naïveté. Two weeks later, the company
didn't move forward with an interview.

Why is my reaction to coding challenges an allergic one? A coding challenge
(completed alone) is a cheap and wide net that is can be costly for the
engineers who fall into it - and yes, I’ve already done enough of them to know
something is fishy.

What if I’m about to spend my best hours of the day producing free,
production-quality code, and you aren’t going to honor that with something as
small as a phone call from the decision maker to talk afterward? Keep in mind,
usually the discussion at this point has been: “Hello.” “Hi.” <Link to coding
challenge />. These conversations don’t look very different, and, empirically,
continuing doesn’t have a great expected value.

~~~
sys_64738
Bill the company for your time doing these tests. $250 is the going rate for
contracting.

------
PebblesHD
On the other hand, 2 hours for a tertiary interview after already spending
time out is a reasonably significant investment of time for someone who likely
already has a position and which I’d suggest isn’t widely required for
positions outside of technology. Certainly no management position has a 2 hour
examination on processes or people retention strategies in addition to the
usual coffee shop discussion or panel interview. I’d be more in the line of
reviewing a GitHub if available or a sample application or even just some
concepts, implementation can happen in a range of languages but as long as the
core is solid that’s what I’d be focusing on that.

The most egregious version of this I’ve seen was an interview for an
infrastructure position where the challenge was a generic coding test. That’s
fantastic if you want a generic developer but it won’t tell you anything about
architectural patterns or common sense. In any case, I’m just a developer and
these tools are usually aimed at managers, and since this can at least give a
non-technical person a hint of whether a candidate is suitable I guess that’s
valuable on its own. I’d be very interested in the tech behind the product
offered by this team to evaluate the candidates though...

~~~
colechristensen
The absolute worst I experienced was an automatically scored coding test.

Given a problem statement and an amount of time, write code and submit against
a test suite to be scored.

Half of the tests provided zero information as to what was being tested or
what failed, so one was left with a black box to guess what was wrong.

~~~
marcinzm
The ones I’ve seen let you add your own test cases (for debugging, not
scoring) and expect you to do so. Which seems reasonable to me, on the job you
won’t have someone to write all the tests for you.

------
newshorts
My first interview was at a marketing company in Boulder, CO.

They stuck me on one side of a long table with all the other developers in the
other side.

They proceeded to grill me on obscure gotchas and had me white board fucking
solves for shit like palindromes.

At one point one of the developers stood up and said “look, it’s so easy!” And
proceeded to write out his obviously premeditated/optimized one liner on the
white board.

I didn’t know enough to respond with something like “I don’t write over
optimized one liners because even I won’t remember what the fuck is going on 6
months from now”.

I remember walking back to my car so ashamed and embarrassed.

Interviews are supposed to be a prediction of your fit in the culture and
ability to contribute.

In the past 10 years I’ve written lots of code, but more importantly I’ve
shown up as a member of my team.

In the end, those interviews are good tests. Tests of culture within the
company you’re interviewing...

~~~
sircastor
Reading some comments here, and having recently gone through a number of
interviews, I'm contemplating taking my own CS questions to which I know the
answers to see how the interviewers solve problems, and what happens when they
don't have the answers to a given problem.

I'm not sure how it would be received

------
mnm1
The spray and pray approach is the default even for great coders unless they
have contacts that will get them a job with almost no interview in which case
none of this applies. As far as coding tests go, they should only be
administered to finalist candidates after the initial interviews. So no, there
shouldn't be automated scoring. That's an indicator that the test is flawed.
The candidate should be able to walk through the code and explain why they did
what they did. They should be fairly confident that if they do well on the
test and subsequent walkthrough, they are getting the job. Companies that
administer the test first either have a ton of qualified applicants competing
for the same job or are doing something wrong.

------
JoeAltmaier
The argument that you wouldn't hire a magician without asking them to do
tricks is laughable. Magicians are hired on their reputation. Like all
professionals - doctor, lawyer, even plumber. You don't say "Doc, treat my
bunion first, then I'll see if I want you to work on my cancer". Its
preposterous.

Except coders get asked. One fool writes this idea down in a blog somewhere,
and we're all saddled with it for a generation.

I've never given in to it. You want to hire me (and I'm always busy), you talk
to a reference and make your decision.

~~~
tluyben2
> One fool writes ...

Was that not Google? I mean I have never seen it in reality; like you, I would
not entertain that kind of nonsense and no-one has ever asked me, but did this
happen at scale before Google became famous for it?

------
acconrad
As a hiring manager I'm literally losing sleep over this problem.

I've been on both sides of this now and it's such a delicate balance.

As a developer, you want to be fairly evaluated to prove your worth.

As a hiring manager, you want to make sure you don't hire a dud (or worse,
someone that will drag down the rest of your team).

I spend more than 1/2 of my day now trying to build out a team that I need to
aggressively hire for. I've radically revamped our hiring questions to ensure
they are:

* Based on real-world code. That means coding up components if you're a front end developer or API contracts if you're a back end developer.

* Includes a breadth of tasks. PR reviews, writing code, writing tests, and systems design are all a part of the assessment - not just algorithms tests.

* Has clear acceptance criteria. I want to remove as much bias and subjectivity from the assessment as possible. Every question we use has a Wiki page designed to formulate the question, the unit tests, and sample answers as well as grading criteria depending on the level we're hiring for. Expectations for Principal engineers are different than those for mid-level engineers.

It's not perfect. I'm not perfect. But I'm trying. _SO_ hard. I hope that
we're resolving this problem in the best way possible for both us as a company
and you as the candidate.

~~~
NeedMoreTea
Hmm, that sounds more than a little onerous. How many expected hours/days does
that translate to?

> As a developer, you want to be fairly evaluated to prove your worth.

Indeed, but what I don't expect is to give a day or three to a potential
employer knowing full well that most applications are going to be rejected.
When a vacancy often gets 100+ applications it's clear people are going to be
rejected for the most tenuous and marginal reasons.

If I got the job, no problem, but if not and I have another 20 applications
with a day's coding hoops to jump through each time, that's an awful lot of
evenings or weekends potentially being burnt...

~~~
acconrad
> How many expected hours/days does that translate to?

The PR review should take 20-30 min. The phone call before the on site is 45
min. The on site is 5 hrs spread across 5 interviews and includes time for
lunch

~~~
NeedMoreTea
That's a fair bit less than I envisaged. Fair play. :)

------
fit2rule
Gave up doing coding interviews long, long ago. Nowadays the only really
effective measure of how useful a person is going to be in the work
environment is to actually put them in that work environment for a day and let
them contribute as best they can.

This always filters out the incompetent from the productive. If we can't
survive a day with you in our environment, games over. If however, you survive
a day in our environment and still want the job - game on.

~~~
firepoet
Do you pay candidates for the day they spend working for you?

~~~
fit2rule
Sure.

------
hysan
> The employer is almost always taking on more risk than the
> applicant—particularly in smaller companies.

I had my reservations with the “solutions” and reasoning proposed in the
article up until this point. This is a “Sorry, you are wrong” moment for me.

Much of this article is based on the premise that this is solving for hiring
_Senior Engineers_. If you are in need of one or one is interested in your
company, then there is equally high risk for the engineer. They are
potentially giving up a stable job, uprooting their family, putting a job hop
on their resume, etc. That’s a lot of financial risk. There’s also the risk of
_your_ company not being as good as they hoped. Yes, that comes down to
negotiations, but you are no where near that stage at this point in the
evaluation process.

Again, I must reiterate that this is wrong because this article is geared
towards _Senior Engineers_. Juniors and Mids, fine. I can accept that.
However, for an article that started out trying to use data from interviews to
understand both perspectives, it started skewing heavily towards the
employer’s side by the end.

------
GlennS
I like coding tests. They give me an opportunity to differentiate myself.

~~~
blunte
And some "musicians" love to play scales, and can play them really well.

~~~
viraptor
There are tests (silly logic puzzles or rehashing cs) and tests (actually
useful practice evaluation). Let's assume the positive side of this idea.

~~~
blunte
I cannot assume that since most of my (personal, anecdotal) experience has
been "rehashing cs".

I did have one fun test, but it wasn't live; it was a do-overnight thing,
creating a URL shortener using Elixir and GenServer. I learned a lot, and I
actually enjoyed solving the well-defined challenge. But that's certainly not
something I've seen in a 1-2hr live coding test.

------
ytch
> When a candidate does run into an interview that represents real-world
> problems that would need to be solved in the day-to-day of the actual
> position it’s MIND-BLOWING

But sometimes if I was asked to solved a complicated problem as a take-home
test during interview, I might be hesitated to do my best because they might
steal my ideas for free like this[1].

[1]
[https://www.reddit.com/r/jobs/comments/4tutqr/potential_empl...](https://www.reddit.com/r/jobs/comments/4tutqr/potential_employer_took_my_ideas_presented_during/)

------
chucke
This article focuses too much on the developer role in the coding skills
challenge, portraying the companies as bystanders waiting for a result.
Usually the problem comes with the evaluation. If you're asking a senior to
bring a different way of thinking, but you ask your team of mid to senior dev,
who never worked anywhere else for the matter, to evaluate, you're gonna most
often get an evaluation based on their current way of looking at things, and
you'll never get anyone to push the team forward.

~~~
viraptor
I'm not sure that generalises. I'm in the mid-senior range and the most
interesting interviews I've done were those where I learned about something
new. Even if people came up with ideas I didn't agree with, they got bonus
points for justifying them or considering the pros/cons. And I know I'm not
the only person that appreciated it.

------
JansjoFromIkea
I don't mind coding skill tests, I'm generally pretty clear about what I've
cut corners on because I can't be arsed spending too much time on them and
it's usually accepted (I don't think I've gotten an outright no following a
coding test in my most recent wave of applications). That being said, I do
point out that my github has other stuff they can look at.

What I DO dislike is ones which have failed to explain themselves clearly
which will result in someone who probably wasn't a good fit for the role
wasting far too much time on trying to get something to work that wasn't set
up clearly enough in the spec. For this reason I think trick questions and
crap like that are the absolute devil.

To me it seems like a big chunk of the reason for the tests is so there's
something to ask in the interviews though. The technical parts of almost every
interview I done in the last few weeks centered around the tests (in one case
I had done them in such a rush I couldn't actually remember my code). I think
a lot of interviewers begrudgingly support keeping the tests in their company
so they don't have to become better interviewers.

------
gorzynsk
There’s stupid simple way to test interview process - let your current
engineers go through it. I bet best performing ones will fail mentioned
algorithmic and abstract assignments.

Sometimes the best value to the team bring a person who doesn’t know
algorithms but can e.g. foresee problems or ask “stupid” questions which lead
others to think differently.

------
wolfgke
In my opinion, simple answer why developers hate codking skills tests is: they
have a huge opportunity cost ([1], see also [2] and [3] for a more humorous
explanation) for the person to be tested. Or, put in more simple words: in the
time that a coding skill test takes, you could earn money.

[1]
[https://en.wikipedia.org/w/index.php?title=Opportunity_cost&...](https://en.wikipedia.org/w/index.php?title=Opportunity_cost&oldid=906612366)

[2] [https://www.smbc-comics.com/comic/2012-12-29](https://www.smbc-
comics.com/comic/2012-12-29)

[3] [https://www.smbc-comics.com/comic/2014-04-20](https://www.smbc-
comics.com/comic/2014-04-20)

------
ahaferburg
The reason why I hate it is because of how disrespectful it is. I would be
fine with a 20-30 minutes test, but if it says "Don't spend more than one day
on this" I would walk.

Writing a resume is fine, because I can copy paste it. Interviewing is fine
because it's me talking to another person. They invest time, I invest time. I
would even be fine with "solve this thing, print it out, bring it to the
interview so we can talk about it". That would be a good alternative to a
whiteboard interview.

But doing a coding test for multiple hours without any compensation, and with
the very real possibility of getting ghosted for whatever reason... That is
just unacceptable.

~~~
pry86
I agree, but what to do? In last 10 years I changed the job few times already,
had many interviews, and 50-60 % of the companies have coding tests. Some even
two (first some algorithms, then interview, and then "proper" business coding
related challenge). Some are easy and require 2-4 hours, the longest ones were
like 20-30 hours of coding (loads of requirements). I see this more and more
(in Germany), especially in the companies which "import" developers from
easter Europe, Asia or South America.

------
fivre
Although not for coding, I like our take-homes for support and sales
engineering staff.

The take-home asks candidates to complete a basic installation of our product
and document what they did in detail. The latter part forces candidates to
provide a writing sample in the domain they'd be working in.

The one problem is that the writeup, which I'd find quite useful in evaluating
a candidate, is just thrown into the bin as best I can tell. It's not
available in our applicant tracking system, and HR doesn't provide it to
interviewers along with other interview materials. I'm not actually sure
anyone reviews it at all.

------
xs83
I think if a developer has nothing in the public domain nor can demonstrate
some code they have written to you in person then that is the only time a code
test is valid. I agree with many here - I have over 2 decades of experience
writing code, I see no reason why I should waste an hour of my time writing
something asinine that I probably did as part of my university course
(Palindrome detectors etc) when you could easily see my capabilities online,
hell I have even linked to them all for you on my LinkedIn....

------
tluyben2
I hire people who are recommended to me and I get contracted by people who
recommended me. This has not changed since the first project I got (almost 30
years ago; I was recommended by my high school physics teacher). If that is
not enough, then I am not very interested; it definitely tells more about me
than some test. And the other way around; I have never had to fire an engineer
that was recommended this way to me. They were always great to work with.

------
amriksohata
What if I told you coding tests were to get the developer to talk through a
problem and see how they go about it, rather than actually see them finish it?

~~~
codingslave
I would tell you all the tough to get into companies only hire candidates that
answer correctly.

~~~
monster_group
I agree. "We see the candidate's thought process / approach" is all bullshit.
If you didn't solve the problem, you are not getting in. It also doesn't
matter how you perform in the non-coding (system-design and leadership /
behavioral) interviews if you didn't solve the coding problems correctly.

------
jaimex2
Whats the tldr? I'm half way through this article and other than talk in
circles it hasn't said a single thing that's meaningful

------
vinay_ys
There are two kinds of programming work.

First kind is that of translating application logic into code using the pre-
established frameworks and patterns. This involves application domain
understanding and modeling of domain into system concepts exposed by the
application platform's underlying frameworks and components.

Second kind is building these frameworks and components and assembling them
into productive platforms with right system concepts that nicely solves for
the application domain.

Why these two kinds? Because the nature (volume, churn, complexity axis) of
the work differs significantly across this cut so much that the background
experience and skills needed by the programmer for these two are very
different.

In an interview, you cannot really verify the direct skills needed for either
of these two types of jobs directly. Instead you can look for proxies. What
are those:

1\. Programming proficiency in a programming language of candidate's choice –
you can verify this with an in-person 2-hour coding round within the interview
loop. Goal of this round is to ask the candidate to demonstrate his mastery
over the programming language of his choice by converting a given application
logic into functioning programming. You can do this by giving a programming
environment (developer laptop with Internet connection) with a written down
problem statement with application logic and test input/output samples. After
the candidate has spent 60-90 minutes on arriving at a working solution on
their own, spend another 30 minutes doing a live code review and then asking
the candidate to make one or two modifications (enhancements) which they can
take another hour to complete. Clearly state the evaluation criteria –
completeness of functionality, test cases, readability of code (passing a
basic code review) etc. For candidate it is 2.5 hours, for interviewer it is
1-hour.

2\. Expertise in relevant area – if the candidate has previously worked on
either the same/similar application domain or built platforms for same/similar
problem space, then that can be explored through multiple whiteboard
discussions focusing on specific aspects of this supposed expertise.

3\. Learnability and Problem-solving – most important skill in software
industry is ability to figure things out from vague undefined scenarios and
learn new things needed for doing so. Problem solving is being able to reason
through a set of stated and unstated conditions/constraints to arrive at an
acceptable way forward in any situation. These can be assessed through
interview conversations as well as reference checks, and inferred from their
work experience and accomplishments.

The above skills list is in increasing order of priority and their relative
importance ratio increases for more senior developers.

The problem with automated coding tests is they only test for the basic
programming skill and the evaluation criteria is not nuanced and the decision
is usually harsh/hard/final. This is a waste of everyone time, but it
definitely far worse for the candidate. Hence any smart and reasonably senior
candidate would refuse to participate in interview loops that have these
automated tests.

------
austincheney
I like coding skills tests because its a quick filter to eliminate people who
shouldn't be there in the first place. A kind of _don 't waste my time and I
won't waste yours_ type scenario. On the other hand I will never take another
hiring code assignment again unless I see the grading criteria up front.

These sorts of hiring filters are not something that can be solved for
industry wide, because many developers don't want a common solution. Otherwise
the solution is simple, because absolutely every other industry has solved
this problem.

The solution is universal, objective, screen criteria backed by industry,
which is also known as licensing. Using a test prove that developers have
required knowledge, can read code (this is the big one), and can make certain
basic decisions. In addition to testing also require documented experience
from trusted and validated sources, continuing education, and some form of
educational baseline (not necessary academic).

Absolutely every other industry has figured this out. Licensing isn't magic
and doesn't create talent, but it is an excellent initial filter.

\---

When looking at potential employers there are two things I look for now:
subjectivity and selectivity. If the potential employer is not very about
skills assessments I get nervous they are looking for the wrong qualities in
applicants such that they may contain a lot of dead weight on their existing
teams.

On the other hand if they are selective about skills, but that selection is
purely subjective I go from feeling nervous to thinking they are phony. I get
the feeling I am on a bad date that will end up as a broken marriage. The
hiring party has absolutely no idea what they want, but they have some
primitive notions about what they don't want. The common root cause of
subjective nonsense is because the hiring party is insecure and seeks to
qualify some level of comfort unrelated to the skills tested.

The reason why many developers don't want licensing is because many developers
lack the skills necessary to complete such licensing criteria. They would be
out of the job. This isn't true just of applicants or aspiring developers, but
also true for many people performing hiring.

In a recent interview I had to prove I could read code in a language I have
never written in before over the course of about 90 minutes, and it went very
well. The goal was to prove I could quickly read/write original logic after
getting over some basic syntax. The hiring party knew exactly what they wanted
and were very clear in achievement criteria. Everything was pretty objective
in that this process was a testing to see how far a candidate could get into a
given problem in the time allotted and the effort that candidate would put
into refactoring. All other technical qualities were ignored to reduce
selective bias.

