
Why Coding Tests Are A Bad Interview Technique - fogus
http://www.brandonsavage.net/why-coding-tests-are-a-bad-interview-technique/
======
edw519
_My biggest pet peeve with these types of tests is this: there are a lot of
companies out there, and I’m sending out resumes to each one that I can find._

That's your first mistake.

Why should a company take you seriously if you don't take them seriously?
Every resume you send out should be custom written and carefully crafted to
address _their_ requirements, not yours. Do you do any research into the
companies you're applying to? You should.

Rather than sending out 50 stock resumes, you'd probably be better off picking
the top half dozen or so opportunities and do everything you can to stand out.
This includes a custom cover letter, follow up emails, thank you notes, etc.,
whatever it takes to make the connection.

 _I simply do not have the time to write fifteen code samples a day, just
because you want to evaluate me against your coding test. Period._

There's your second mistake. Misdirected attitude. Again, you should be
focused on _their_ needs, not yours. If you don't have time to do 15 poorly,
then just do one or two very, very well. No employer or recruiter gives a damn
that you've chosen a shotgun approach.

Give of yourself without expectations of something in return. You may be
surprised at the "unexpected" dividends you'll reap later.

FWIW: I have conducted over 2500 technical interviews and every single one of
them has had to code, regardless of experience or background. What good is a
programmer who doesn't want to code at the one point when it would provide the
most benefit to everyone involved? In my experience, the best programmers were
the most eager to give it a shot.

~~~
tom_b
How deep have you taken interview coding? Have you found that pseudo coding at
a whiteboard is as effective as giving a candidate a test assignment on a box
in the office?

I've also wondered about the effectiveness of reviewing candidate-submitted
code samples vs some type of coding during the interview. Any thoughts there?

Have you found any other strong correlations between finding the best
programmers in interviews other than the eagerness to tackle a programming
problem?

I'm about to start searching to fill two positions now - I've struck out in my
personal network, mainly because at this point I know people who are more
senior and the positions I have are on the junior side, both in pay and
responsibilities.

~~~
edw519
Discussed before:

<http://news.ycombinator.com/item?id=89669>

<http://news.ycombinator.com/item?id=834513>

<http://news.ycombinator.com/item?id=835092>

Hope these help.

~~~
tom_b
Thanks!

------
gruseom
This article, and the one it links to for support, are self-refuting as far as
I'm concerned. They make me think "be sure to have a coding test" if it keeps
people who think like this away (specifically, people who think that they
shouldn't have to try very hard and that their experience entitles them to
preferential treatment).

~~~
GiraffeNecktie
Hey, that sounds like a good idea all round! It will also let experienced
programmers know that they shouldn't bother applying to your shop (where their
experience is not going to be recognized and valued).

~~~
gruseom
They'd be right about that. Experience in and of itself counts for nothing.
There are a great many programmers who have years of experience programming
badly. Nor have I ever met a great programmer who cared a fig about how many
years of experience they or anyone else had.

What counts is the value you can add now.

~~~
GiraffeNecktie
Good luck figuring out from a programming test whether someone is actually
going to add value (rather than just be good at solving tests). Experience
delivering quality work on time is a reasonable predictor of future success.

~~~
slpsys
Wrong. It can easily be an indicator of a company (or series of companies)
that doesn't want to go through the hassle of firing you.

~~~
GiraffeNecktie
I wrote "Experience delivering quality work on time" I didn't write "has a
resume that only proves he showed for work for x number of years".

~~~
throw_away
Those two look mostly the same on paper. I've looked at the linked-in profiles
for people that I've worked with before. There are ones who absolutely sucked
and were a net negative for the team, but who "helped deliver feature X" or
"drove completion of project Y", but in fact, they probably actually delayed
or at least lowered the quality bar of those very things they tout.

------
kscaldef
> But asking a programmer to write a complete application for you is like
> asking an electrician to build a small electrical grid for you before they
> work on your house. Can you imagine if every plumber had to prove that they
> could snake pipes before they could work on someone’s plumbing?

I wonder if he realizes that we require electricians and plumbers to be
licensed?

~~~
fburnaby
And furthermore, this is the reason why we have them licensed.

------
fogus
I always ask coding questions and I always ask a question or two that are
extremely difficult. The goal is not to get the correct answers (although that
is a nice bonus), but instead to see how the candidate thinks a problem
through. A candidate's thought process is more important than their mad
hacking skillz.

~~~
jacoblyles
"how a candidate thinks" sounds like a very subjective criterion.

I wish there were more data behind people's interview practices. Do we know
that people that think in way X are better employees than those that think in
way Y? Why has HR mostly resisted empirical analysis?

~~~
fogus
> "how a candidate thinks" sounds like a very subjective criterion.

Of course it is. What part of the interview process isn't subjective?

~~~
jacoblyles
Yes. It is very subjective. Which is a shame. Empirical methods have
revolutionized so much of the world, yet this piece of it remains resistant to
positive change.

------
mrduncan
If you were interviewing to be a juggler, would you find it odd if they asked
you to juggle during the interview?

The reason why plumbers and electricians aren't asked to perform on a small
scale is that they are required to be licensed. Until developers are licensed
(and I don't see that happening any time soon), small-scale tasks are the best
way to learn about a candidate.

[Apologies to whoever I stole the juggling analogy from (I think it was on
Reddit).]

~~~
btilly
I first saw the juggling analogy in _Peopleware_.

------
willwagner
If I'm really excited about a position, I'll write my own code sample before I
go to an interview. I'll typically write either a simple igoogle module or
possibly a simple feature on my own website, proxying some of their calls
through my own web server.

I find it does a tremendous amount of good on both ends. It lets me explore
their website and look at their apis, I get a better understanding of their
tech stack, and by looking at their code (at least their client javascript
code), I brush up on areas that they've tackled.

On the company's end, we can dive into some tangible code that I've written
and they can code review my work and ask technical questions based on that. It
also shows them I'm serious and will hit the ground running.

I much prefer talking about relevant code that I've just written to a trip to
the whiteboard to answer an esoteric question or hard brain teaser. It's more
like a real work environment and the questions, at least I think, are more
authentic to the task: why did you implement that algorithm this way instead
of another? what are the code's limitations or where can it be improved? how
would it scale?

------
btilly
I've been on both sides of this situation. I don't think there is a clear
answer.

The fundamental problem is that interviews are a horrible way to select
people. There is little correlation between doing well in an interview and
being capable at your job. So in theory the more you make the interview be
like the job, the more accurate an assessment it is.

For instance I have never seen an interview method produce better results when
looking for a front end HTML person than giving them a picture of the page you
want, giving them cutouts of all of the pieces they need, and asking them to
write a web page that looks like that, will scale reasonably as you resize the
browser, and with reasonably clean HTML. Seriously, I've seen the results that
this test produces, and invariably it gives you someone that can actually
produce web pages. Occasionally it lets someone with poor work ethic slip
through, but that seems to be better than the alternative. _Particularly_ if
the interviewer does not have the skillset you are looking to hire.

And then there is the time issue. I have seen interviewing for a new person
suck up top talent for weeks on end. You need more resources, and you are
losing proven, trained resources to do something they probably don't like! Yet
unless you can have someone with the skills you're hiring in the room, the
interview process can't weed out people who don't have the right skills. And
as Joel's fizzbuzz test demonstrates, most candidates don't have the skills
you need.

How bad is it? Well Joel claims a significant portion of candidates can't
write a program to count numbers, but with every multiple of 3 replaced by
"fizz", every multiple of 5 by "buzz", and every multiple of 15 replaced by
"fizzbuzz". I believe it. I have found that over 2/3 of candidates when given
a double-loop to compute the intersection of 2 arrays can't figure out how to
make it more efficient. And good luck if you ask people to do something
unusual like a depth-first traversal of a tree!

But there is a limit. It is easy to produce a simple application that looks
reasonable, but which takes a lot longer to put together than you think it
will. If you can't, knowing how it is supposed to go, personally do the
exercise in an hour or two on a freshly set up machine, the exercise is too
hard. Personally I far prefer having a test with some simpler questions rather
than a full blown mini-application.

But the opposite extreme of depending on interviews to filter out bad people
is worse. Much worse.

------
known
I agree. I feel insulted when somebody asks me to write code during interview.
Wondering whether one can ask the interviewer to write code?

~~~
fogus
It's pointless to downvote the OP as his attitude is pretty common in my
experience. There are _many_ programmers who react with disgust if you ask
them anything that might be construed as beneath them... embrace that
attitude, because it can help to end the interview early and prevent wasting
everyone's time.

~~~
dschobel
This is the truth. I've had candidates walk out of interviews when they learn
that they'd be asked to write pseudo code showing some basic algorithmic
competence.

It was bizarre.

