
Ask HN: How do you identify a great software developer in an interview? - cte
Several young (fresh out of college) engineers on my team that I interviewed last year impressed me with their intelligence by way of excellent algorithmic thinking. On my recommendation, they were hired. However, several months into the job, they continue to ask trivial (read: stupid) questions, and don't seem to want to figure things out on their own. Yet other engineers, who I was not impressed by during our interview, have absolutely exceeded my expectations. They have the spirit of a hacker; they dig deep into problems, learn everything within our code base, and are invaluable contributors. How do I identify those people during an interview? Why do my intuitions seem backwards?
======
Locke
Despite how hard you try, you can't get this 100% right. This is how I
approached hiring:

1\. Weed out the truly incompetent. A _very_ simple programming exercise is
sufficient. Keep in mind that the goal of the exercise is not to identify a
great programmer, but rather to identify those who can't program at all.

2\. Set expectations. During the interviewing process, try your best to work
out what you would expect from the individual if hired. Would you expect them
to churn out a ton of quality code? Do you expect them to be great at
determining business needs and writing less code, but code with a high impact?
Is this going to be your go to guy for technology X? Do you expect your junior
person to learn Y and Z so they can start contributing after N months?

3\. Communicate your expectations clearly. Make sure the candidate knows
what's expected of him or her. Towards the end of the interview discuss how
they feel about those expectations. Ask how they plan on going about meeting
them.

4\. After hiring: If you get it right great, if not, let the person go sooner
than later. You do yourself and the individual a great disservice if you try
to turn them into something they are not. Don't try to turn a lump of coal
into a diamond. It's a trap. It doesn't work. They will be unhappy and
resentful if you try. Nobody likes to be fired (or do the firing) but you have
to tell yourself it's best that they get back out there and find the job that
they can be happy and successful at.

------
giardini
The answer is obvious and right before your eyes: you can't.

This problem has been examined again and again. The only way to find people
with sufficient skill in any field is to test them (formal graded test, not
idiosyncratic problems).

No one has ever demonstrated a reliable technique for selecting "great"
software developers (or great scientists, or great engineers, &etc).

~~~
ScottWhigham
Exactly. It's akin to asking, "How can I tell that I've found Mrs. Right (or
Mr. Right) during the first hour of talking to someone?"

~~~
byrneseyeview
I read about a study a few years back that required participants to spend some
time with a member of whatever gender they're most attracted to. Each pair was
required to (I think) do something slightly exciting and challenging together,
talk about something significant, and look into each other's eyes a lot.

If I remember it correctly, several of the pairs from the study were engaged
shortly thereafter.

So the answer to your question is simple, but not for the reasons you think.

~~~
cte
Would love a link to that if you can find it (assuming it exists somewhere on
the interwebs).

------
ojbyrne
I think its obvious someone fresh out of college will have "excellent
algorithmic thinking" since that's how they spent the last 3/4 years. So in
that case you should be spending more time during interviews looking at the
more likely areas of weakness, which I'd sum up as "real-world experience." If
you're interviewing experienced candidates, then you can generally hazard a
guess that their areas of weakness are elsewhere (lack of passion, knowledge
and experience that hasn't kept up with the state of the art, etc).

Interviews are all about probing. They shouldn't be a set of rote questions
that you apply to every candidate regardless of background. You should make a
guess at what the candidate's weaknesses are, and then push on them, push on
them some more. This will also help you learn how well they deal with
adversity.

Of course, before this, testing is a good way to determine what a candidate's
strengths and weaknesses are.

As a total aside, I've found that US Customs and Border Patrol (the ones who
do "secondary inspections") are about the best interviewers out there. They
probe so much you end up feeling a little violated.

~~~
anamax
> You should make a guess at what the candidate's weaknesses are, and then
> push on them, push on them some more. This will also help you learn how well
> they deal with adversity.

I disagree.

Find out what they think that their relevant strengths are. This is usually
fairly easy because they'll almost always tell you, often without being asked.

Test said strengths because that's where they'll go first, when things are
tough, etc. Also, said strengths are probably where you want them working. If
your needs are elsewhere, they're not your solution.

If their actual strengths are elsewhere, they've got a problem. Are they worth
making said problem yours? (There are lots of reasons why their strengths
might be elsewhere, some worse for you than others.)

------
swombat
I find that a 15-minute chat about a related topic that's not on their CV
(i.e. something tangential to their "career") is a great way to sniff out
hackers. Being a hacker myself, I have a very sensitive bullshit-o-meter and
fairly varied interests, and I can quickly figure out whether that person is a
"career programmer" who will end up disappointing, or someone who's got real
passion for the job.

Add to that an intelligence filter (well, I have an affinity for smart people
too), and maybe an informal chat about some past projects that they've worked
on if still in doubt, and I think I can recognise good programmers even on a
text chat system.

In fact, I spotted our last hire on IRC and was about 80% sure that he was
someone we wanted to hire, after just 30 minutes of chatting.

Someone already posted the link to my article on how to recognise a good
programmer - that breaks down the process I follow (though it's more
internalised by now).

------
woid
[http://www.inter-sections.net/2007/11/13/how-to-
recognise-a-...](http://www.inter-sections.net/2007/11/13/how-to-recognise-a-
good-programmer/)

~~~
cte
I'm not a business guy.

~~~
notauser
When hiring you should be thinking like a business guy.

You are dealing with a high level of cost (direct cost and opportunity cost),
not to mention a massive degree of risk (people are not predictable).

Therefore it is worth some mitigiation - and the budget for it depends on your
assessment of (risk * the total cost of a mistake).

The mitigations (such as a few months initially contracting for potential new
hires, running internship programmes, recommendation bonuses) can then be
costed and compared against the savings you make by reducing the probability
of, or the cost of, a hiring mistake.

------
watmough
Obvious answer: You probably can't identify the good ones without actually
testing them.

I worked at an oil services company where I fairly regularly interviewed
candidates. I was 180 degrees wrong on every interview I did. The bad ones
turned out good, and the good ones turned out bad.

I would not, knowing what I know now, bother with anything else except a chat
with some managers, and a some pseudo-code testing of the kinds of problems
I'd expect a computer science grad to be able to solve.

This opinion is backed up by a basic training course for some new hires I gave
recently, where my ranking basically flipped around from my first impressions,
and the guys with actual real ability didn't really float to the top until
after about 10 or 12 hours of training / evaluation. However, this could be
shortened easily into an interview format.

There's a lot of people who hide their light under a bushel, and a lot of
people who talk a good game, but are basically just warm bodies.

------
gaius
I interviewed someone recently, someone much older than me, and it was a
strange experience. We talked through his career, early on he had done some
really interesting things, attacked some problems from unexpected angles and
won big. But as we progressed through his career, it seemed to fade. Gradually
all that early fire was crushed out of him and he became a 9-5er. So I
thought, do I hire him and maybe try to turn him back into what he was? Or is
it too late? In the end he was rejected by the other interviewer anyway, but
it was a tough decision, and I could have forced it if I really believed in
him. But it just goes to show, an interview can open up more questions about a
person than it answers, and there's just no way to tell without actually
working with them for some time.

~~~
jodrellblank
You considered hiring someone to work 9-5 on your companies pet project as a
way to break him out of his his dispirited 9-5 coder profile?

~~~
gaius
Oh, we'd have found something interesting and useful for him to do a couple of
times, then we'd have expected him to identify such things for himself and
just get on with it. Him 20 years ago would have loved it and contributed a
lot - him now would have to either get it back, or sink without a trace.

------
maxklein
I'd give a real world problem that is quite large, for example: Develop a
school management system which synchronises across several computers.

I'd ask for 2-3 possible ways of designing the architecture. Then just drill
down to the details, ask about the technology they would use, how they would
go about implementing the algorithmic parts of things, the networking part of
things, etc.

You'll get a good sense of how the person works on a high level as well as on
a more detailed level. You'd also find out if the person is current on
available technologies.

Algorithms are pointless to know, very rarely does one use that knowledge as a
real life programmer. I know how quicksort works, but I've hardly ever
_needed_ to know, and I've NEVER had to implement it.

~~~
gaius
They don't teach you the sorts in CS because you'll ever need to do them. It's
merely a means to teach you Big-O by example. Being able to tell what's O(n)
and what's O(n^2) _is_ a very valuable real-life skill for a programmer.

------
cosmo7
I've seen people do great, well-commented code at interview, and then after I
recommend them, they turn into mediocre 9-5 coders.

People always misrepresent at interviews; I can't think of a way around this.

------
bitdiddle
I ask questions about two things that seem great predictors. The first is
interest in music. I'm not sure why but people who can read music and play any
instrument well tend to make great programmers. I have no explanation for
this. The second question is what the highest level math course they had was
and what they found hard about it. This one reflects a personal bias. Knowing
the answer for myself gives me a rough measure of how hard the person can
think and their ability to handle abstractions.

~~~
huhtenberg
> I'm not sure why but people who can read music and play any instrument well
> tend to make great programmers.

For what it's worth I came to exactly opposite conclusion, which probably
means that this 'predictor' is a bogus one.

------
eries
I would add, work together with them on a problem that is completely out of
their comfort zone and/or background. See what kind of questions they ask, how
fast they recognize a dead-end, and whether they learn from what you are
trying to explain. If you have a hard time teaching them something new in an
interview, probably it's not a good fit.

------
qhoxie
Try to give a (not too outlandish) problem that they cannot prepare for.
People memorize algorithms or can adapt their mathematical knowledge, but when
given an obscure or abstract scenario, you can get a good idea of how their
mind works. Encourage them to take their time and think out loud so you can
understand their process.

~~~
cte
I was never a fan of contrived problems with arbitrary solutions. I agree that
it offers a certain amount of insight into the interviewees thought process,
but I tend to be a fan of problems with provably correct answers that require
some rigor. Perhaps that's my problem?

~~~
qhoxie
I don't really mean problems that are unrealistic, just ones that are not
everyday issues. Along with that, I think "provably correct answers" are
mostly good, but it does not have to be just a single possible answer. I think
the best problems are the ones with multiple good avenues, but you can still
objectively identify if they take a wrong turn along the way.

------
timcederman
I honestly think too much emphasis is placed on fixed questions and a
particular style of interviewing. I've had a huge amount of success with
improvised interviews that have both conversational elements drawn from the
candidate's resume and some kind of problem solving task drawn from the type
of work they will be doing.

------
furiouslol
I would probably give them a problem that involves processing/handling a large
copious amount of data and compare their strategies.

Usually the great programmers come up with clean, elegant, efficient and
scalable solutions.

------
tomjen
Set up a competition geeks would be interested in, then try to hire the bests
(for your definition of bests).

~~~
maxklein
I'd not hire someone who had so few personal ideas to hack on, that he'd do
every programming competition that came along.

------
vaksel
Setup a dozen seemingly random questions that will help you know what
personality the person has.

------
albertcardona
Sit down for a _week_ with each candidate, perhaps with all candidates, in a
hackathon. Solving a real world problem. Then you can see who is worth what,
their strong points, their weaknesses. It costs you time, but it's --in my
experience-- worth every minute spent.

~~~
jm4
So how many week-long interviews are you planning to attend?

I don't doubt that this would work in a perfect world, but it's going to be
difficult finding candidates willing to go through with it. Right off the bat
you're going to have a severely limited pool of talent to choose from. It's
also important to think about what kind of candidates would be up for this.
They obviously wouldn't already have jobs and that could be for a variety of
reasons - good or bad. These candidates would likely have to put off other
interviews for a week, assuming they have other leads. Would you pay
candidates during this interview process? That's going to have an effect on
the types of people you can bring in for consideration. My guess is that in
the end you're going to have just as many difficulties finding a suitable
candidate.

~~~
albertcardona
All the problems you bring up are real. Perhaps the hack-together approach
could be rephrased as a screening stage, not a hiring stage. No point in
blindly hiring people after testing them with some random, one-day stressful
sessions.

(It's like scoring students with exams; makes no sense at all, when an
informal oral exam would really tell you how much the student knows. Scoring
students while having a beer with them may not be conventional, yet my former
PhD advisor used that as a pre-screen for hiring new grads.)

