
Hiring Developers: You're Doing It Wrong - Udo
http://devinterviews.pen.io/
======
sp_
I worked for a German startup too and our main problem was not vetting
interviewees but finding people who want to interview at all. In the five year
history of the company I think only one single person was hired who was not
already friends with someone at the company.

Other people just never applied. I remember manning the booth at one of those
college campus events and it was very lonely. I probably talked to three
students that day. Nobody followed up with us. I even had the distinct feeling
that students avoided eye-contact with us and made beelines for the booths of
the big established companies.

In the end, our hiring interview process for interns was 'do you want to work
for us? yes? you're in' and for full-time applicants it was simply non-
existent. I think in the last three years we did not interview a single
person.

I often wondered why that is but I have never found a good answer. In the end
it worked out for us. The company was acquired by Google.

Still, I would have liked to have some applicants and interviews once in a
while because I keep reading so much about them and I wanted to practice being
an interviewer to avoid pitfalls as described in the article.

~~~
lwat
Wow here in Australia we get HUNDREDS of people lining up to interview for
every job we advertise.

~~~
Dylanlacey
This is the same thing I've found, in Brisbane.

People apply for jobs without skills, without hobbies involving development,
without experience and without even looking at the website.

I simply don't understand it... Are they hoping to get employed to twiddle
their thumbs? Do they think they can pick it up on the fly?

~~~
lwat
When advertising on seek.com.au we've resorted to adding additional filters,
otherwise we get so many resumes that we just can't read them all. And yes,
most of them are clearly from 'spray and pray' applicants with no relevant
skills.

~~~
Dylanlacey
Why not add a code puzzle that has to be solved before accepting a submission?
Then you'd weed out _some_ of the spray.

------
jasonwatkinspdx
The only hiring process I have found to work for developers is to sit down and
work on real code together.

This gets to the heart of the matter, and you very quickly feel out someone's
knowledge, ability, and most importantly, how well they collaborate on a
problem. Because in a startup you will need collaboration, and likely under
the highest stress moments you've seen in your life.

I also feel like this gives applicants a much better opportunity to learn
about their possible future company and coworkers, and whether they themselves
would like the fit. If you have not done this sort of interview, even if you
have never pair programmed, try it out. It's very effective.

What I'm still trying to learn, is what screening process to use ahead of
this. Sadly, you can't invite everyone for an on site day long interview. The
best I've come to is to look at what applicants have made on their own time or
alternately how they talk over the phone about topics and problems they're
excited about. Resumes are nearly useless.

~~~
SkyMarshal
I wonder if you could give them access to a custom subdomain of a test site
and tell them to build a web app, anything they want, and upload it. Give a
suitable time limit, a day to a week, depending on how much you want to see
and whether they already are a full-time employee or student. Then screen
based on what people came up with.

That's a bit more involved than the lighter-weight solution in the same vein -
remote coding tests facilitated by a collaborative website. But it's something
I haven't heard of yet.

~~~
roel_v
Right, that's because no sane prospect (or rather, a prospect you'd want to
hire) would go along with it. A reasonably competent developer has several
options to choose from - it's just not rational to spend 10+ hours on each job
opening; you wouldn't be able to coordinate and assess 5 or so offers in a
reasonable time (when looking for a new job, people will apply at various
places, compare the offers and take the best one).

Most competent people would scoff at a suggestion like this, and I suspect
that competent employers know it.

~~~
mcherm
As a SCREENING mechanism 10 hours might be too much for the candidate to
spend. But once past the initial screening, if there is serious consideration
of an actual job offer, it is quite reasonable. The company will spend a good
deal more than that on every candidate brought in for interviews (if you add
up the time of everyone at the company). If the applicant is considering tens
of different offers then they're Doing It Wrong(TM) -- they should pick the 4
best and choose amongst those. If they're considering 4 offers and can't spend
10 hrs on each, then why on earth not?

~~~
roel_v
Because it's 40 hours wasted. 40 hours = one full working week = for (I think)
the type of professional we're talking about $2000.

I guess we can differ in opinion on what is 'reasonable' but I say that
requiring what you're proposing is too much for me personally to take any such
employer serious.

(on the part of the 'past initial screening', the employee can't know or
verify this. If there are 50 applicants, and there are 15 selected in the
'initial screening', is it reasonable to ask for 150 hours of unpaid labor
from these people? I posit it's not.)

~~~
InclinedPlane
$2k to find a good hire is nothing. Signing bonuses, referral bonuses, and
interview related expenses (air travel, taxis, hotels) are often on that scale
or larger. Also, your numbers are way off the mark, you're not going to bring
in 15 people for such an interview, you'd be extraordinarily lucky if you even
managed to have 15 people that looked good enough to go that far, but even if
you did you'd screen it down to a lower number to make it more feasible.

~~~
roel_v
Uh, what? I was talking about a $2000 cost to the job seeker. It may not be
_that_ much for professional, but it's nothing to sneeze at either (I've seen
people making a multiple of 120k fret over much less than 2k).

The GP was talking about doing the 'screening test' for everybody who passed a
first qualification stage. I don't know where you work, but having 15 'might
qualified from a cursory glance at CV' applicants is very normal. Basically
it's everybody who can use a computer and format a CV without using Comic
Sans, and is smart enough to phrase his work experience or degree so that it
sounds like it might have something to do with the advertised job offer.

~~~
MikeCampo
Agree with roel_v. There is no way I'm going to spend a week working on a
screening project unless I'm extremely confident in my chances. And usually
when you're really confident you already have a friend in the company. If the
company paid for the effort that would be a different story, but advertising
that could cost them quite a lot.

------
bugsy
The spare time issue is an interesting one. One one side we have companies
saying that they look for programmers who are working on projects after they
get off work and on weekends. On the other side we have companies having
employees sign a contract claiming that everything they do 24/7 belongs to the
company. Often both sides are the same company.

It's not an issue for me only because I work for my own firm, something I had
to do to get out of such bizarre situations. But it's an issue for many
programmers who are told that working on their own projects is stealing time
and mental energy from the company. I can certainly sympathize with the
talented developer who, told that the company owns all his private projects
done in his own time on his own equipment, simply chooses to leave at 5 and
spend time with his family rather than have passionate private work seized and
shelved by a firm who had nothing to do with its creation. It's really the
rational choice if you think about it and something very valuable in a
developer is rational thinking.

~~~
PlayerJ
To be honest everywhere that I have applied as a programmer has either had the
24/7 rule or a contract so unspecific that it leads me to think that they will
try to take ownership of something I write if it does indeed lead to a
valuable product. As such I am usually working on projects in my free time but
I rarely want to tell them about a real project for fear of them trying to
take it. It creates a difficult situation in an interview, when I have
projects but I dont want to discuss them.

~~~
khafra
Have you tried asking for a written addition to the contract saying work you
do on your own time, unrelated in subject to what you do for the company,
belongs to you? Contracts aren't always necessarily one-size-fits-all, and if
that's the only thing keeping you from a good job it might be an option worth
exploring.

------
jacques_chester
My previous employer (<http://thefrontiergroup.com.au>) had a process where
candidates would come and spend a day onsite. You'd be paid for the day.

The process was to work together with a senior coder on problems of escalating
difficulty. Starting with

    
    
        1.upto 10 do |i| { print i }
    

"What does this do?"

And ending with "Here's a legacy application we maintain. Add a new widget to
the dashboard. Think aloud."

During the day you had lunch with the team.

Even so, that process didn't work perfectly for them. They hired me and about
6 months later they decided to fire me.

Subsequently they've focused on hiring people they already knew. For example,
they've hired Darcy Laycock (<http://sutto.net/>), who they've known through
the Rails community for years.

... and in all fairness I'd sack about 5 of me to get a Darcy on board.

~~~
grammaton
> 1.upto 10 do |i| { print i }

My answer would be "bombs with syntax error," assuming this isn't something
that just looks like Ruby.

~~~
jacques_chester
Well now we know why I was sacked!

------
tt
In my previous "day" job, I probably interviewed more than 100 candidates and
hired about 15-20. Here's what worked for me:

1\. Phone screen (~45-60 mins). I'd spent ~15 mins going over what the company
does, what the work environment is like, the team structure, the
personalities, the technologies, etc... I'd try to "sell" our opportunity to
jazz up the candidate and get them at least curious and hopefully very
interested. I'd weave in a little of why-are-you-changing-job, what-was-your-
experience-like, or what-would-you-prefer, and then ultimately we spent the
majority of time on a recent project that he/she can talk about in depth. I
pushed on a few related technical aspects to those projects to see if the
candidate could clearly explain himself/herself technically.

2\. First round (1.5-2 hrs). If I liked the candidate's potential after the
phone screen, I'd invite him/her to our office to meet me and another
engineer. We'd go into more depth about their recent project(s), and then go
through a design/code exercise for a web application (the company builds web
apps). The exercise was very open-ended (there's really no one correct way to
do it). The design/coding choice depended on many factors, and we helped steer
the conversations so the candidate could talk about those factors and what
he/she would do to handle each of them (including when not to handle certain
situations).

Along the way, we also probed into how he/she would work with other team
members, how/whether to "challenge" another colleague, what/if any development
process he/she would follow.

There was no trickery, no reversing a string, no linked lists. But I might ask
how one would deal with concurrency conditions (two users trying to claim a
single resource), how to scale with traffic, etc...

The candidate was encouraged to ask questions throughout the interview. It was
a two-way street after all.

3\. Second round (another 1.5-2 hrs). If first round went well, I'd invite the
candidate back to meet a few other employees (e.g., another engineer, a
Product Manager, a designer for example). I asked these other employees to ask
anything they'd like, but at the end be able to tell me if this candidate
would a) be smart, b) get things done, and c) have right cultural fit. If
anyone had strong objection from this round, it almost always resulted in no-
hire.

Equally important to hiring well is to let employees go quickly if they don't
fit. That's a separate subject worth its own post.

------
btmorex
Seems to me you just shouldn't ask typical "CS problems". The problem with
those is that a positive result doesn't necessarily mean that the candidate
knows anything. It could be that they've just interviewed so many times that
they know a few common answers to memorize.

I still think asking coding questions is extremely important. Programming is
one job where you can actually test skill in the interview (unlike management
for example, where you mostly have to rely on personality, work history, and
references). I don't know why you woulnd't take advantage of that opportunity.

As for culture fit, obviously you need to do that too, but that's somewhat
orthogonal to what else you ask about in the interview.

~~~
msluyter
I agree that these types of questions can lead to false positives, but what
about the negatives? Don't these have some value as filters? After all, if I
can't answer a simple question like string reversal, it suggests that I can't
code _AND_ I'm not enterprising enough to do some rudimentary Googling to
practice technical interview questions.

------
michaelchisari
Despite having written tens of thousands of lines of open source code, I have
yet to have an interviewer who has looked at that code and asked me about it.

~~~
joelhaasnoot
My last interview at the company I'm at now went somewhat like this: "Your CV
looks fine, we wouldn't care if you're an axe murderer, now what would you
like to do/tackle?". Should have been a warning sign: interesting company to
be at.

~~~
biafra
What do you mean by interesting? Do they have a lot of axe murderers there?

~~~
joelhaasnoot
Everything practical-wise is done my the boss, who is painfully slow at
anything practical (salaries, supplies, workstations, etc etc). My project is
a sideproject, others have projects with constantly changing features,
deadlines and requirements.

------
cpt1138
Speaking from personal experience, a bad job can leave your "passion" somewhat
lacking. I think if you asked anyone I work with today, they would definitely
describe me as passionate. There are indicators that I am doing well there. I
can't agree more with this article. I absolutely hate doing tech interviews. I
can not code under pressure. Even in the most high pressure situations with an
outage or huge customer issue, I always take a step back, think, and try to
see what else might be affected before doing anything.

I hate tech interviews so much, that I never do them. I mostly do
unconsciously what this article lays out. I would say personally I've
experienced better than average hires if I happen to be positive on the
person, doing what this article suggests in an interview. I have missed good
people (was overridden) and have failed tons of tech interviews where I
thought I would be an excellent fit on the other qualifications.

You'll have to ask my co-workers if I'm good, but they tell me I am.

------
emehrkay
While at my last job I was tasked with being apart of the interview process.
My favorite question was "how do you keep up with web technology and trends?
(web development)" and I was surprised at how many people had no answer for
that simple question.

My response to my manager was "this dude wont work"

~~~
bugsy
My answer to that one would be "I've found it's more efficient to not keep up
with the latest fads."

~~~
sofal
That's understandable, if a little antagonistic. I'd probably probe further
just to make sure your definition of "fad" is not "anything newer than Blub."

------
eaxitect
Here's my 2c's on this as a 12 yrs Software Eng and a startup founder:

1) Software development is a TEAM work, so asking one questions like "how did
you design this or that" is just pointless. We did design that on that
specific period of time...Our motivation was ... 2) Good Developers Copy,
Great Developers Steal (altered from P.Picasso), so we all use Google, HN,
GitHub, wikipedia and alike to innovate our solution, so if you ask me B-Tree
algos, I'm sorry, but I've to look at wikipedia... 3) Hiring process is just a
scumbag for both parties, you impose big, the other party imposes big. So
what? You're not Google, and he's not Kevin Mitnick... 4) I think, companies
should try to get the personality, not the KLOCs of applicants. If you gonna
ask programming puzzles, please be authentic and ask something related to you.
Not just copy a Googlr interview question because you liked it.

------
staunch
My current system is:

1) Simple programming challenge (< 30 minutes) by email. To see if they can
actually write good code. Resume/age/experience/location mostly disregarded.

2) Casual discussion-style interview to get to know how they think and behave.

3) Short term contract with a predefined project (< 3 months) to see how they
work over time.

4) Full time hire with salary + equity.

~~~
tptacek
If (3) is working for you, great; however, bear in mind that it's a seller's
market for talent right now. I'd neg an offer contingent on doing a 3 month
contract first, and I'd advise my friends to do the same; why should I
shoulder that risk, if there are 2-3 other good positions open that will take
it on for me by offering FT right away?

~~~
ams6110
I don't know if it's the same everywhere, but a 90-day "probationary" period
is absolutely commonplace in the USA. Not sure I see much difference.

~~~
tptacek
Virtually all employment in the US is at will, meaning you are continually on
"probation". The difference between a contractor and an FT is that the
contractor is 1099'd, pays both halves of FICA taxes, and isn't provided
health insurance; more importantly, when the employer is as overt about the
issue as "not hiring you full time", there's no implied social contract or
norm ensuring you'll even end up getting the job.

Contractors make _significantly_ more money than full time devs partly for
this reason. They're compensated for shouldering the risk of keeping a full
book of work. Getting a good developer to work as a contractor for what will
eventually be their FT wage is almost kind of a scam.

This is a simple, practical issue. You're Sam, a developer with 5 years
experience. You have a choice between two roughly equal jobs. One prospective
employer offers you a full time job tomorrow; the other offers you a 3 month
auditioning contract. Which offer do you pick?

~~~
swampthing
Try looking at it a different way - assume, you will have a full time job at
either of these prospective employers. Do you want the one that makes FT
offers immediately, or the one that makes them only after testing new people
out for 3 months?

~~~
tptacek
I want the one that makes FT offers, because I don't believe that the best
programmers, who can write their own ticket in this business climate, would
put up with this contracting bullshit. This approach to hiring is a cop-out.
It says, "we don't know how to hire properly, so we're going to push the risk
onto the candidates".

~~~
swampthing
I've seen the contracts with my own eyes. But I guess I can't prove that the
developers in question were good without divulging names.

Anyways, I don't see why this is a cop-out any more than any other hiring
procedure. No company has anywhere near a 100% success rate - this procedure
acknowledges the shortcomings of a traditional interview process (namely that
succeeding on an interview and succeeding as a developer are two very
different animals) and tries to address them.

~~~
mavelikara
> Anyways, I don't see why this is a cop-out any more than any other hiring
> procedure. No company has anywhere near a 100% success rate - this procedure
> acknowledges the shortcomings of a traditional interview process (namely
> that succeeding on an interview and succeeding as a developer are two very
> different animals) and tries to address them.

Agreed. But tptacek's point seem to be that this new process addresses the
perceived weaknesses of traditional processes by moving all the
downsides/risks to the potential employee. I think the hiring firm can signal
their honorable intentions more clearly if they would pay a significantly
higher salary during "probation" - something that compensates for lost
benefits etc.

Edit: staunch mentions elsewhere on this thread that he is offering higher
rates during probation.

------
petenixey
I once learned a shocking lesson about the influence of my preconceptions when
I accidentally took a developer through three rounds of interviews because he
previously worked for Pivotal Labs and had a German (aka technical) accent.

It wasn't until the third interview that I asked him to write an algorithm to
sort an array and when he couldn't I suggested instead writing an algorithm to
see whether the array was already sorted. When he couldn't do that either he
told me I was asking the wrong questions and not letting him show off the code
he was good at.

To this day I have wondered and never discovered what type of code doesn't
involve arrays, loops and comparisons however I do now ask candidates to write
code right at the start of the interview loop. I simply never anticipated how
many non-programmers would apply for programming positions.

~~~
enjo
I haven't written code to sort an array in nearly two decades of professional
experience:) I've been lucky enough to work on some pretty non-trivial
problems as well. If you asked me to do it, I could probably crank out some
sort of bubble-sort (maybe).

It's not to say I haven't sorted arrays, I certainly have, but whatever
standard library I'm using almost always does it for me... and pretty damn
well at the same time.

~~~
X9
While it is largely based on the context of the job, I'd think some
demonstration of fundamentals, like sorting (even if it's just bubble sort
:-)), is important. It's another way to see which mental tools the candidate
can use when debugging problems and they're forced to drill down through the
layers of abstraction provided.

------
joshsegall
It seems fairly obvious that a pure tech interview doesn't screen for cultural
fit, and that you need to have some form of behavioral/cultural assessment as
part of the interview process.

Depending on how much time you have to interview (and I suggest you take the
time if you're going to hire someone!) the "standard" dev interview can do
pretty well at weeding out candidates who clearly lack basic skills. Joel
Spolsky is a proponent of this and I agree.

I've found that passion is a great indicator of future success, and you can
usually get a good quick read on whether they are serious about software
development by asking people about their favorite projects, what they spend
their time working on, and what interesting things they see happening in the
field.

------
dminor
I once decided to skip the standard programming problem during the interview
process, and I ended up regretting it extremely. The person we hired was nice,
and a good "culture" fit, but couldn't code for beans. We had to let them go
and I felt pretty bad about it.

Programming questions certainly aren't the be all and end all, but as a filter
they are useful.

~~~
jharjono
Programming questions seem also useful for companies trying to make a good
impression on the candidates. I find that companies with really interesting
technical questions are more likely to spark my interest than companies whose
hardest question is "How would you reverse a string in Java?"

------
ianbishop
I never really understood why companies don't just take a non-trivial real
problem that they have run into during development. Just ask the candidate to
talk it out, see if they are able to at least get on their feet toward a
solution or an idea of possible solutions.

I've never hired anyone, but I can tell you that I could write a linked list
in a handful of languages. I can also tell you that it doesn't say much about
what I know (or perhaps more importantly, don't). Problem solving is what is
important, more important than ability to write code on a whiteboard.

------
okaramian
Definitely agree with this post. I've been interviewing for the last couple
months and one of the better/more fair interviews I've had involved a nice
back and forth (mostly the types of questions posed in the article, including
a lot of stuff about my outside projects).

Then they brought me in for a tech interview where they plunked me down in
front of a computer to architect a small application for a type of problem I
might encounter on the job. I thought it was fair and it probably gave them
reasonable insight into how I code/think/present my solutions.

I've had a lot of "write a linked list" style and they're draining. The
quality of the applicant being pulled in to do CS trivia is going to be all
over the place since that type of an interview can be gamed fairly easy if
someone has a desire to do so.

------
mncolinlee
I was very impressed when an employer used an open notes, take home exam in
the second interview. Initially, they had tried a written, proctored exam
during the first interview.

Although I was the only candidate among many who did not claim any experience
in the actual technologies listed in the position, I was also the only
candidate who turned in the take home exam. I also completed my working code
in 8 hrs rather than the week time limit. The others all quit the exam at
various stages because they did not possess the tenacity to hunt down
technical knowledge online required to implement the proof of concept
software. At the third interview, they asked me how I came to my solution and
why I chose certain methods in order to validate that I completed the work
myself.

This seemed like an great alternative to other exam experiences I'd had. At
another interview, a non-technical company officer asked me a technical
question and then couldn't explain it. The question was so opaque that it took
thirty minutes of probing and badgering him to even understand the question.
By this point, I was ready to write them off and leave. They lost their
largest customer the next week and I would've been laid off if I'd joined
them.

------
MichaelGG
Writing code is just another type of conversation. Sure, you're going to ask
many questions. Having a candidate code a bit in front of you, going back and
forth, provides a lot of info.

As far as "CS puzzles" - binary search, trees, linked lists, hashtables, etc:
None of those should be puzzles. If you're giving interviews that people can
"memorize" an answer to, then the problem is how you're doing the interview. A
proper conversation, including code, will quickly sort out if the person just
memorized a one-line Haskell quicksort, or actually knows what they're talking
about.

~~~
bugsy
Do you actually find people that expect candidates to implement quicksort?

I've certainly implemented it, but there's no way I have it memorized, it's
not an obvious algorithm at all. I'd be flabbergasted to be asked to implement
it from memory with no warning. It seems clear to me that one would be testing
for if the person happened to have looked at the algorithm recently, not if
they were competent.

~~~
kaiwetzel
In college ("computer science part A",using Haskell) we played out the various
sorting algorithms we learned in class, with students holding up a number (to
determine sorting order). Neat to actually experiencing the slowness of
bubblesort vs. mergesort/quicksort ;) We repeated the performance in front of
a local mall - I can't recommend trying something like that in a crowded place
from that experience.

I think expecting recent CS graduates to implement a basic sorting algorithm
or binary search, etc. (say, in a garbage collected language they know well)
is reasonable, for people with actual work experience I think it makes less
sense and the focus should be on more recent experience if relevant to the
job.

~~~
bugsy
Yeah doing a binary search in an array is pretty standard. What I was asking
though is quicksort since that was mentioned. It's not obvious by any means
how it works and I recall that early reference implementations had bugs that
were unknown for years even though widely used. It really sounds to me like
asking someone to implement the FFT in an interview. What are you really
testing for when you do that?

------
joakin
Bit offtopic but that CSS made reading the article a complete pleasure. With
the mixture of nice readability and interesting content this post turns out as
a really good essay.

------
usaar333
I have to disagree about the uselessness of a technical interview. I'm 100%
with RethinkDB that any competent programmer should be able to figure out how
to reverse a singly-linked list in some reasonable time.
([http://www.rethinkdb.com/blog/2010/06/will-the-real-
programm...](http://www.rethinkdb.com/blog/2010/06/will-the-real-programmers-
please-stand-up/)).

" Some were shooting their pre-canned answers at me with unreasonable speed."

This should not be possible for all questions in a technical interview.
Especially if you are a startup, you can use your own original problems,
especially ones you've actually had to solve. Aim for actual coding questions
though rather than too puzzling ones that are often just memorized (implement
quicksort, detect cycle in linked list in O(1) space, the random puzzles
companies love to ask).

Of the people I've worked with, there is a high correlation to job performance
to performance in a tech interview. (which for new grads is strongly
correlated to their college CS grades).

Of course, passion is also a decent indicator, but it is highly correlated
with raw competance. That said, I wouldn't hire anyone lacking either.

Disclaimer: I'm coming from the background of a systems company. For those
making CRUD apps, tough coding questions might be less relevant.

~~~
BrandonM
> I have to disagree about the uselessness of a technical interview. I'm 100%
> with RethinkDB that any competent programmer should be able to figure out
> how to reverse a singly-linked list in some reasonable time.
> (<http://www.rethinkdb.com/blog/2010/06/will-the-real-programm...>).

I had the initial phone interview with RethinkDB and they were asking me
questions about what is a B tree and how to use them to implement a
filesystem/database, what is the pros and cons of structure A vs. structure B,
etc.

I understand wanting the best of the best, people who have implemented
filesystems before, etc., but I think it's pretty unrealistic to expect
someone three years out of college who has not been working in the same area
of specialization to just know this stuff off the top of his head.

Wouldn't it be much more realistic to basically give someone the Wikipedia
entry for various data structures, tell them something about the data and
workflows, and then ask which structure(s) they would use and why?

After that interview, I was sure I wasn't getting the job, but I wasn't very
upset. They seemed to have unreasonably high expectations, and that's not
exactly the most healthy work environment.

------
amurmann
Why not just pair with the interviewee for half a day or a day? You will learn
how they think and work and if you like them. It's a riddle to me. I've never
heard from anyone actually trying it and not being happy with the insight won.

~~~
cageface
I'm a lot more interested in how someone breaks down an architectural problem
than I am in how well they can implement heap sort on a whiteboard. The closer
the interview is to real coding the more you learn.

Of course, github is the real resume now.

~~~
amurmann
Why not pair on whatever you would be working on anyway. I usually have to
solve many architectural problems every day but honestly never had to
implement a heap sort all my life. So there should be ample opportunity for an
interviewee to show his architecture-fu

------
vladhorby
I get a really good insight in the candidate's programming skills by pair-
programming a simple problem with them. For TDD, I write a test and ask him /
her to write the code to make it pass (then repeat, refactor). That way, I can
see how fluent they are in the programming language, at the same time evaluate
their problem solving skills and see how they work in a team.

~~~
yxhuvud
I have had that happen to me a few times.

They used vim and textmate. As an emacsian that didn't have any firm
understanding of those editors, it was .. frustrating. Lots of times with
'oops, what did I do now?'.

------
Osiris
It seems that one simple thing you could ask is for usernames or profiles on
sites like StackOverflow or other programming related websites to see what
kinds of contributions or questions they've been asking.

~~~
mc2k
I think websites like Stack Overflow, GitHub and HackerNews attract a certain
type of programmer, which might not always be the sort of person a business is
looking for. Or, at least, it will exclude a lot of people who aren't
interested in being involved with online development communities. That is,
they are too hardcore.

~~~
smlacy
Yes, but there are tons of other data sources out there, right? For example,
how many developers are members of mailing lists for a given API, service,
language, tool, etc.? How many contribute? Plus, that doesn't even include the
world of personal blogs, twitter accounts, etc.

I think we're entering an age for software engineers where expressing yourself
externally is going to be even more important than your resume. For example, a
really intelligent, well-thought-out post like the one linked above can really
move you forward in your career.

I'm focusing on making a startup that captures all this stuff and uses it for
both display and matching candidates. I'm still in the very early stages and
would love some feedback! <http://proovn.com>

------
chopsueyar
When I hire a developer, I ask him about the scene in 'The Social Network'
with the Winklevoss twins, and ask if he knew Armie Hammer's face was CG'ed to
another actors body.

Depending on his answer, I know right there whether to end the phone
interview, or fly him to the Valley so I can see him write code on a
whiteboard.

~~~
btilly
I guarantee with that hiring process, that you will not want to work with me.
Given your interview process, the feeling is likely to be mutual.

(I don't watch TV or most movies, haven't seen _The Social Network_ , and have
no interest in doing so. If that is what you value, then I'm going to be a
poor fit from both sides.)

~~~
enjo
I think he was making a joke about a submission here a day or two ago...

------
cdegroot
Fwiw, my current screening is: first resume screening, then I send candidates
a simple programming exercise that sort of is in our domain. I can assess the
delivered work in around five to ten minutes, so it scales well. Then a
further skills/team fit check through interviews, and for those who survive
that, the expensive bit: pairing with developers on actual production code.

Especially the exercise works as a good screen. We get quite a number of
"deafening silence" responses, which says something about the candidate's
motivation. We also get responses where people try to show off alternative
language skills but deliver something totally non-idiomatic. And we get good
submissions, showing some patterns knowledge, unit testing, at cetera. These
typically end up being hired.

------
Jeema3000
Knowing all about data structures and other computer science implementation-
related stuff means exactly nothing if a person can't think creatively enough
about how to approach a real-world problem to begin with. Those data structure
CS questions are down in the weeds and you haven't even figured out if the
person can get down _to_ the weeds if you're asking those sort of questions in
a first-round interview.

I mean think about it: do you approach programming tasks top-down or bottom-
up, usually?

Critical thinking is the most important skill IMO, and it can only be tested
by giving someone open-ended questions on how they would approach _real-world
assignments_ , preferably in a low-pressure take-home test kind of thing -
again IMO...

------
KeyBoardG
Totally agreed. As a team lead I've hired my best talent with interviews like
this.

------
ses
I couldn't agree more with the sentiments of this post. Apart from anything
else 'software developers' vary massively in their core strengths and job
positions require different strengths in the candidates. Skill in developing
algorithms and solving puzzles on the fly is just one of those strengths.
There are so many more in terms of software engineering practices, knowledge
of programming patterns, awareness of different programming paradigms, ability
to solve higher level problems / combine technologies effectively to solve a
business problem rather than a low level programming problem etc.

------
damovisa
My last interview process was intense:

1\. A write-a-document test. I was given a fairly simple (contrived) scenario
over the phone and was asked to write specs and a deployment plan with a
deadline of an hour.

2\. A programming test. Again, a description given over the phone and someone
available to answer questions if I had them. After delivery, there was a
followup programming task to extend it from a client app to a web app. It took
about 8 hours in total.

Provided you're willing to put in the time (and I was), it seemed to be an
excellent way of judging whether I could really do the work they needed me to.

~~~
bugsy
I hope they made sure to get you to sign a copyright transfer for the work.

------
AlekseyB
Hi there! Good article, I've seen similar situation in several companies too.

Few words about one startup-like company(based in Kiev, Ukraine) I have been
working for 1 year as developer. It was focusing on .NET Enterprise apps.
Their interview approach: \- 1st day: tech-interview via phone(0.5-1hr); \-
2nd day: (companies office) simple technical tests(choose-right-answer on
paper), short technical interview(technology basics, fizz-buzz tasks, sql-
tasks, usually join/group-by, having-based), coding-task on laptop( _the most
significant and essential(for evaluating developer) part of whole interviewing
process_ write application for calculating bowling scores, Console app or GUI
application - no matter, for candidates choice, time-bounds - for candidates
choice; rules were provided and explained to each candidate + full access to
internet resources + possibility to ask interviewer == real working
environment); \- 3rd day: final technical interview(patterns, OOP, code-
design, etc.), general interview with CEO;

When I came to it, there were 3 devs(including me). The next(after me)
developer were hired after ~60 interviewed fellows. Yeah, the total interview
time is tend to be long and hard, as for candidate such as for company, BUT in
such way were established the best TEAM I had honor to work in.

Thanks for attention:) Cheers Aleksey

------
larsberg
"we ended up hiring the candidate with the smoothest answers"

I mentioned this in another thread, but the point of coding questions is to
see how they think about code, whether they understand how to walk through it
(have a machine model), if they can pick out and evaluate edge cases, and how
they can work with hints from you on how to improve their answer. Hiring based
on ability to answer a particular problem is probably only a slightly better
indicator of ultimate job success than height.

------
meatpeople
Having only recently read Peopleware 2nd Ed., I've been mulling over the
audition concept in the "Hiring A Juggler" chapter - effectively having a
developer present on a project they worked on, what they contributed and
learned and so forth.

It seems like it would have value - allow the candidate to prepare and present
on a topic they're strong on, and in-depth enough to allow cross-examination.

Has anyone any experience of this from either side?

------
utahmadmike
Great article! For 15 years I have conducted interviews and been the victim of
the interview process. Your article sounds familiar; I have been telling my
wife the same thing (out of frustration) for years. On several occasions I
have made it through 3 or 4 levels of the interview process and then blocked
in the final interview-step by an insecure or egotistical technical person.

I would add the following (problem) points as well.

* Some technical people like to hire clones of themselves, creating conflict and diversity issues.

* Some technical people, conducting the interview, use the interview process to posture and show how smart they (the interviewer) are.

* According to the BLS, the demand for programmers is decreasing while demand for analysts is increasing. Programming has changed a lot with tools like intellisense and included libraries.

* If you can’t answer the question, "why did you ask that question?", you should not ask the question. Each interview question should have a specific targeted purpose.

I hope my additions help. <http://www.linkedin.com/pub/michael-
pritchard/3/93b/4ba>

~~~
utahmadmike
Also:

* Realize that answers to questions like "what is abstraction" might vary from one text book to the next.

* Establish context for your interview questions. I was once asked "What is Application?" (too vague). In response I said, "In what context?" The interviewer smugly said, "What comes to mind?" . . . He did not like my answer. Hum?

------
acp20
As a former head of HR for three successful start-ups and now as the founder
of my own, I (in the words of our former president) feel your pain. I've
reviewed 10's of thousands of resumes and interviewed over 1,000 candidates so
I know what you're going through.

Your points are well taken and insightful - and while they're especially true
of technical hiring, they are also applicable for all start-up hiring. A smart
person can learn the tech skills you need but it is impossible to impart
passion and an intuitive understanding of culture where it isn't already
present.

I eventually honed my interviews down to one question: "Please describe your
most significant team accomplishment, where you were a key member of the
team."

Then I'd follow that with numerous follow up questions until I really
understood the person's role. This will uncover the candidate's passions,
communications skills, leadership abilities, and their technical prowess. Then
I'd ask the same question about another accomplishment.

Good luck. I'll write you a note on the side with my contact info. I'd be
happy to talk over any "people stuff" if you'd like.

------
JerryH
Couldn't agree more.

The focus that the short sighted interviewers and companies have on how fast
your can solve some pointless algorithm that a CS has figured out 30 years
ago, vs how you get on with the team I find mind boggling.

[http://www.jeremyhutchings.com/2010/11/interviewing-
dating-f...](http://www.jeremyhutchings.com/2010/11/interviewing-dating-for-
techie-playing.html)

------
dspeyer
The problem with "tell me about your projects" is that it doesn't distinguish
good programmers from bad. It isn't even gaming. A programmer who makes a
negative net contribution to a large project may sincerely feel he is
responsible for its success. All you can pick up on is their perception.

At least when you ask them to write code, you can tell if they got it right or
wrong.

~~~
yannickt
You can actually learn a lot from a candidate's past projects if you know how
to follow up with the right questions. Someone who had a negative net
contribution on a project will not have anything interesting to say about the
data structures or algorithms they used. They won't have interesting war
stories to share either (e.g. nasty bug fix, conflict with management or team
and how they overcame it, etc.).

------
motters
In the last year I've been on interviews like this, where I've been presented
with sorting problems or quirky pointer arithmetic conundrums. However, these
bear no relation at all to the kinds of practical software problems which I've
had to tackle in business or industrial contexts in the previous decade or
more.

~~~
martincmartin
That might be a good thing. It shows how you think in new situations, how you
can problem solve, and how you deal with complexity. I used to give "reverse a
linked list," and it's essentially a book keeping problem: you're changing
links around the time you're trying to read them, so you need to keep all that
straight. If there's a bug in their code, you can see the thought process they
go through to find and fix it.

Unless the job they're interviewing for is almost exactly the same as their
current job, it's important to understand how they think through new
situations.

------
joshu
A very candid self-analysis. Thank you.

------
ScottBurson
>But how did the candidates we selected measure up? The truth is, we got very
mixed results. Many of them were average, very few were excellent, and some
were absolutely awful fits for their positions. So at best, the interview had
no actual effect on the quality of people we were selecting, and I'm afraid
that at worst, we may have skewed the scale in favor of the bad ones.

This doesn't follow at all! Consider, as is surely the case, that only a small
fraction of the applicants would have turned out to be average to excellent.
The interview process can filter out the vast majority of the sub-average
applicants and still leave you with a significant fraction of sub-average
employees.

------
sl_a_sh
I just had a thought: How about having a phone round first, and ask all
applicants to provide a sample of their coding skills, like suggested in the
article. Then, in the second round you leave each applicant for a couple of
minutes to analyze the code from someone else and find out what it does and
what's so special about it.

That way your work of looking into the code samples will be minimized. Even
better, you'll get an idea of how well a coworker with a similar skillset can
cope with that code. You can also judge how well people are at dealing with
code from their coworkers and don't have to come up with examples yourself.

It does sound pretty good, no?

------
timedoctor
I think interviewing altogether is an ineffective way of evaluating people.
And the whole process of interviewing and hiring staff members is an outdated,
slow bureaucratic way of doing things.

Instead why not work with several people on small projects, ask 5 people to do
2-4 weeks of paid contract work for you and then select the person who was the
best fit for longer term work. This tests the real world skills of a person
much more effectively than an interview.

------
civilian
s/'Me and my cofounder'/'My cofounder and I'

But it was a really good article besides that! Sorry for being a grammar
german.

------
ez77
Their smart quotes are ”wrong“.

~~~
chrisbroadfoot
❜❜Nice, but that doesn't work for large, successful companies. Your idea
doesn't scale.❛❛

------
jyap
This is a great read. Exactly how I have approached hiring developers.

------
tripa
What does 'I only went "full Trump" once on an employee.' mean?

~~~
mattm
I'm assuming it means outright firing someone.

~~~
Udo
Yes, it does. Sorry, it's a bad allusion to an even worse reality TV show.

What it means is that letting people go happens. Usually, when people were
underperforming or showing other signs of problems, I just talked with them to
clear up what the matter was. Sometimes, they had personal problems, or other
temporary and resolvable issues. Sometimes, they were unhappy and needed to be
somewhere else.

I remember one guy, an intern with horrible attitude, who said to me in
confidence: " _I regret this so much. I always wanted to become a baker, but
my parents wanted me to have a real job._ " I kid you not!

The point is, it has been my experience that even layoffs can be borderline
mutual. But this once, the latest screw-up in a long line of screw-ups... it
was so monumental and malicious even. Worst of all was I felt kind of
powerless, having this adversarial force on my team. Like a Trojan horse that
wasn't even making an effort to disguise itself. I fired the guy on the spot.
Of course, you can't do that in Germany, so eventually I paid through the nose
for it following the lawsuit. But in that split second it was worth it just to
have this guy off the floor immediately.

------
grahammather
Thank you for posting this! This really resonated with me.

------
carphill
Very lucid and reasoned argument that you make. I intend to blog about it, and
my forthcoming experience of using it on my blog at: carphill.blogspot.com

------
androck1
Not sure I agree with the alternative, but besides that, I think the nail was
hit directly on the head.

------
am_a_droid
Hello Mr. Schroeter,

I have enjoyed your post on DZone
([http://www.dzone.com/links/r/hiring_developers_youre_doing_i...](http://www.dzone.com/links/r/hiring_developers_youre_doing_it_wrong.html))
and have decided to let you know about it. This topic of 'interviewing the
right candidate' has always intrigued me. I have always wondered if the
superstars amongst the technology-driven companies have devised a surefire
method to pick the right candidate from thousands! In my experience, the
decision to go for a candidate or not, is very subjective as you have
mentioned. At a certain point in time during the interview, one listens to a
inner voice and decides between 'Yeah!' or 'Nah!' and not always that decision
is based on pure facts: precise answers to precise questions. But, to know the
personality of a programmer - through some questions and possible answers in
an hour's conversation - is a difficult task. You have not mentioned if you
had cracked it finally in your star-up or your later efforts/engagements. I
will be keen to know.

I am based in India currently. Here, because of the business success of the IT
Service companies, most of the companies try to follow their business model.
However, about 95% of the work that these companies take up are low-tech,
mostly repetitive and disincentivize breakthrough technical work. Their need
for good programmers is not much; they are content to see that the client is
that much happy to keep the projects continuing: that is the main objective.
Moreover, you seem to say that 'scaling' is not an insurmountable problem. It
probably is, if you see that the Indian IT service companies cite hiring
figures in tens of thousands in an year. So, the interview process that these
companies stick to, aims to find out how much is the probability that the
candidate is going to prove himself a misfit. Subjective, as usual; for a good
measure, the companies take in people in huge numbers simply hoping that the
probability of a 'good fit' increases. For a bit more experienced people, it
is almost always aimed to find out if the candidate can 'manage' a team, and
certainly not whether the candidate understands the world of programming at
all.

I work as a freelancer and in India, that doesn't help me. I claim to know and
understand what it takes to 'design' and 'develop' a good piece of software.
More than that, I carry a certain amount of intensity about the whole act of
programming and usefulness of well-written programs, even at this advanced
age. However, that doesn't interest most of the potential employers because a
good programmer - who knows some but is eager to work in a team where people
know more than he does - is not valued as much. Importantly, it requires a
keen pair of eyes to spot these qualities and that is largely missing.

Recently, situation here has made me so despondent that I have begun to look
for opportunities outside India, Europe in particular (I have stayed and
worked there before, hence the bias). I am hoping that there will be some
people out there who will take some interest in not only what I know and have
done, but how fast I can know more and can do if I join them. I am looking for
those keen pair of eyes. Agreed that there are those issues of geographical
distance and visa papers etc., but if someone finds me suitable, s/he may want
to go that extra mile.

If you have read till this point, thanks for your patience. I read your post
at a moment when I was in despair and wanted to demonstrate that your post
chimed with me.

Warm regards

\-- Nirmalya (sengupta.nirmalya@gmail.com)

\-- Software Technologist <http://www.linkedin.com/in/nirmalyasengupta>

------
rdorfner
Well stated and pretty much spot on, with one exception...

The notion about "Programmers who are good program all the time, even in their
spare time."

It has been my personal observation that this period of "Programming is the
only thing worth doing with your life" (And yes, I know I am paraphrasing this
in an unkind interpretation of what you said, I'm not doing it to be mean
though, I'm doing it to illustrate a point) is true, for the first 5 to 7 or 8
years of a programmer's career. The reason seems to be that the deep
fascination with what makes programming interesting as a career, continues to
hold them even in the evening hours when they are at home. When not actually
programming, they talk about programming, or programming problems or some
aspect of computers and computing technology, when they can find friends to
talk about it with, that is.

However, the basic drive behind this seems to be finding out as much as you
can possibly find out about your chosen field or niche within the field. After
that time period, though, something interesting happens. The _MYSTERY_ of it
all, starts to fade. You begin to see patterns in how things work, and
similarities in how things are done, or are implemented, or can be
implemented. Other interests start to take hold and after a while, family life
starts to become important as well. At least, for a well balanced human, it
does.

And that, I think, becomes my major point. The kinds of programmers who live
eat breathe and drink programming, non stop, past this time period, are RARE,
and usually are not well socially adjusted.

This is something that I think is not accounted for when interviewing more
experienced software engineers. When looking at the scads of job openings for
folks with 3 to 5 years of experience, your methodology and assumptions should
work well, however, when you are looking for that really senior guy or gal who
has been doing this for over 20 years, then the methodology, in my opinion,
will not yield to you the best results. ie. teasing out the best candidate
from the stack.

As an aside, I once told this to a young friend of mine who I met in college.
I had already been programming for nearly ten years when I returned to school
to get my bachelors in CompSci. I was always fascinated with just HOW MUCH he
would go on and gush about computers, tech, sun workstations, sgi's, their
hardware, their software, how to program them, linux and unix tricks etc etc..
One day I smiled and said to him. "I'll give you 5 to 7 years, and this utter
all consuming fascination with computers will tone down to seeing them as
nothing more than tools." He denied it, vehemently, he would _NEVER_ have
computers be something so mundane! It was, about 7 years later that I got an
email from him where he told me "You were right." He's a googler now, and was
one of the lucky ones who was in early, pre ipo and can now retire if he wants
to. I myself tried getting into google, but made some mistakes in the
interview process, trying to be 'clever'. It backfired, but that's okay. And
not once did they ever ask me anything about what it is I do, or what I do
best, or what I was interviewing for, board bring up, device drivers and
kernel porting. Things which I'm very good at and have done for over 25 years.

Anyways, just my .02, your mileage may vary!

Sincerely, Richard Dorfner

------
reubenyeah
I always find the emphasis put on algorithms 99.9% of developers will probably
never have to implement after the interview totally astonishing.

------
haploid
Too bad this site's domain name wasn't issued by the Iceland TLD registry.

~~~
Udo
What's your point? You realize it's an anonymous publishing service that,
although being awesome, has nothing to do with the article at hand?

~~~
amurmann
No humor on HN allowed!

~~~
amurmann
I am getting down votes for this?! WTF? Look at the domain name and understand
the original post, before downvoting! Think and then act!

~~~
amurmann
Yeah, keep downvoting me and don't even have the curtesy to write a comment as
to why!

~~~
gruseom
You're being downvoted because you broke two explicit guidelines (the last two
here: <http://ycombinator.com/newsguidelines.html> \- you should probably read
these as a linked list, not an array) as well as an informal one, that this
community frowns on superficial one-liners. It's not that people here hate
humour, it's that they're sensitive to how other online communities
deteriorated once such comments became the norm. They are like weeds that
spread rapidly and choke out the flowers and vegetables. A few weeds are ok,
amusing even, but they don't make for a satisfying crop.

The weeds are one problem (users here do a pretty good job of rooting them
out); a harder one is the crops themselves getting more mediocre.

