

Ask HN: What do you think is the ideal programming interview process? - mckee1

After the recent thread on Google rejecting the creator of Homebrew. If you could design the process from scratch, what would it involve?
======
audieleon
First, do a phone screen, not more than 1/2 hour of questions around the tech,
team fit, and motivations. Involve a few people from the team the person will
work with. This phone screen should weed out 80-90% of candidates. If you have
a doubt about them on the phone, that's a no.

If they pass the phone interview, invite them back and tell them to be
prepared to code during the interview.

Start your actual interview with code test like FizzBuzz or something similar.
Have them do this in front of you and one or two other competent developers.
Many candidates will outright fail this type of test. If they do, politely end
the process there.

The rest of the interview should be a few panel type interviews with the team
and both technical and team fit questions. Include non developers if you can
(testers, marketing, ops, etc.). Leave empty spaces in the conversation for
the candidate to fill. They more they talk the more they will reveal. If you
start getting a preponderance of thumbs down, politely end the process
immediately. Don't waste time.

Take everyone's feedback seriously, and arrange to get it quickly from
everyone. Preferably you would have a thumbs up/thumbs down from everyone
immediately. (Know your team well enough to know who gives only thumbs
down...) Look for and pay attention to biases within the group. As the
manager, sometimes you have to make a call that the team might not like. Be
prepared and courageous enough to do so. (For instance, hiring a process-
oriented qualified woman candidate might not sit well with your team of male
cowboy coders, but it might make the team better...)

Typically, if the candidate has gotten this far, then I take them and the team
to lunch to get them in a more informal setting. As long as the candidate
doesn't say something stupid or otherwise fall on their face during lunch, I'm
ready to hire them.

If you are in the position to do so, have the offer ready, and make the offer
right then and there. If you find the right person, ACT. Cancel remaining
interviews with your apologies.

Hope this helps. Good luck.

~~~
ljk
> _If you are in the position to do so, have the offer ready, and make the
> offer right then and there.

only if more hiring managers did that...

> _They more they talk the more they will reveal*

The* more they talk?

------
steven2012
Have a deep conversation about technical topics, have more conversations over
lunch. Try to see if the person is a good fit with the team. Check references.

Then hire this person.

If they can't code, if they are too slow, if they wont be productive at the
end of one month, then give them 2 months severance and let them go.

~~~
gargarplex
I have never received severance from an employer or known personally someone
who has (USA). How common of an occurrence is this?

~~~
sheepmullet
Very few will if you are fired, especially in the first month. In reality it
is win/win situation for the employer to offer a decent severance.

Without it bad fits and bad developers stick around for years or you get the
reputation as a "fire fast" company and a lot of good people don't want to
work with you.

------
MichaelCrawford
I would find it far easier to persist in my job search, as I expect would many
others, were my interviewers to avoid needless cruelty.

"I'm sorry but we are unable to offer you a position" while unpleasant to
hear, is at least what I expect when I don't get a job.

Just last week I was given a 24-hour challenge test. As I had to take an early
morning call from a potential client, I chose to get some sleep rather than
staying up all night to work. I did complete the challenge, with beautiful C++
code and a couple dozen unit tests.

Unfortunately they had a test that I did not, so my code tripped an assertion.
The CTO emailed me to say "According to the rules of our process, the
conversation ends here".

I'm dead certain he did not even look at my source; one of his engineers ran
the test. Most coders don't know what assertions even are - expect most HN
members do, but not coders in general - and it is uncommon for C++ coders to
write exception safe code.

This company has been advertising the position for months. Why was it so
important to stick to "the rules of our process"? Even if it was so important,
did he really have to say "the conversation ends here"?

I don't have a problem with whiteboard tests but this was actually the very
first time I've so much as attempted a challenge question outside of the
interview.

I'm going to bill them for my time; if they don't pay I'll take them to small
claims court.

~~~
lgieron
> I'm going to bill them for my time; if they don't pay I'll take them to
> small claims court.

After you're done, please please do a writeup of how it went.

~~~
MichaelCrawford
Your Wish Is My Command.

------
nate510
tl;dr: Essentially agreeing with soham.

I don't believe that there is any ideal process. A lot of it comes down to
having good hiring managers that know how to evaluate a candidate based on
their alignment to the company goals and the role they are being hired for.

Reid Hoffman wrote a great piece recently calling for a change in the
company/employee relationship from "join our family" to "help us while
improving your skills". He said that a standard interview question at LinkedIn
is (paraphrasing) "What would be your ideal role after you leave LinkedIn?"
The point of the question is to A) identify candidates who are motivated to
grow their talents, and B) communicate to the candidate that it's OK to
imagine quitting someday.

That's an example of what LinkedIn believes works, but it might not be
appropriate for another company, say that wants to hire programmers into more
of a stable career pipeline. Similarly, some shops might be highly specialized
in certain roles/technologies, making deep-dive technical interviews more
valuable than at a company that needs bright generalists to solve poorly
defined problems. There can be no one-size-fits-all solution.

One thing that's worked well for me is to have candidates meet a significant
number of the peers they will be working with. Another is to ask a quick
"weeder" tech question and then an open-ended architecture design question
(i.e. Describe how you would build the Facebook activity feed). Checking
references is also valuable. But these are just ideas and they may not make
sense (or be possible) in every situation.

------
soham
Interviewing is not a standardized test. On the spectrum of human
interactions, it's closer to a date than it's to a class test.

i.e. by definition, there isn't a perfect interview process. It's different
for different companies, teams and people. Everyone hires people who "fit"
their thinking. Case in point: Despite Google having <1% pass rate, nearly
everyone who applies gets a job somewhere.

It also follows, that the "ideal" process, even if one exists, is going to be
a combination of techniques. And even then, it's not going to be perfectly
correlated with job performance. Simply because there are so many human
elements in it and (figuratively speaking), human is the antonym of ideal.

You can take your time to really evaluate an engineer different ways, if you
are hiring slow, say 1 person a month or so. But the moment you hit scale (say
15 engineers a quarter), it's very hard to do better than what we do today.
The process today is the path of least resistance, which is what humans will
take when left with a deadline and quota to hire.

(About me: Founder of
[http://InterviewKickstart.com](http://InterviewKickstart.com))

------
staacyc
Interviewing and hiring is an expensive process so a one size fits all model
would be ideal. There are a handful of variables to consider, like company
stage, who's hiring, etc. At Grok + Banter, an early stage startup, my co-
founder and I have vetted a number of potential developers who were either
someone we know or they were recommended to us. Keegan sticks to conversations
about tech while I try to uncover goals and agendas to see if we can work
together to align individual goals and company goals. We look for people who
challenge our thinking in constructive ways (rather than demeaning ways). When
we speak with someone who seems adept at solving real world technical problems
as well as someone who is clear and forthcoming with regards to their goals,
we work together to create a 30 day contract that will either end employment
or continue employment with an extended contract after 30 days. What I have
found is that with in 2 weeks, give or take a few days, Keegan and I both know
if the person is a good fit. With in a year, we've ended three contracts.
Interestingly, all parties look great on paper and are able to talk (white
board) their way through problem solving, all three are highly intelligent,
unfortunately, all three lacked the ability to execute. We've also interviewed
and hired people that lack an ability to demonstrate their defining
attribute(s). This attribute inevitably starts to appear around week two-
three. If you can determine whether someone will execute or not as well as
draw out defining attributes in an interview, you may be well on your way to a
one size fits all model.

------
speedyapoc
After a basic call to make sure that the culture fit is ok and to learn some
background about someone, I believe the best way is to assign a small (4-10
hour), realistic, paid (market rate) task for the interviewee to work on.

I don't think that programming on a whiteboard is a good way to evaluate how
someone works, and many of the problems don't reflect the real world.
Interviews without programming can also favour someone with good charisma and
ignore programming skills. Instead, by providing a realistic task for someone
to work on, a company can better gauge how a person works, what solution they
come to, etc. The interviewee is also at home and in a comfortable environment
versus one with time pressure.

I've had two contracting jobs that evaluated this way, and both of the teams
were nothing but joy to work with. In the first one, I was commissioned for 10
hours to evaluate an existing codebase, refactor a little bit, give input,
etc. Their CTO agreed with my feedback and I was hired. In the other job, I
was commissioned to layout their mobile application's view controller and
navigation architecture. Once again, my solution was reasonable and I was
hired.

Neither of these tasks are unnecessarily algorithmic, tricky, etc. They're
simply real world problems and in both cases, even if I wasn't a good fit, the
company would have (hopefully) benefitted in some way or another.

~~~
carlosdp
I actually don't like the take home problems solution. If I'm currently
working somewhere, I really don't want to work on some interview task after
hours. I'd rather interview in person, talk about what I've worked on before,
go into specifics about my experience, and have the employer trust that the
fact I've worked at the places on my resume and am able to explain programming
methods as enough to evaluate my competency.

I don't know of any other industry (other than show business) that requires
people to audition for a job, especially not engineering fields.

~~~
brianwawok
Maybe offer a choice?

Some people hate homework. Same hate whiteboard. Offer the person their
choice?

------
NathanKP
I've been pretty involved in the hiring process at the startup I currently
work at. Our process works like this:

1) Phone screen of roughly 30 minutes.

2) Coding challenge. There are a couple different ones you can work on, such
as exploring and scraping data from an HTTP REST API programmatically,
building it into a report and submitting it to a different endpoint, or
slightly more complex one that involves reading raw socket data to download a
video stream, validate it's checksums, and extract it. This is an offline
challenge that candidates can do on their own time, using whatever tools, docs
they want. But the point is that candidate's choose which one they want to
work on, giving them some choice to pick the easy thing or the harder
challenge, and send it to us as a sample of their coding style.

3) Assuming the challenge code they sent us wasn't god awful they can usually
get an on site interview. During this interview they sit with a few engineers
they would be working with and each engineer has their own type of approach to
interviewing. Personally I have developed the following test that I find works
quite well:

[https://github.com/nathanpeck/wordladder/blob/master/ladder....](https://github.com/nathanpeck/wordladder/blob/master/ladder.js)

One advantage of a code exercise like the one above is that you can go as deep
as you wish depending on the candidate's skills. When interviewing a very
junior candidate you can focus on their understanding of how to organize code,
refactor it, and their knowledge of the basic JS toolkit.

For someone more advanced you can get down into details like "Is this a
breadth first or depth first algorithm?" or "How can this be optimized to make
it more memory efficient?"

------
zerr
Assuming we're talking about mid-career professionals. The fact that someone
is invited to the interview means that you liked CV. Now the task is to ensure
that CV is truthful - by discussing previous projects, experience...

After this discussion, if the interviewer is unable to tell whether the
candidate is lying or not about his past experience - this means he shouldn't
be interviewing in the first place.

And there is one thing to remember - false negatives, i.e. skipping over good
candidates, is _very_ expensive. Big G might ignore this fluctuation, but it
is much more crucial for those wannabe-be-big-G startups who mimic even how
interviews are held.

------
andrewstuart
No company gets hiring right. Accepting that is step one to rethinking hiring.

