
How to interview engineers as a startup - tomblomfield
http://tomblomfield.com/post/61760552398/startup-series-part-1-interviewing-engineers
======
shaydoc
"...calculate row n of Pascal’s triangle or implement a basic sorting
algorithm, for example"...

I assume you let your candidate use google to search for solutions to these
problems if they ask ?

Personally speaking, I think if you are going to put an engineer on the spot
about some stuff they may or may not be familiar with, you have to allow them
the appropriate toolset to solve these problems...you see for me, asking a dev
to implement a sort algorithm or solution pascal's triangle doesn't really
tell me a whole lot on its own, especially if the person has no entries in his
"memory" for these problems, I think it is unfair and probably irrelevant to
what they may do in a normal situation.

~~~
tomblomfield
Yes - absolutely. We'll actually give a definition of whatever algorithm we're
looking for, often with specs provided.

I don't really care if you remember the definition of merge sort or bubble
sort, but you should be able to code it up in a few minutes given the
definition.

~~~
tstyle
For the homework assignment, do you give them a time limit(i.e 1 week) to turn
in their work? : )

------
probablyfiction
> In my experience, relevant, realistic coding challenges are by far the best
> indicator of ability. They should take roughly 2 hours, to be done in the
> candidate’s own time

That's a great idea for the company, but from the perspective of the
candidate, what a gamble of their time. They give away two hours of coding for
free for a job they may not get? If you're going to demand that the candidate
do some coding for you, at the very least give them a token amount of money in
exchange for their time.

Yes, when you're hiring a lot of people, the cost can add up, but you
shouldn't be demanding such a large slice of someone's time without being
prepared to compensate them fairly.

~~~
solve
I've done a few take-home interview coding challenges -- absolutely would
NEVER do them again. Both that I did ended up just getting tossed in the
trash. I was able to get in contact with my interviewers later, not one of
them had even seen my take-home work.

If you're a company using this tactic, hopefully you're aiming for only the
most desperate or green candidates, because that's exactly what you're going
to get.

~~~
coreymaass
I've done a few as well. It left a bad taste in my mouth every time, too. It
never showed what I could do, and I felt it was wasting everyone's time - the
company's and mine.

Now I usually make a counter offer. I ask them to find a small project they
can hire me for. It's a better use of everyone's time.

~~~
tomblomfield
We spend a long time (30+ mins) reviewing every single coding challenge we
receive. We also use it as the basis of an hour-long interview - probing on
the choice of libraries, asking extension questions, etc.

But it's important that you weed out no-hopers at the CV- and phone-screen
stage, otherwise you're going to get hundreds of coding challenges and you
won't be able to give them due attention.

------
columbo
Step (1) Know what you need.

I wish more companies were honest about the exact type experience they
actually need.

[A]: "I need someone to fix bugs. We're talking head-scratching-mind-numbing
bizarre behavior bugs. Most of the time you'll be starting out in the UI
tracing connections back to the API then walking through the API into the core
system until you can find out what actually happened."

[B]: "I need someone to make the application faster. It's an archaic
enterprise system that is responsible for performing very important
calculations. It's hundreds of thousands of lines of banking logic wrapped
behind all sorts of enterprisey buzzwords. The first task you will have is to
sort out why we are getting intermittent heap errors. You'll need to setup
everything yourself as we have no dev environment."

Both these roles might be in the same technology space, and both will probably
have identical resumes (5 years of experience with <foo>) but the types of
people you will want to hire and the questions you will want to ask them are
going to be vastly different.

~~~
eshvk
I have thought about this problem for a while both as an interviewee and
interviewer. Especially because this problem is magnified when you want to
hire a data scientist; your ideal data scientist can be anyone from a SQL
monkey to a statistician to a programmer who likes the title data scientist.
The problem however is fear. Most small companies have short term goals that
are in flux. So managers want this caricature of a human being: the ubermensch
who does everything. The "best" of the "best". So the interview process is
fucked up.

------
Peroni
_Get an applicant-tracking system that stores CVs and the result of each
interview (with detailed notes), so that you can compare candidates based on
data._

I highly recommend trello for this if you're a startup. It's free and it's
incredibly simple to manage CV/Candidate tracking with simple lists.

Great article Tom.

~~~
benjaminwootton
Is it OK to store candidates personal information in a free SAAS?

The fact someone has applied to you could be highly confidential.

~~~
Peroni
When a board is private, only those users added as members to the board can
view or edit it.

~~~
Sae5waip
Thats not what he meant.

What he meant is that you are putting the private data of the applicants on
the internet, on the servers of another company. I too find this highly
dubious.

~~~
Peroni
That's how every candidate tracking system works. Paid or otherwise.

~~~
Sae5waip
Not everything has to be a web application, and not all web applications have
to be hosted by a third party.

------
jurre
I've interviewed with them for an internship about a year ago and although I
didn't end up getting the job, it was a very pleasant experience and I
actually learned quite a bit in the process.

~~~
mathattack
It's a good sign of how humane the process is if you left it with a good
feeling about the company.

------
walshemj
You should not need 4 interviews to hire some one and I have my doubts about
cutesy algo based coding challenges.

~~~
mseebach
No hire.

Candidate came off as smug and didn't even try to qualify his opinion with
supporting evidence or experience.

~~~
walshemj
:-)

Well I do have a number of decades of experience dealing with hr/ir issues and
I am sure that any HR professional would agree with me that 4 interviews is
two to many - I might allow a phone interview plus two FTF.

------
Bahamut
I just changed jobs recently for a company currently growing rapidly - I only
had one in person interview and one phone interview where I got notified the
intention to offer. Nobody gave me any coding problems or such (although the
company does ask these questions).

My situation might be more special though - I'm almost certain my GitHub &
LinkedIn and how I came off when interacting in person was more than enough to
convince the company. I'm clearly one of the most knowledgeable in a
particular technology in my area. I was very happy that my time was not wasted
though, and the company was good at assessing me (maybe not even realizing how
much more I could do).

I think it's best to be flexible depending on the candidate.

------
n1ghtmare_
Wow honestly what is this company ? 4 interviews ? Hours of coding ? Seriously
? You better be the best bloody company in the world.

~~~
eshvk
What surprises you here? This is standard operating procedure for most
startups, including the 'hip' ones that write navel gazing bullshit.

~~~
n1ghtmare_
Yeah unfortunately it seems to be a standard these days its kinda ridiculous
if you ask me. I'm pretty sure I'd give them a special finger after the second
interview, but that's just me.

------
bane
My list:

1) Be respectful of candidates: their knowledge, their background,
communicating with them, and most importantly their time. You may think your
startup is a very special snowflake, but to a candidate, it's application #107
out of 243. Nothing gets my goat more than a company that absurdly and
absolutely wastes my time.

2) If a candidate makes it through a contact, that means they made it through
a level of the interview process and were downselected and are now part of a
smaller pool. That's how they'll perceive it, that's how you should perceive
it. I've seen all kinds of nonsense about repeated interviews where the
candidate thinks they're getting close to landing the job, but the company's
internal processes are just getting enough interviews to fill out the hiring
committee before rejecting the candidate.

3) If you're going to do the wrong thing and make a nepotistic selection, save
everybody the time and energy and just hire that person from the start. Don't
engage in kangaroo application processes just so you can claim it was
competitive. It wasn't.

4) Have a solid idea of what position the candidate it interviewing for. Don't
engage in "holistic" and speculative hiring where you hire a guy because he
seems swell and you end up bouncing him around from position to position to
try and make him fit.

5) Ask questions that are actually relevant for the work the person will be
doing and might grow into. If it's a CRUD developer working on backend data
stuff in SQL and Python, don't waste their time with irrelevant algorithm
questions and score them based on their knowledge of node.js and functional
programming -- or even worse, non-engineery things like how to negotiate a
fulfillment center contract and keep inventory loss to a minimum. Chances are
you need them to do lots and lots of X, not write bubblesort or regular
expression optimizers or b-trees. If you _do_ need them to do that kind of
work, interview them for that.

6) If you want to have a quiz, or coding challenge or whatever, give them
problems that represent the simplest possible version of what you need them to
do. Fizzbuzz is a great example. It exercises very basic programming skills
without spending time asking them to write a bubblesort. If they list SQL on
their resume, have them write a very simple SQL statement. 9/10 can't do these
basic skills tests and there's no reason freaking them out and wasting your
time on more advanced questions. You already know that if they can't do these
things they won't fit in.

7) Contact your candidates and reply to them. When they apply, acknowledge the
application. If you reject them, be a grownup and at least send them a nicely
worded email, "...wouldn't be a good fit here...". When you are looking for a
job, and you don't know the outcome of an interview at a place you really want
to work for, you might delay or pass on faster moving opportunities to wait it
out. Rip off the bandage and let them know they didn't make it so they can
close it out and move on. It's just common courtesy.

8) Don't be confrontational. Many engineers are terrible at the soft skills
during an interview and are already freaked out by the experience. It's on
your home turf, you have all of the advantage, and it's concerning something
that might have impact on the rest of the candidate's life. Be a good host!

9) Interviewers need to show up or make themselves available by some alternate
means. You won't believe how many times I've heard stories of people showing
up for an all-day interview (hard to get the time to do that if you already
have a job), only for half the panel to not show up that day. Those people
shouldn't be part of the interview panel if they don't take it seriously.
Remember, #2 above, if the candidate finishes out those who are there, and is
then called back in to interview with those who couldn't be bothered to make
it...they perceive that as advancing in the process, not filling out your
internal hiring board process.

~~~
tomblomfield
Great list.

Regarding points 4 & 5, I think it really depends on what stage of company you
are, and what kind of work you're doing. When you're making your 4th, 5th, 6th
hires into an engineering team, you normally want a pretty diverse range of
skills.

We've had front-end guys work on SQL code and contribute significantly to our
devops tools. People who are interested enough in programming to be constantly
pushing themselves to learn new technologies are extremely valuable.

~~~
potatolicious
The "everything person" is a valid position - though your likelihood of
landing this sort of unicorn might be questionable. I think GP is arguing
against the extremely open ended sort of hiring - the "wow you're crazy good,
but we don't have a real need for your niche right now so we'll bounce you
around until you get frustrated and leave".

At some larger companies also, interviews tend to be "holistic" even when the
roles are not. You might be hiring a frontend person but the interview panel
will be consisted mostly of backend people. I've unfortunately been through
interview cycles (from the employer-side) where this was the case.

Worse perhaps are when recruiters deliberately mislead the candidate into
coming in. I've seen in the past situations where frontend or backend-centric
people come in to interview for the exact opposite position, and are pissed
off (rightly so) because they've made their specialization clear to their
recruiter contact but was told the position was still relevant to them.

~~~
bane
You pretty much got most of what I was thrusting at.

I'd only differ with you on larger companies wanting more holistic folks. In
my anecdotal experience, large companies are usually hiring to fit somebody
into a particular business process block and want ultra-specialized people for
that block.

In my area it's not even uncommon to see job reqs for experienced people using
very specific development tools as well.

It can get weird when that role is a new industry one, like less than 5 years,
and they want a senior person for that role with 5-10 years of experience in
it...which is clearly impossible.

------
progx
Really? For a startup ?

Yes, you need good people, but you need good people that work in a good team.

Only after some time you can check if somebody is a good programmer or not and
if he/she fit to the team or not.

All you write reminds me on big companies, they test the people, because when
they hire them, they hire for many years (if you are in, you are in).

But on a small agile startup, is that really necessary? You see within some
days if somebody perform or not.

~~~
tomblomfield
Yes, for a VC-backed startup between 8 and 50 people, which is what I'm
broadly trying to write about.

You can check whether someone is a good programmer pretty quickly during
interview & practical exercises, it turns out. The best way is a complete-in-
your-own-time 2-hour coding exercise that's relevant to the job they'll be
doing.

The cost (both economic and in terms of team morale) of sacking someone is not
inconsiderable.

------
pnathan
I particularly like the point about how each question should result in a
hire/no-hire result.

