
Ask HN: Whiteboard interviews don't work, what's your ideal technical interview? - coltonv
We&#x27;re hiring developers right now and from what I can tell recruiters haven&#x27;t figured out the ideal way to interview tech hires.
What do you guys think? I&#x27;d love to hear some opinions from the developers themselves.
======
BoysenberryPi
Dev here. I wish I could find a company that interviewed me like this:

\- Take home work immediately. Maybe it's just me but I'd rather not get this
after the phone screen and whatever else. Just give me the work and have me
submit it with my application or after I apply.

\- Phone screen. This is important. When I say a phone call, I mean a phone
call. I've interviewed with companies that refuse to have a regular phone call
and will only do video calls over skype/hangouts/whatever. This is such a
Silicon Valley thing. Guess what? Not everyone is in a situation where they
can easily setup a video call. I know I'm not.

\- Technical interview. Honestly the best technical interview I've seen is
where the company ask you to get one of your old projects, walk them through
how and why you structured the code the way you did, and then pair program
with someone on the team and add a feature to the project. If you don't have
past projects like that you can pair program on their code base.

When it comes to technical interviews and take home work, if you are asking
questions that can be googled in 5 seconds then you are just wasting my time.

~~~
afarrell
My company (GoCardless) does pretty much this except:

1) There is first a very short phone chat with one of our recruiters to see
what sort of role within our team you're interested in, let you ask questions,
and to make sure you're clear on the scope of the take home challenge.

2) For the first technical interview, we have you talk through your approach
to the take home challenge. Not everyone has publicly available code that they
can walk through and not everyone's publicly-available projects are good for a
discussion with any given one of our engineers. The pair programming interview
is also on a challenge that we've standardised and calibrated with the team.

------
conlinism
To me, the best technical interview is one where I am asked about the projects
I have worked on in the past and the ideas I have for future projects. When
considering the questions to ask, I think the most important thing is to
realize that not all developers have had the access to experiences that you
might have had.

Just because someone doesn't have many side projects doesn't mean they aren't
a good developer. Often times we assume that other people will have had access
to the same opportunities that we had - in college, in the workplace, and in
life in general - but this is a fallacy. Perhaps a developer had to work their
way through college and didn't have time to go to hackathons or work on side
projects. Perhaps she was so busy running a developer network at her last job,
in her last city, that she didn't work on something besides that. Or maybe
they just don't care about side projects.

I think, ultimately, the goal of a technical interview is to get the
interviewee talking about something that excites them. Seeing that an
applicant can get excited, spend hours learning something, and dig deep into
whatever they are interested in is the most important part of an interview.

Be careful with things like "cultural fit" as that nebulous term is often used
to excuse personal problems. If you can, initially evaluate the interviewees
work without looking at their name, race, gender, or other identifying
information. By doing this, you help avoid inherent, subtle biases that your
recruiters may have. Those biases exist everywhere, and the best we can do is
take steps to mitigate their effects.

Finally, I think trial jobs are greatly under-appreciated. Offer an
interviewee non-critical, non-trivial work on a week long contract basis. Pay
them industry standard and work with them on the feature. If that feels right,
hire them. If it doesn't, evaluate what went wrong and move forward from
there.

~~~
coltonv
So in the case of trial jobs how do you narrow down the candidate pool to the
point that you have few enough candidates that you can bring them in office
and let them work for a day without the office just becoming an unmanageable
Fiesta?

~~~
conlinism
Trial jobs happen at the point that you would be ready to hire that candidate.
This is after you have evaluated if they are excitable if they fit well in the
current culture of your company or would bring a good change to your company
culture, and if they have some level of technical expertise (they went to
college, worked at another development position, or have side projects).

~~~
jdhopeunique
Trial jobs would be ok for some candidates, but I would be furious if, after
spending a lot of time in the interview process, a company tells me only at
the end that the job is a contract to hire job. Trial jobs should be
advertised up front at the start of the process.

------
tutufan
As far as I can tell, whiteboard/quiz interviews are no better than coin-
flipping.

The best technical interviews I've been involved with seem to be mostly
conversation about what the candidate has done and likes to do, and what the
company wants and is like. Typically the candidate has a general impression of
the company from having heard about or read about it (or having friends that
work there), and the company has a general impression of the candidate from
their resume, cover letter, some public code/etc., and possibly common
acquaintances. That seems to produce the best results.

EDIT: More importantly, the key problem in hiring is not finding people that
are technically qualified. The key problem is assembling teams of people who
mesh well and are productive as a team. The world's companies are filled with
teams of brilliant people who don't work well together.

------
jdhopeunique
Personally, I like the hackerrank or leetcode automated challenge as an
initial first filter. It can be gamed by cheaters of course but that requires
some work on the cheaters part. It allows me to start the test at a convenient
time and in a standard format. Hopefully, it allows the company to be less
stringent about skill sets and other resume keywords and simply test more
people. Hopefully it makes job searches harder for resume spammers, cheats,
and fakes. I suspect even a few really easy problems from these sites would be
a useful filter.

Then perhaps a phone interview so that the company can confirm I'm likely not
cheating.

Finally an onsite white board interview or written test or at the computer
programming test. Yes I get nervous doing white board interviews but if the
difficulty level of the problem is not too great it is a useful filter.

I dislike long take home work because it requires too much of a time sink and
it could be abused by employers fishing for ideas, competitor information, or
just wanting free work for one of their problem areas. Algorithm tests on
hackerrank or leetcode or wherever are still useful practice but take home
tests can be too job specific and therefore prematurely requires candidates to
commit to learning a particular companies needs without the company committing
anything in return.

Questioning candidates about their experience in an informal manner by
conversation may be of some benefit, but is likely just testing how well the
candidate has prepared various stories about their past employment and past
projects. Unfortunately, abandoned projects that taught a candidate a lot and
served their intended purpose may not make as good a portfolio or story as
completed projects that were just cookie-cutter implementations of projects
found in books or tutorials or elsewhere.

The important thing for all of these filters though is that they be carefully
calibrated in difficulty to the companies needs and job sourcing pool. If your
large top-N company carefully optimizes it's customer funnel with a/b testing
but has no real employee hiring sanity checks, then that is not good. Randomly
have one employee at the company white board interview another employee at the
company from a different department. Secretly test your interview process with
known good programmers from outside the company.

------
skylark
From Laszlo Bock, former SVP of People Ops @ Google.

> [1] The best predictor of how someone will perform in a job is a work sample
> test (29 percent). This entails giving candidates a sample piece of work,
> similar to that which they would do in the job, and assessing their
> performance at it.

Based on the statistics he provided, it seems like a combination of work
sample and behavioral questions will typically produce the best results.

The problem is that Google gets ~2 million applications per year, so there
needs to be an aggressive, quick funneling process. If you're a smaller
company, you can definitely afford to slow down and give candidates more of a
chance.

[1] [https://www.wired.com/2015/04/hire-like-
google/](https://www.wired.com/2015/04/hire-like-google/)

~~~
jotux
From this paper:
[https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2853669](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2853669)

> On the basis of meta-analytic findings, this paper presents the validity of
> 31 procedures for predicting job performance and the validity of paired
> combinations of general mental ability (GMA) and the 29 other selection
> procedures. Similar analyses are presented for 16 predictors of performance
> in job training programs. _Overall, the two combinations with the highest
> multivariate validity and utility for predicting job performance were GMA
> plus an integrity test (mean validity of .78) and GMA plus a structured
> interview (mean validity of .76)._

------
bjourne
I believe pg wrote in one of his essays that a hacker can immediately
recognize another hacker, but it is impossible for a non-hacker to do so.
Therefore the best method would be to let your best developers handle
interviewing as they would be the best at recognizing skill.

~~~
essofluffy
In that same essay I think he also mentioned you can't know fully how good a
hacker is until you've worked with them. I still agree that the best
developers will be able to see talent.

------
probinso
Ask hard questions. It is important to understand the boundaries of a
candidates knowledge. I don't trust company that lets me off easy, it feels
like they aren't actually looking for a good fit.

Asking for a prepared presentation is a nice alternative to take home coding
(If applicable to your work culture).

Phone call trumps videochat every time. Our communication isn't strengthened
by a video chat unless there are prepared visual aids, don't solve problems
that don't exist.

Let me know who I will be meeting during 'in person' interview. What teams are
they on.

Live coding with a current employee is a good practice. Keep problems small,
and loosely specified.

------
mbrodersen
Whiteboard interviews DO work - if done right. Don't ask computer science
questions. Instead give the interviewee real world problems to solve. And find
out how he/she attacks the problem. What he/she would look at to solve the
problem. What tools he/she would use and why. The key thing is to learn how
the interviewee _thinks_ through a problem. And how well the interviewee
explains his/her thinking. I have interviewed and hired people for 10+ years
and can testify that this approach work.

~~~
wapz
If you've hired a lot what do you recommend for someone like me that solves
with paper and pen always. I always have my tiny notebook and I'm very good at
solving problems writing things down but on the whiteboard I just can't see
everything. Not only that, but while I'm writing I tend to lose my train of
thought. I'm fairly certain it's not from being nervous but just being so
different than normal.

Would you throw someone out if they told you they are very comfortable solving
with a pen and paper in front of you instead of a whiteboard?

------
chollida1
I've hired maybe 30 to 40 people over the past 15-20 years and interviewed
atleast an order of magnitude more.

I don't claim to have things figured out but I can create a list of things
that owrk and things that don't work.

Things that work:

\- white board interviews, they let you get a dialog going with the candidate,

\- take home work, they let the candidate work at their own pace

\- showing work samples, whether that's a hacker rank type thing or github
repo or open source work that you use.

Things that don't work:

\- white board interview, some people just aren't good at them and they don't
map well to real world work

\- take home work, its insulting to candidates that you expect them to do
something on their own time without pay

\- showing work samples, github repos, etc. To easy to fake or take others
work as their own, some people don't work on open source-able code, etc

Hmm, so we can establish that no matter how you interview you'll cut out some
good candidates. I think we've all come to the conclusion that there is not
global maximum on how to hire. In my opinion, you're really just going to have
to pick one option and stick with it.

As an aside, as a candidate that kind of sucks, if you don't like white board
interviews then you're probably not going to work at a company like Google,
Microsoft or Facebook. Your choices are to accept that you won't wokr there or
to get better at white board interviews.

Well then we could start by looking at what really successful companies do.

The worlds most successful hedge funds tend to not allow people to really
apply, instead they look around for who is doing great research and then reach
out to them.

Now its easy to start by saying this isn't applicable to most other companies
and you'd be right, your company probably can't have their pick of MIT
professors, but it does open up an interesting line of thinking.

What are you looking for in a candidate and how do you find people who are
doing great work in that area.

Honestly, I think finance has this area figured out better than tech does.
They either hire new grads by doing very heavy university recruiting or they
hire from competitors.

In finance the saying is that by the time you are 30 you don't need a resume
anymore as wall street is so incestuous that everyone knows everyone else. You
are only 1-3 degrees of freedom away from any other person, which means that
to hire someone, I just need to pick up a phone and ask the 5 people I know
that the candidate has worked with to see if a candidate is worth hiring.

~~~
mrfusion
Can't you tell if someone is taking someone else's work as their own? It seems
like they wouldn't be as familiar with the code and wouldn't be able to
explain why things are done certain ways. I feel like I'd be able to tell.

~~~
wapz
If they show you a mid-sized project it would be very hard to look at it and
have a better understanding than someone who intentionally ripped it off and
read up on it themselves. That being said, I'm sure if you _really_ grilled
them on specifics you might be able to catch them off guard. The problem
though is that is showing that you don't trust them from the start as opposed
to showing them that you are interested to see what kind of coder they are. I
would leave an interview if the interviewer seemed to only be asking questions
to throw me off intentionally (I've done it once before).

