
Interview Questions for Hiring Top Performers - gkop
http://firstround.com/review/hire-a-top-performer-every-time-with-these-interview-questions/
======
kafkaesq
_What to ask:_

 _We look for a time the candidate wanted something so badly, they were
unstoppable in pursuing it. Or a time they overcame an obstacle,” Hamilton
says. As you listen to the answers to those questions, pay close attention to
both the tasks and the duration described. “Try to get a sense of how long
that person can stick it out. How long are they going to beat their head
against a stats problem?”_

Good qualities for an academic (or inventor type), but doesn't seem to have
much correlation with the kind of "grit" needed for most commercial
development work.

In which, the vast majority of the time, we benefit the most from "triaging":
that is, discerning when _not_ to go down rabbit holes, and when _not_ to
"bang our heads" against problems when it may be better to simply... route
around them, or leave them be until some sunny day arrives when there's a
legitimate business need to properly resolve them, and we have appropriate
resources available to do so.

------
brogrammer90
If a candidate actually had any of those attributes, never mind all seven,
they wouldn't be interviewing at your company. Companies need to realize 99%
of candidates are B players, and 99% of their own employees are too.

~~~
soham
Very well said.

Interviews are not standardized tests. They are closer to a date/sale, than
they are to a test. Same answer in an interview is likely to be interpreted
differently by different people, very much like you're on a date.

The more you think of interviews like a test, the more you'll agonize about
it, and the more you'll feel writing and believing such gospel-ic articles
like this. True, there are a few basic common things, but beyond that, it's
subjective by definition.

Whoever YOU think fits your values and ideas, is an A player. Everyone else is
"B", because they are not exactly "compatible" to your thinking.

At [http://InterviewKickstart.com](http://InterviewKickstart.com), that's what
we find. Every company has a different viewpoint of what a great engineer is.
After a generic training for interviews, it's about compatibility with the
companies.

~~~
hinkley
My current gig asked me a bunch of questions about tools and techniques...
that I am unable to use to do my job.

Suffice it to say that there were some hurt feelings when the contract
actually started and I saw how they were actually doing their work. If I knew
then what I know now I would have continued looking.

------
dropit_sphere
I have a sneaking suspicion that hiring is like stockpicking: after an initial
filter, monkeys with darts would do just as well.

The problem is: if the interviewers are "constantly learning," (and they are,
just ask them), then they are consistently making mistakes that the future,
more learn-ed version of themselves wouldn't make. To take an example from the
article: interviewers are to ask candidates about some time that they used
data to make a decision. What if a candidate responds with a story in which
they realized the data were pointing the way to a local optimum that the
candidate cleverly avoided? To you, smart HN reader, you are perhaps
impressed, and tempted to slap the candidate on the back and rejoice in your
shared membership in the club of those who've discovered ambiguity
("Complexity, such a bitch, right?"); but if the interviewer but dimly
remembers calculus but is proud of themselves for using a sentence with the
word "data" in it, then the candidate's response will come off not as an
indication of the ability to "ask the right questions," but a sign of non-
cooperativeness, "irrational refusal to let decisions be driven by the data,"
and perhaps intellectual arrogance.

For interviewers, I have no idea what to tell them. For candidates, I can only
say: Lie, and then follow through. If they are looking for "grit" but you had
a privileged upbringing, make something up and commit to yourself to be grit-
ful[0], rather than saying what's in your head, which is "I haven't had much
hardship but I get the importance of resilience." If they ask for your
greatest weakness[1], say _anything but your greatest weakness_ ; instead talk
about the difficulty of coping with _other_ peoples' weaknesses ("I don't deal
with my father's alcoholism as well as I ought," or "I get impatient with lazy
people.").

[0]This is basically what the sales people in your org do, which is why you
get those requests.

[1] This is a dishonest question, because there are wrong answers. The only
acceptable answers are weaknesses that the interviewer shares; but by making
that the only right answer the interviewer is implying that they have the
"best" weakness and therefore have _no_ "real" weaknesses. If they don't
introspect honestly, why the hell should you? Or, more charitably, which is a
better approximation of reality, that you are both about equal on your paths
to enlightenment, or that you are an imbecile? Is truth best served by
communicating the former through omission, or the latter through facts?

~~~
chad_strategic
\-->I have a sneaking suspicion that hiring is like stockpicking: after an
initial filter, monkeys with darts would do just as well.

I couldn't agree more. I have probably had more interviews / initial
conversations in the last ninety days with tech recruiters. I would say about
95%, knew absolutely nothing about what they where doing, they ask me the
generic questions... What are you strengths, weakness? What was a time you
where in a bad situation... A lot of times I could figure out that they hadn't
read to the bottom of my resume.

Then there is the Tech guy questions: What is OOP? What does it mean when you
get this error? What is the difference between PHP 5.6 and PHP 5.5, tell me
right now!

I did run across some recruiters that actual read to the bottom of my resume.
Usually, they where the ones that figured out I was over qualified for most
positions.

The best job I ever had was in the Marine Corps and they never asked me any
questions, they where happy to have me sign the contract!

In the meantime, I'm looking for work... [http://www.strategic-
options.com/resume/?=HN](http://www.strategic-options.com/resume/?=HN)

~~~
e12e
> In the meantime, I'm looking for work... [http://www.strategic-
> options.com/resume/?=HN](http://www.strategic-options.com/resume/?=HN)

Clearly I'm not the target audience (I'm not in a position to hire, in fact,
I'm also looking for work), but I'd be more inclined to read to the end if a)
It rendered anything before allowing js, and b) if there was a link (or
preferably content) at the top, just listing the bullet points.

Looking at the html-source, I see there's a link to a doc-document buried at
the end of the rabbit hole -- I humbly suggest whatever hides in there should
also be exported as html, and linked prominently from the landing page.

That said, I'm assuming your reference to people not reading your whole resume
is from applications where you've submitted a cover letter + traditional CV?

I should also add, that your resume-site has some impressive points, and good
visuals -- I just don't think busy, technical people would want to waste time
on that, if they could just see a short list of some of the good things you've
done. If those details means nothing to them without pictures, then they're
probably not technical enough to be able to make good decisions regarding
hiring for technical positions. Personally, I don't think being overlooked by
companies that have bad management/HR is a bad thing. I guess it depends who
you see as your audience?

~~~
chad_strategic
Thanks, I agree with most of what you said.

I made the visual resume, mostly for tech recruiters and non technical people
/ business owners. (Definitely not CTO's), I also use it to see if the tech
recruiters actual view my resume, which a lot of time they don't look it at.

Your right, being overlooked by bad management is a blessing in disguise.

------
WhitneyLand
This (like most interview processes) seems very unscientific and error prone.

She doesn't mention correlation of their process to successful outcomes and
until you do that you're guessing.

Also doesn't mention reviewing quality of past work which I'm guessing has a
better correlation if the contributions are properly vetted.

Unless you're doing A/B testing you'll never know, and are certainly screening
out candidates who could have ended up being star players.

~~~
evanpw
Here's a meta-analysis of the empirical literature:
[http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...](http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%20Validity%20and%20Utility%20Psychological%20Bulletin.pdf).

TL/DR: Give the candidate an IQ test and a work-sample test. The next best is
an "integrity test", which actually measures mostly conscientiousness. You can
totally ignore years of experience and education.

~~~
e12e
What is an "integrity test"?

~~~
e12e
Reflecting a bit on this, and on my comment above re: CVs -- I'm inclined to
just start applying to jobs with something along the lines of: I'm above
average IQ[1], you can send me a work test at <email>. If the test goes ok,
lets talk.

Actually, in this light, codeval.com makes (even) more sense.

[1] Sadly, not impressively high. I forget what score I got last time, IIRC
around 130 or so. Then again, I also have pretty high "Social IQ" (self-
assessed ;-), and the tests don't cover that.

------
PaulHoule
Why don't we ever hear about management tips to get more performance out of
the people you have?

~~~
TheCowboy
That information is out there, but it's much easier to focus on changing or
improving things that don't involve management practices and organizational
behavior, especially focusing on people who aren't part of the business yet.

There is a common flawed view that if you can focus on hiring the best people
then you won't have problems with performance. It can also result in high
turnover, and hinder performance when the rest of the equation isn't weighted
heavily.

Focusing on things other than hiring also comes with politics, because it
involves admitting that something isn't perfect with the organization.

~~~
vonmoltke
> There is a common flawed view that if you can focus on hiring the best
> people then you won't have problems with performance.

I think, more specifically, there is a common flawed view that hiring the best
_engineers_ means you can ignore _management_ and have them "self-organize".
Related to this is the false idea that a room full of really smart people
don't need dirty things like "process".

Nearly every time I get someone to give me a concrete example of why a false
positive is dangerous I find a management flaw that is the root cause of the
damage.

------
dkarapetyan
This just in: "How to do X by doing Y". The only way to get good candidates is
to optimize your hiring pipeline to the point that you can hire someone in
less than two days and fire them within 2-4 months if it becomes clear things
are not working out. Doing anything else is just a waste of time, money, and
effort. There is no proxy out there that will give you only the candidates
you're looking for.

------
gearoidoc
This is a puff piece thinly veiled as a guest post.

~~~
CPLX
The "First Round Review" is an embarrassment, they push out content-free
fluffy valley conventional wisdom every day or so, an unfortunate number of
which make it to the HN front page for some reason.

~~~
gearoidoc
I'm confused as to why this made the front page (and is _still_ there).

The content isn't new or noteworthy.

I suspect some strategic upvoting but maybe I'm just paranoid.

~~~
mildbow
See what you did there?

You called attention to the strategic upvoting.

Which means you get downvoted.

Welcome to real HN where groupthink is the only think.

~~~
gearoidoc
Couldn't care less about karma tbh.

------
mildbow
Meh.

Here's what all these HR sourced "top performer" filters are missing: top
performers are a nurture not nature thing i.e they are a function of the
environment.

What's the incentive for a top performer to work for you? We hear a ton about
how the best people "take ownership", but a lot less of why they should build
your company for you (rather than starting their own).

The most important thing a manager/CEO can do is to align the company's
success with the individual. That's how you make top performers out of people
you already have.

~~~
czivelonghi
I agree, if you want good people you need to create an environment in which
they can thrive. They need challenging problems, stake in the company, and
most importantly get out their way and let them build stuff. Great companies
are built by engineers and not the HR dept. or CEO's with an accounting
background.

------
lordnacho
Still waiting for an HR blog where they checked the people they didn't hire
against those they did.

~~~
ryandrake
Yep. Nearly every 'article' about how to hire suffers from selection bias.

Here's what I'd like to see: Find some super-talented folks who are critical
to the success of whatever they're currently working on. Have them name names
of companies that turned them down in the past. Then interview HR from those
companies to find out what they did that filtered those people out of their
organization. This kind of study would probably be impossible for a number of
reasons, legal and otherwise, but I bet it would be a fun and informative
read.

~~~
vonmoltke
The big problem with this idea is that many will assume, rightly or wrongly,
that in spite of someone's success at $COMPANY_X that they still would have
been bad for $COMPANY_Y. I am personally of the opinion that a good engineer
is a good engineer, and a bad engineer is a bad engineer. Organization and
culture can amplify or attenuate the "goodness" or "badness", but cannot make
a good engineer bad or vice versa. I also think _way_ too much emphasis is put
on culture fit, however it gets defined.

------
aaroninsf
TL;DR: "How would you convince me you are willing to exchange professional
services for currency?"

------
trjordan
Specifically, I'm not sure these characteristics are the perfect set for all
interviews. I only spent 5 minutes reading the article, after all :)

But, I like that these are all personality traits with no easy answer. I get
the feeling that an interviewer trying to craft questions to test "grit" or
"impact" will discover good stories about the candidate. They'll probably push
the candidate to talk about real challenges, or explain below-the-waterline
projects they thought were important. Assuming there's a technical screen that
validates that they're woefully uneducated (I'm thinking FizzBuzz, not 3 hours
of red-black trees), it feels like an interview designed to capture these
attributes would be pretty probing, and if the topics were divided among
different interviewers, they'd all get different angles on the candidate.

That seems good. Interviewing for the nominal job description is easy to game.
This checklist, if thoughtfully prepared for, seems harder to game.

------
yanilkr
All Hiring/Interviewing blogs should have a disclaimer like this.

Hiring/interviewing is like judging a medicine by its packaging. It is error
prone and there is little you can do to avoid making bad decisions.

Many people get good at it after making so many mistakes. It is much easier to
ask an expert in the field help you choose a candidate for the role.

------
78666cdc
As a person that has interviewed for and gotten jobs at multiple Fortune 500s
(whose tech departments varied from abysmal to impressive effective) and as a
person that has been a part of the interviewing/hiring process several times,
I appreciate the attempted formalization of interview metrics and see more
than a little value in the metrics described in the article.

To me, though, there is one "metric" that trumps all the rest, and that is
personality fit with existing employees. If you have two brilliant software
engineers that just absolutely cannot stand each other, nothing will get done.
The best interview experience I've had, personally, has been to be placed into
a technical interview - "start programming application X that does Y" \- with
two existing employees looking over my shoulders asking me why I'm doing what
I'm doing and why not something else.

The technical justification for my chosen solution to the technical interview
question didn't matter much once it was clear I covered a base level of
technical ability, but it was incredibly valuable that I and the employees
looking over my shoulder didn't want to get into a fistfight after an hour -
it meant that we could amicably disagree, support our reasonings for our
arguments, and then not have sticks up our asses when the other person's
proposed solution ended up being the right one, and then gracefully defer and
say "thank you for the conversation and debate." That meld is immensely
important.

That said, I do recognize that it becomes increasingly difficult to gauge this
sort of interaction with increasingly large teams. My experience interviewing
and being interviewed is limited to cases where it's been a team of 3 - 8
people. For larger teams, perhaps such a formalized approach makes sense. I do
not have any experience that informs me as to whether it is or isn't.

But then I have to ask whether there should be a team of more than about 8
people without a dedicated manager. I don't have anything more than a gut
answer to that question, but my gut says no.

------
brianpgordon
There seem to be a lot of questions of the form

 _Have your candidate tell you about a time ________

I hope they're giving their candidates time to prepare answers to these. I
feel like I wouldn't be able to think of anything during the interview.

------
deeteecee
I will just say again like I always do, there is no solution to the problem.
You hire, you try, and if no good, you fire.

I have some thoughts regarding possibly giving the interviewer some more
leeway (like for engineers, don't start off with solve this first) but haven't
thought enough about that.

------
klenwell
Interestingly, this rubric outlined in article applies the advice Daniel
Kahneman gives in his section on interviews in Thinking Fast and Slow. As he
acknowledges, it isn't perfect but in studies it generally shows an
improvement over the sort of gut-level hiring most people end up doing.

------
jameshart
So yeah - if you think the technical interview process is broken, take a look
at this and realize that at least in our business there is _hope_. Hiring
someone for _non-technical_ jobs? This is the best you can do.

------
wyclif
"Top Performers" [https://youtu.be/yoy4_h7Pb3M](https://youtu.be/yoy4_h7Pb3M)

------
graycat
Given a person who knows nothing about medicine, then might sell them on leech
bleeding.

Given a person who knows nothing about technical work, might sell them on the
OP.

Bet if take five interviewers and have each of them read the OP and apply it
to interviewing 100 people and compare the results, will find (1) validity,
the interview results will on average be a poor predictor of anything
important and (2) reliability, the interview results will scatter with huge
standard deviation. That is, it's throwing darts where the scatter is huge and
the center of the scatter is far off the center of the target.

Uh, didn't see where there was some good double blind tests of the OP for such
_reliability_ and _validity_. Uh, the OP was big on _analysis_ \-- good, now
have them tell us about their 7 devices and reliability and validity, that is
social science 101.

Let me just guess, from reality and common sense, what is going on with the
OP: The whole OP is essentially just _plausibility psycho-babble_ that HR
interviewers would like to entertain. It's aimed at HR.

The OP is yet more evidence that under no circumstances should anyone in HR
have a substantive interview with an important candidate for anything
important.

Instead, HR should smile, be nice, offer water, coffee, tea, soda, smile, be
nice, give directions to the rest rooms, help with the interview schedule and
the organization chart, hand over the benefits packet, smile, be nice, help
with the travel and lodging arrangements, help with expense reimbursements,
smile, be nice, etc.

E.g., the OP wants to ignore school work but wants to emphasize ability with,
what was it, _analysis_?

Okay, I can understand why an HR humanities major would get confused here, but
a good solution is to look for majors in math, physics, electronic
engineering.

The OP wants to give as a test of ability with _analysis_ something with
Excel. Upchuck: I avoid Excel because it is nearly always way beneath me -- I
use much more powerful tools. Excel will do Lagrangian relaxation, the fast
Fourier transform, Wiener filtering, least squares, multi-variate spline
interpolation, L. Breiman's random forests or even CART, stochastic optimal
control, solve some differential equations? I don't think so. I don't waste
time with Excel. That's like testing a cardiac surgeon on cleaning bedpans.

"Top Performers"? Right, maybe for toilet cleaning. Even then they'd be at
risk of mixing the Muriatic stuff with the Drano stuff -- don't do that!

HR tries again, fumbles with the ball, drops the ball, trips over the ball,
falls on the ball, loses the ball, is face down in the mud.

HR: Smile and be nice.

------
sidcool
A lot of questions here seem superfluous. They can be easily bluffed on. Also,
the questions assume that perfection exists.

------
emodendroket
Well, at least it's a change of pace from the same handful of technical
questions people post over and over again.

------
lazyant
"For more technical roles, you can even build a timed Excel test with some
practice problems or logic tests.", what? is the Excel page for the
interviewer or the candidate?

In any case, you can use whatever questions you like, they are very trainable
and people good at BS and making up anything quick will do well at those.

------
unixhero
As a candidate, this is the best interview guide I've read.

~~~
kabdib
Trust me, if they're interviewing like this, you don't want to work there.

~~~
unixhero
I'm an MBA type with a lot of compsci background, and I get this bullshit all
the time. These advice were poignant and to the point of actually suggesting
to me how to think and how to answer those incredibly silly questions. That's
what it's like over at the Corporate Overlords in Scandinavia at least.

------
speaknotyet
Where is the actual data that back up this recommendations?

------
rhodysurf
Aren't these all pretty obvious?

~~~
WhitneyLand
They may seem intuitive, but that doesn't prove the questions or approach
valid.

------
a3voices
This would be very politically incorrect, but I'd wager the best way to find
top performers is to ask "What do your parents do for work, and how much money
do they make?" There is a very high correlation between the success of parents
and their children.

~~~
emodendroket
I don't think it'd be hard, if you gave it a moment, to think of
counterexamples whom we'd almost all recognize.

~~~
vinceguidry
Avoiding false positives is much more important than eliminating all the false
negatives. Ugly, but true.

~~~
vonmoltke
Much parroted, rarely if ever supported by hard data.

~~~
vinceguidry
One does not need data to support it.

Attention is your scarcest resource at any company. You hire people to add to
this resource. Bad hires do not add, but subtract from this pool, by forcing
you to pay attention to them rather than on running your business.

No amount of "hard data" can make that untrue.

~~~
vonmoltke
> One does not need data to support it.

I'm an engineer; one _always_ needs data to support something.

> Attention is your scarcest resource at any company. You hire people to add
> to this resource.

Not sure what you mean by "attention", but I think it is the same thing I
would call "throughput": the rate at which the company can turn customer
requirements into engineering products (and thus money). If so, I would agree,
as pretty much all resources of the company combine to affect this.

> Bad hires do not add, but subtract from this pool, by forcing you to pay
> attention to them rather than on running your business.

Still on board with you. I would agree that someone whose net throughput
contribution is negative is a bad hire.

The problem is, this has nothing to do with the statement I was originally
reacting to: "Avoiding false positives is much more important than eliminating
all the false negatives." _That_ statement needs hard data about:

\- The expected number of false positives

\- The expected magnitude of the throughput loss from each false positive

\- The expected number of false negatives

\- The expected magnitude of the lost potential throughput due to the false
negatives

~~~
vinceguidry
>The problem is, this has nothing to do with the statement I was originally
reacting to: "Avoiding false positives is much more important than eliminating
all the false negatives."

You don't quantify the impact of false positives the same way you do the false
negatives. A false negative simply means you take more time to look for a
candidate. A false positive's impact is so huge that you can't come up with
numbers large enough for false negatives to outweigh it.

To make a broader point, not everything can be tracked and measured. Things
that can be tracked and measured often can't be compared to each other
usefully.

~~~
vonmoltke
> You don't quantify the impact of false positives the same way you do the
> false negatives. A false negative simply means you take more time to look
> for a candidate. A false positive's impact is so huge that you can't come up
> with numbers large enough for false negatives to outweigh it.

I vehemently disagree with this.

A false negative means either work isn't getting done or current people are
overworked. This is costing your business. If it isn't, why are you hiring in
the first place?

As for the cost of false positives being "huge", how? All you have stated is
that a bad hire brings down the net productivity of a team. _How much_ is very
relevant and something you can get a rough feel for. If I hire someone who has
a slight net negative impact on team productivity, and they show no signs of
improving, I fire them and move on. Is it a worse outcome than not hiring them
or anyone else for that period of time? Yes. Is it a _huge_ impact? Not at
all.

You can come up with a hypothetical worst-case, company-destroying false
positive, but that would be comparing apples to oranges based on your position
concerning false negatives. The worst-case false negative situation is also
company-destroying.

