
What do the best interviewers have in common? - leeny
http://blog.interviewing.io/what-do-the-best-interviewers-have-in-common-we-looked-at-thousands-of-real-interviews-to-find-out/?p=1
======
Havoc
I feel this is somewhat industry dependent, so take this with a grain of salt
(finance answer).

They cultivate a friendly banter type atmosphere. As to the actual test part:

The ones that have asked questions that require a judgemental answer.

Bad interviewers try to ask difficult or complex questions. (Stupid monkey
puzzles / google style "trick" questions")

The good ones assume you're technically competent and throw you questions
where the question is crystal clear but the scenario is ambiguous & open to
interpretation & vague. I don't need people that can do monkey puzzles I need
people that can respond dynamically in the face of uncertainty.

e.g. How would you go about detecting fraud in a scenario like XYZ.

That has a million possible answers...but depending on which route you chose
it'll become painfully obvious to the interviewer whether you know what you're
talking about. It doesn't even have to be the right one...the language and
logic alone will betray you if you don't know what you're talking about.

~~~
mratzloff
I agree completely, and it applies equally to software engineering. It's how I
interview. Now, if it becomes clear that there are gaps in their knowledge, I
will spend some time on those areas so I (or the relevant hiring manager) can
make a decision regarding if the trade-offs are something I (or they) can live
with.

For engineers, that can sometimes mean an algorithm problem--but something
they haven't likely seen before. The key here being that I don't want to see
someone regurgitate a solution, but to understand how they think. If they
can't solve a problem fully but can come up with a list of 10 extremely
pertinent questions I intentionally failed to mention, that is good
information to have.

------
alexbeloi
It looks like their metric is: "good" interviewer means the interviewee
enjoyed the interview and to want to move forward if asked.

Likely a very useful metric for figuring out if some interviewers are turning
people off about the company.

I (and probably everyone else) would really like to see if this at all
correlates with future performance metrics.

~~~
namelezz
> I (and probably everyone else) would really like to see if this at all
> correlates with future performance metrics.

I don't see how it's not possible. If your manager likes you, s/he will give
you good reviews.

~~~
indubitable
That's one oddly insightful little comment. It would be interesting to see the
relationship between perceived performance and actual performance (by some
impartial metric) in teams/management made up of people that get along well
versus teams that don't particularly get along, but also don't actively
dislike each other.

The trends in 'emotional intelligence' are largely based upon one psychology
paper that did not reasonably show what many think it did. The author broke
people into groups and found that average aptitude test scores did not work as
a predictor for tasks such as 'planning a shopping trip as a team' whereas the
average scores of another test, "seeing in the mind's eye", did.

The conclusion was therefore that a team's performance is not determined by
the aptitude of its individual members, but their 'emotional intelligence.'.
That is quite absurd since in order to make that conclusion you'd need to take
the people who are individually best at 'planning a shopping trip' and put
them in a team and compare their results to another team that scored well on
the "seeing in the mind's eye" test but individually not as well on 'planning
a shopping trip.' I think the author did not do this as she was well aware of
what the result would be, and it's not publishable.

The reason for this tangent is that 'emotional intelligence' is now being used
as a cornerstone for many things, and this article/company is yet another
group feeding off of this. Yet wouldn't be it be quite remarkable if having
groups that get along 'too well' could end up being counterindicative for
performance. Most people find it difficult to be objective around those
they're fond of. Create teams/management systems full of people fond of each
other and everybody's going to say everybody's incredible and doing incredible
things -- regardless of whether or not that's true.

~~~
pzs
Can you please share a link to the paper (if available)?

~~~
indubitable
[http://science.sciencemag.org/content/330/6004/686](http://science.sciencemag.org/content/330/6004/686)

It's paywalled. There's an informative article on Wiki about scientific
paywalls [1].

[1] - [https://en.wikipedia.org/wiki/Sci-
Hub](https://en.wikipedia.org/wiki/Sci-Hub)

------
yesenadam
Oh, it's only about job interviews. Sigh. Maybe that should have been obvious
to me from the title or URL? I'm fascinated by interviews, but job interviews,
not so much. Except this, from my favourite book by a mile about philosophy of
art:

"Two men meet; one is the applicant for a position, while the other has the
disposition of the matter in his hands. The interview may be mechanical,
consisting of set questions, the replies to which perfunctorily settle the
matter. There is no experience in which the two men meet, nothing that is not
a repetition, by way of acceptance or dismissal, of something which has
happened a score of times. The situation is disposed of as if it were an
exercise in bookkeeping. But an interplay may take place in which a new
experience develops. Where should we look for an account of such an
experience? Not to ledger-entries nor yet to a treatise on economics or
sociology or personnel-psychology, but to drama or fiction. Its nature and
import can be expressed only by art, because there is a unity of experience
that can be expressed only as an experience. The experience is of material
fraught with suspense and moving toward its own consummation through a
connected series of varied incidents. The primary emotions on the part of the
applicant may be at the beginning hope or despair, and elation or
disappointment at the close. These emotions qualify the experience as a unity.
But as the interview proceeds, secondary emotions are evolved as variations of
the primary underlying one. It is even possible for each attitude and gesture,
each sentence, almost every word, to produce more than a fluctuation in the
intensity of the basic emotion; to produce, that is, a change of shade and
tint in its quality. The employer sees by means of his own emotional reactions
the character of the one applying. He projects him imaginatively into the work
to be done and judges his fitness by the way in which the elements of the
scene assemble and either clash or fit together. The presence and behavior of
the applicant either harmonize with his own attitudes and desires or they
conflict and jar. Such factors as these, inherently aesthetic in quality, are
the forces that carry the varied elements of the interview to a decisive
issue. They enter into the settlement of every situation, whatever its
dominant nature, in which there are uncertainty and suspense." \- John Dewey,
Art as Experience

~~~
perl4ever
I'm not sure I've ever gone through an interview process where it was just me
and one interviewer. Well, there was one time, but that was an exceptionally
weird one. Frequently I meet with the department head and someone from HR. Or
the whole team. Or a bunch of department heads. Also, the last way I would
ever describe an interview is as a "unity of experience". Every person has a
different perspective (Rashomon comes to mind - maybe someone should make a
movie like that about an interview) and while the interviewers may reconcile
theirs among each other to some extent, the interviewee frequently does not
see things their way at all.

~~~
colorint
He probably means "unity of experience" in the sense that, one's experience is
a unity. Look at the whole sentence:

>Its nature and import can be expressed only by art, because there is a unity
of experience that can be expressed only as an experience.

This sounds like more of a phenomenological point, that there's a richness and
a completeness to experience that isn't captured in data or theory, and that
only something with comparable richness can begin to approach it.

~~~
yesenadam
Ah sorry, it was a bit random of me pasting that quote. His prose is fairly
repellent. To explain a little, he uses "an experience" in a particular way in
that book, not in the usual sense.

The book was a revelation to me, having read a lot of books about analytic
philosophy of music and art, each analysing their own little corner of
art/music, me feeling like I understood less after I finished each one. Then
suddenly came this book, relating everything to _life_ \- he talks about
poking a fire, firefighters, a job interview etc and how all the stuff going
on in these are the building blocks of art. A section of the book is "The Live
Creature" \- it's remarkable how many aesthetics books neglect the
human/life/planet context of art. "An experience" means something like an
aesthetic whole, like the experience of reading a story, watching a movie,
hearing a piece of music etc. The chapter's called I believe "Having an
Experience". Anyway..after that I stopped feeling the need to read books of
philosophy of art/music, as all my questions had been answered. (I'm a
musician + artist)

------
User23
If people didn't routinely fail simple "can you think and write it in code"
problems like fizz buzz I wouldn't ask them. But they do so I do.

Dijkstra observed that the majority of working programmers can't even write a
correct binary search. I tried using it as an interview question and gave up
because nobody got it right.

~~~
sidlls
The majority of working programmers rarely have to write a correct binary
search from scratch. Why should you ask it? A better question would be to ask
when it's appropriate to use the algorithm, and possibly provide a set of data
and a scenario (i.e. context) that the candidate might encounter in which
using the algorithm is an appropriate solution but not necessarily obviously
so, then see if his process of thinking about the problem leads him there.

~~~
AnimalMuppet
Rarely _have_ to? For a professional programmer, writing their own binary
search for actual use at work is nearly a firing offense (with a _very_ few
exceptions).

Use the library. Don't write your own.

(Yes, I exaggerate. Train such programmers, don't fire them... the first time.
But they're wasting your time, and probably introducing bugs in the process.
It's deeply unprofessional.)

~~~
throwaway613834
> Rarely _have_ to? For a professional programmer, writing their own binary
> search for actual use at work is nearly a firing offense (with a _very_ few
> exceptions).

You really don't sound like you do much coding. I've lost count of the number
of times I've had to write my own binary search simply because the existing
ones were painfully inadequate.

Pretty much every standard library implementation, for example, expects the
data to be in an array in memory. There's no provision for the data to be
dynamically obtained (e.g. from disk or generated on the fly) or in any other
form in memory (e.g. unsafe/native int pointers in C#). And have fun running
your standard library's binary search, whether Python or C++ or Java or
whatever, on something more abstract like the numeric interval [0.5, 1.0].
There's just no way to specify alternate termination conditions like
tolerances.

Oh, and this is completely ignoring more mundane shortcomings in a significant
fraction of implementations, like how in C# there is (or at least was, last I
checked) no way to directly obtain both the lower and upper bounds of an equal
range via binary search. At least C++ has equal_range!

I could say the same for practically any classical CS 101 algorithm like
depth-first search or Dijkstra or whatever you want, too, except those don't
even exist in most standard libraries in the first place... and I suspect
their lack of existence is not unrelated to their likely practical inadequacy.

~~~
aangjie
Either I've been in only mediocre to average roles(for any of the reasons,
including my inability) or you're a very small minority not aware of the fact.
I'd like to believe in the latter, for peace of mind, but better to believe in
former for driving personal growth. hmm.

~~~
throwaway613834
I also forgot to mention another case: where the needle you're searching for
is of a different type than the items in your haystack. If you're searching
for an integer in an array of strings that you know are integers... good luck
convincing the type system to let you even call the binary search method, let
alone figuring out which argument to the comparator is what type (if they can
even be of different types).

------
sskates
As an interviewer, I’ve found that the ability to talk through the problem at
multiple layers of abstraction to be one of the strongest indicators of
mastery of a subject. This is independent of the topic as well- whether you’re
interviewing for coding ability, sales skill, or good management practices.
Glad to see it show up in these findings as a great way to interview people.

~~~
RubenSandwich
One of the best questions I've ever been asked in an interview is: "Can you
explain what a RESTful API is and pretend I know nothing about programming but
am familiar with web surfing." For the simple reason that it asks the
interviewee to show mastery by using use clear, simple language to explain
something that can quickly turn into a bunch of rabbit trails.

~~~
itronitron
so if the candidate had never heard of 'RESTful' but the hiring manager hired
them anyway, how long do you think it would take for someone on the team to
teach the new hire what they needed to know about 'RESTful API'ness ? I
seriously wonder if the person asking you that question is hoping you can
explain REST to all of their coworkers :)

------
tyingq
>What do the best interviewers have in common?

Not sticking to some rigid formula or format. Being able to adjust to the
particular candidate in a way that works for both sides.

For example, some people aren't great at answering questions rapid fire.
Giving them a chance to mull on something, and come back to it later, might
reveal something you would have missed.

Or, if they miss a question you feel was obvious, ask them what area they feel
strong in...then ask a question in that space.

Interviewing is stressful, and so you're not necessarily getting a clear view
of their skills if you aren't flexible.

------
NTDF9
>> If you’re asking a classic algorithmic question, that’s ok, but you ought
to bring some nuance and depth to the table, and if you can teach the
interviewee something interesting in the process, even better!

This! This! This!

If you're going to ask a textbook dynamic programming problem, tell me why
that is important.

If you're checking to see they can think top-down/bottom up, that's fine. But
penalizing (aka no hire) for not knowing "the trick" of that dp problem is
futile.

In the end, you want a guy/girl who can do the job well. What signal does a
candidate solving max subarray give you? Does it tell you they'll write your
mission critical service well?

Also, LOL at "How to be human". It's sad that smart people have to be taught
this.

------
dba7dba
_better interviewing through data_ Quote from
[http://blog.interviewing.io/](http://blog.interviewing.io/)

 _Writing Good Code is like Writing a Novel_ Quote from
[https://hackernoon.com/writing-good-code-is-like-writing-
a-n...](https://hackernoon.com/writing-good-code-is-like-writing-a-
novel-33973645be01)

When I put above two ideas together, I realized following.

If you were running a publishing house and needed to a writer to publish your
next book, how do you go about picking the writer?

You usually look at the body of work from the writer. You read the books the
author wrote in the past to decide if the writer is any good.

You don't decide if the writer is good or bad from just

1) reading a page of resume and

2) asking him to write out a clever sentence on a whiteboard in the middle of
the interview.

That tiny slice of data is useless for deciding on anything. If you are all
about deciding based on data, why would you make an employment decision based
on the most tiny fraction of the actual data (resume and 30 min interview)?

Programming/IT/DevOps workers now have ample ways of showing past work. Often
it's personal projects in github or a blog with tutorials on how to secure
servers. These are far better tools to measure a candidate. Just like reading
books by an author is a far better way to predict how that author's works
would turn out.

And my personal experiences have shown me that majority of interviewers don't
read my blog or my github. I find out in interviews that some don't know it's
there. It's at the top of my resume for a reason.

And no one's asked about any question what I've written about in my blog or in
my github repo. And these are topics directly related to the position.

------
brudgers
If you enjoyed this article, you might like HN'er tptacek's
[https://sockpuppet.org/blog/2015/03/06/the-hiring-
post/](https://sockpuppet.org/blog/2015/03/06/the-hiring-post/)

~~~
ohyoutravel
Isn't this the streetfighter.io guy? Pass.

~~~
rhizome
_Isn 't this the streetfighter.io guy? Pass._

And, ladies and gentlemen, that's how we can get subtle insight into a
person's attention to detail.

------
cerealbad
we are transitioning from a knowledge based economy to a decision based one-
it's useless to ask questions which test skill competency if those questions
could be answered by a few search engine queries.

your goal as a person hiring (or firing) is to assess the candidate's ability
to consistently make efficient decisions given complex scenarios with many
rotating pieces. can this person handle their responsibilities and grow with
them, or will they be difficult.

some of the most technical and capable people also have terrible interpersonal
skills and a fickle work ethic, which means they follow their interests and
not yours. do you want to herd cats or work as a team? teams need leaders,
leaders need respect and to be followed. a working environment is a benevolent
dictatorship not a democracy. if flat structures worked they would be applied
broadly across fortune 500 companies, are you a wheel inventor?

it takes about 6 months to train almost anyone to do almost anything in the
modern digital/information world. it takes a lifetime to be agreeable, polite,
conscientious, patient, dedicated, unbiased. you'd learn a lot more about
someone watching them interact with strangers and friends for 5 minutes than
from hours in interviews (which ostensibly they have crammed for and are
hiding as much of their weaknesses as possible). i assume this is why so many
companies want private information on their employees, social media, public
profiles. hiring at the moment works as a great filter, there is no best
candidate for any job, a better strategy would be to filter out personality
traits which would clash with company direction and assume some level of skill
training will be required. pretty sure this transition is already happening.

the real stickler will come when hiring agencies realise basic fitness and
hand eye coordination tests are better predictors than technical ability, then
you enter into some genetic discrimination areas which will only accelerate
the relevant bio-technologies, a fissile cavitation.

~~~
aangjie
>leaders need respect and to be followed. This works two ways. Parts of it has
to be earned, I've worked with founders, whom I would gladly followed when it
comes to s/w engineering, but disagree violently when it comes to
product/business calls/experiments to do. The reverse has been hard to detect,
but my first instinct is that it's a lower percent.

------
ChrisSD
How good are even good interviewers at picking the best candidate for the job?
IIRC what research there is shows mixed results at best.

Or is the only point of an interview to motivate the new employee?

------
btilly
Here is a crazy interview trick that I've seen work well. Group interviews! It
isn't as crazy as it sounds.

Suppose that you're hiring for multiple related roles. You have a bunch of
people who are ready for their in-person interview come in as a group. You go
to lunch with some select employees, and explain that they are not in
competition with each other because you're hoping to make multiple hires. Then
you come back and are asked to do a basic hackathon over the afternoon. (The
available projects for which are all small but real stand-alone things that
you could use.) Through the time, you take each person aside, ask them for a
basic demonstration of ability, then return them to the group.

Hire all who seemed competent and worked well in the group.

This saves a _lot_ of time over interviewing each one, and shows you something
important about how they will be in the workplace that you otherwise wouldn't
see in the normal interview structure. And it is more fun for the candidates!

~~~
seanwilson
> And it is more fun for the candidates!

Really? It would be difficult to shake the thought that you're being directly
compared and in competition.

~~~
btilly
That can be a challenge for some. But it is more common to see people swapping
phone numbers, leads, and tips.

Looking for a job is an isolating experience. And you don't often get the
chance to connect with people who are in the same boat as you.

And for the people who really can't get past trying to compete against each
other..would you want someone like that on your dev team? That's one of the
things that this is intended to weed out.

~~~
seanwilson
Personally, I would hate preparing for a normal interviewing experience and be
thrown into something completely nonstandard like this.

Why would they be swapping tips and numbers if they're in competition with
each other?

~~~
smichel17
I've never done a group interview, so I'm speculating, but the concept sounds
nice, so long as there is also sufficient 1-on-1 time.

> I would hate preparing for a normal interviewing experience and be thrown
> into something completely nonstandard like this.

Would it a problem if you told candidates they're doing group interviews?
Might be a flaw in this particular variant, not the concept itself.

> Why would they be swapping tips and numbers if they're in competition with
> each other?

From the top level comment:

"You [...] explain that they are not in competition with each other because
you're hoping to make multiple hires."

~~~
btilly
That explanation is critical. If people can get into that mindset it all
works. If not..it is a disaster.

------
d--b
When we do interviews, at least 6 people evaluate the candidate. that would
certainly smoothen the cons and pros of individual interviewers and provide a
fairly honest picture of the company's culture...

------
aorloff
The best interviewers have 1 big thing in common : they are the actual team
the candidate will be working with.

~~~
mathgeek
For short term employment, sure. Teams change, though, so you need to be sure
you're hiring for overall ability and not just current fit. Same goes for
"culture" as well. There's an argument that "who you'll be working with" as a
metric leads to homogenous teams.

------
sushisource
I'm so, so tired of doing interviews filled with "did you memorize this
particular algorithm" whiteboard coding questions. Especially given I'm not
some fresh-out-of-college hire. The industry's research on this is pretty
unequivocal that it's a terrible way to evaluate candidates for anything other
than "isn't a complete idiot" which would also be accomplished by more
reasonable questions.

Why can't the industry finally just drop stupid whiteboard coding and move on
to more practically focused questions?

~~~
opportune
Honestly I find most white board coding questions to not veer into the "you'll
only get this if you've seen it before" territory. Not to say that it can't
happen, but in some of my interviews recently, I got the following whiteboard
problems (all asking for optimal solutions):

Find the largest palindrome substring of a string

Shortest path solver (Djikstra's)

Compute a power if the base is any double and the exponent is an integer

Take an unsorted list and put all the odd numbers on one side, all the evens
on the other

Given an array/list of numbers that first increases then decreases and a
number, determine if that number is in the array/list.

Out of those five, only one requires you to have something somewhat memorized
(path-finding), but even that is pretty basic, and I think they would have
accepted non-optimal solutions that worked anyway, so really all you needed to
know how to do was write a BFS that maintains the cost of the shortest-yet
found path to each expanded vertex. The rest I think anybody - given enough
time - could solve if they were a good programmer. Of course sometimes there
are problems where you don't figure it out in time, and definitely (a minority
of) problems that are too hard for a whiteboard interview, but you're not
going to get _every_ technical question right anyway.

It increases the stochasticity of interviewing when you can fail the technical
challenges of interviews for jobs you, on another interviewing day/with
another set of question, would otherwise qualify for, but this is logical from
a hiring perspective. It costs less to interview 20 people and accept one
qualified candidate than to interview 5 people and accept an unqualified
candidate. It's not that you _need_ to know how to solve these problems to be
a good programmer, or even that being able to solve these problems is
indicative of a good programmer, it's that a bullshitter/unqualified candidate
will fail these every time, while at least a few qualified candidates won't
fail.

~~~
Franciscouzo
Finding the optimal longest palindrome substring algorithm surely is in the
"you'll only get this if you've seen it before" territory.

Implementing a binary search algorithm is anything but trivial, you've got
overflows, off-by-one errors, passing arguments by value, etc. Sure, the
_idea_ of finding the point in the array that starts decreasing with a binary
search is simple, but the subtle ways you can write it wrong are practically
infinite.

~~~
opportune
That's true about the binary search. I didn't complete that one in time
(although I did relay the algorithm structure). Although I think white
boarding gets a lot of undeserved hate, one gripe I have is with its
occasional focus on compilable code. Obviously you want to hire a developer
that is actually capable of implementing solutions to problems, and one that
can code in a timely manner.

However, it really stings to get the algorithm right and know _what_ the
implementation concerns (overflow/OBOB) are, but simply not have enough time
to get them written in a correct/compilable manner. I've failed a few online
interviews where I knew the solution to a problem but simply didn't have the
time to write it. For example, I got a question that required building a heap,
but at some points you needed to remove items from the middle of the heap.
This would require being able to call sift-down/sift-up on the underlying data
structure. I didn't know how/if I could call sift-down/sift-up on the priority
queue implementation of the language they made me use. So I essentially ran
out of time because I couldn't look up an algorithm I knew how/why to use.

I have to disagree about the palindrome question only because I got it in the
last ~15 minutes of a 30 minute first round interview. With 10 minutes passed
and 5 remaining, I still only had the naive solution on the board with less
than five minutes to go. After a few more minutes of thinking I eventually
realized that I had only been thinking of palindromes from outside-in. With
about 1-2 minutes left I hurriedly explained to my interviewer that you should
check each character and find the largest palindrome that character is the
center (or half of a paired center) of, by expanding outwardly. He asked a few
questions, then assured me that it was ok to not write it on the board and
that he could tell I understood it. It really gave me a good impression of the
company that their hiring practices were so rational, and I now work there.

I really think anybody would have figured out the problem if they looked at it
long enough, it's just that sometimes things like this take 4 minutes and
sometimes they take 4 hours. It depends on the person but not strictly an
"intelligence" way, I just think there's a lot of variance in how long it
takes people, even pretty smart people, to solve sufficiently hard problems.

~~~
SamReidHughes
That's a good solution but it's not the optimal solution. In the worst case
that's still n^2. Nobody expects candidates to answer the linear time solution
for that question.

------
watwut
Best interviewer is aware of what the actual position really requires and will
adjust interview to find match for that exact position instead of some other
generic one.

------
known
[https://en.wikipedia.org/wiki/There%27s_more_than_one_way_to...](https://en.wikipedia.org/wiki/There%27s_more_than_one_way_to_do_it)

------
Xeoncross
The best interviewers are the ones that see your dribbble/github/youtube/etc..
matches their company needs and reach out directly.

~~~
rco8786
What sort of companies need people like me with empty YouTube/dribbble/github
accounts?

~~~
Xeoncross
You've built nothing others can see that would make them want to hire you?

~~~
rco8786
Nope. I've built tons of stuff for companies though. Stuff you've almost
certainly used before, in fact. I do _have_ a github account, but the publicly
available stuff is just a bunch of garbage from me tinkering with new
technologies or whatever.

The idea that I need to have some sort of public portfolio to land a job is
ludicrous. I have a life outside of programming.

~~~
sjellis
"I do have a github account, but the publicly available stuff is just a bunch
of garbage from me tinkering with new technologies or whatever."

Speaking as an interviewer, that's actually still pretty valuable: it tells me
a lot.

------
B1narySunset
I have really bad anxiety during interviews and it clouds my mind. I'm
considering taking a beta blocker for my next interview.

~~~
hackinthebochs
Do it. It's an open secret in musician circles that many rely on beta blockers
for high pressure performances.

------
gesman
"When can you start?" question :)

------
minimaxir
I saw this was posted yesterday. For reference, don’t delete and resubmit
things to HN if they do not get traction.

~~~
dang
It's true that submitters aren't supposed to delete and repost
([https://news.ycombinator.com/newsfaq.html](https://news.ycombinator.com/newsfaq.html))
but as the FAQ also says, a small number of reposts are ok when an article
hasn't gotten traction. And in this case we invited it, which we do a fair bit
of.

