
People suck at technical interviews - themonk
http://seldo.com/weblog/2014/08/26/you_suck_at_technical_interviews
======
Jemaclus
My primary criteria when interviewing junior candidates are:

1) Do you have basic problem solving skills?

2) Can you communicate clearly?

3) Do I want to sit next to you for the next 6 months or longer?

If you don't know Ruby, I can teach you. If you don't know Elasticsearch, I
can teach you. What I can't and don't have time to teach you is how to solve a
problem on your own without me holding your hand, and I especially don't have
time to waste trying to communicate poorly with you.

And obviously, I want to work with someone who is pleasant and interesting. I
don't wanna sit next to someone for 40 hours a week who stinks or is rude or
can't carry a conversation.

If you meet those three criteria, you've beaten 90% of candidates that walk
through the door. The last 10% is what gets you the job. (Obviously, if you DO
know Ruby or Elasticsearch, that's a huge plus... but it's not one of the bare
minimum requirements.)

~~~
fnbr
As a junior developer, I find myself confused by your comment. It seems as if
a large proportion of the posts on HN claim that, when looking to hire
someone, they focus on problem solving abilities and and culture fit.

However, in May when I was interviewing with companies (including YC backed
startups), almost every company focused on quizzing me about trivia [1]. I was
actually given a paper quiz by one of the companies on equality comparisons in
Javascript (e.g. "Does 4 == '4'?"), which I found pointless, as I can test
that within _seconds_ at a computer.

[1] The major exception was 42Debut. Even though they didn't give me an offer
(just stopped replying to my emails, but c'est la vie), I was treated with
respect throughout the whole process, and the interview was very well done,
with challenging questions that forced me to think, and didn't rely on
language trivia. I would recommend the company to any of my friends, whereas I
can't say that about almost any other company I interviewed with.

~~~
mikeleeorg
_I was treated with respect throughout the whole process, and the interview
was very well done, with challenging questions that forced me to think, and
didn 't rely on language trivia. I would recommend the company to any of my
friends, whereas I can't say that about almost any other company I interviewed
with._

This is a really important point to remember for interviewers. The interview
process isn't just to help you find good candidates, it can be a great
branding/marketing vehicle as well. I've gotten many good people from former
candidates who enjoyed our process and spoke highly of our team.

I wouldn't have just stopped replying to emails though. An interviewer should
always reply, even to politely decline. And it can help to add a reason why
you are declining a candidate too, since it can help them improve and perhaps
return for another interview in the future (I've hired a few people that I
originally passed on too).

~~~
danielweber
Of the many companies that have rejected me, Suitable Technologies (makers of
Beam robot doubles) were the best. The biggest thing was that they rejected me
_as soon as they knew they had rejected me_. Within a few hours.

Which hurt, in the short term, because I am a human being. But it was better
than many other companies that would say "we'll tell you later today" and then
wait a week, or wait for two weeks since my last interview and then reject me
in the middle of the CTF they are running and I'm participating in (what, did
my answers in the CTF suck?) and then invite me to apply for a job at the end
of the CTF after rejecting me. That's _really_ yanking my chain.

------
cjslep
For my first job out of uni, I was asked to write pseudocode on a whiteboard
to solve a simple scripting problem (call an executable repeatedly, changing
the command line args). My background is nuclear engineering, and I was
interviewing at a cloud/networking business, so I was already slightly outside
my experience comfort zone[0]. I was explaining my thought process while
writing on the whiteboard, trying my best to be transparent about how I was
thinking, when the interviewer interrupted me.

"What is that?" he asked, pointing at the whiteboard.

"...pseudocode?" I replied, hesitantly, frantically looking for some mistake
where he was pointing.

"That's not pseudocode..." he said as he started to berate me for _not writing
bash_.

After that (I did not get an offer), every interview I went to when someone
asked me to write pseudocode I'd always clarify "Is there any particular
language you want me to use?" because I never want to relive that experience
again.

[0] I had previous experience interning at Cisco.

~~~
seanflyon
Expecting pseudocode to be in a particular language is objectively wrong. It's
annoying that you lost a job opportunity because your interviewer was
incompetent, but I would recommend not asking "Is there any particular
language you want me to use?" when they ask you to write pseudocode. They
might think you don't know what the word means, which isn't that big of a
deal, but leaves a bad impression.

------
dopamean
I am an instructor at a coding bootcamp and I conduct technical interviews of
our applicants pretty regularly. I have recently been very concerned with
tackling the idea the author first addresses: not hiring (or in my case
admitting) for what people already know. Our program only has 12 weeks with
the students and so what I am most concerned with is the applicant's ability
to pick up new concepts quickly.

In the past we have tested students on the little bit of Ruby or Javascript
they had studied to prepare for the interview. I am of the belief that that
method has helped determine who knows a little bit of Ruby but _not_ who can
ramp up on complicated topics quickly during the program. My attempts to
address this have led me to doing a 15 minute lesson on something totally new
to the applicant and then having them answer questions based off of that
lesson. So far I've found it to be useful.

Technical interviews are hard. It's easy to suck at them.

------
nkcpiytdhauwuhc
Fizzbuzz isn't supposed to be a test of your knowledge of the modulo operator.
It's supposed to be a very low bar that can filter out a large percentage of
candidates who have no business being there.

If you spend 20 minutes on Fizzbuzz, you're doing it wrong. It should take 10
tops, and be limited by how quickly the candidate can write.

~~~
chrisbennet
It isn't _supposed_ to be a test of whether you know the modulo operator but
in practice it works out that way.

True story; my wife was going over her (correct) answers to a coding test with
the interviewer and he asked her "What's this?" in reference to the modulus
operator. The interviewer didn't recognize it.

I use the modulus operator almost daily so it _seems_ obvious to me but I
realize that everyone has different experiences or things they are ignorant
about. IMO, fizzbuzz is a test of whether you know the modulus operator
"trick".

~~~
lutusp
> It isn't supposed to be a test of whether you know the modulo operator but
> in practice it works out that way.

Yep. And at HN, actually knowing this or anything else assures that you will
be downvoted with a probability of 1.

> I use the modulus operator almost daily so it seems obvious to me but I
> realize that everyone has different experiences or things they are ignorant
> about.

I see the modulo operator applied to a common undergraduate math question --
if a given date lands on a Tuesday, and if you add 500 days, what day of the
week does the result land on?

------
greyskull
I'm just about to finish a B.S. in CS, so I've recently been on the other side
of the table. I like to think I'm "aware" enough to give good feedback about
my experiences.

I've interviewed with two of the large "top" companies. They were two very
different experiences.

One decided to have me do multiple interviews, with whiteboard coding. The
first interviewer was my favorite, because we got to talk about the design of
my internship project, how it could be extended, pitfalls to consider, etc.
That actually let me stretch my legs a bit and show that I can make
intelligent software decisions. The rest of the interviews were basically
worthless; the classic small algorithm problem that I could easily figure out
with a bit more time or by working with another more experienced engineer. The
code I wrote on the board showed that I could write for loops and use a
standard library; it was very difficult to modify or refactor if we saw an
issue with what I wrote. Why not at least a laptop, basic text editor, and a
projector?

The other company gave me a few hours to write up a solution, in an IDE, to a
somewhat beefy problem. We had a couple of discussions about my approach,
potential pitfalls, cases that I couldn't handle, optimality, etc. I liked
this, since it was much closer to being representative of real software work;
collaboration, discussion, and I also got to show how I would actually create
a solution.

That's a stark contrast. I'm sure there are flaws in the latter approach, but
it is if nothing else a much lesser evil that will likely bring in more
valuable engineers.

~~~
lostcolony
My favorite interview, ever, was a phone screen. Untimed. They gave me a
weekend to solve two not really difficult problems, with well defined
interfaces. The catch was to write it in a production ready style. That is, do
everything I would do for actual production code. Documentation, test suites,
write the code to be as elegant as possible, while failing as cleanly as
possible if invariants failed.

I have no idea their results with that, and I ended up taking a job before the
in person interview, but I enjoyed the screener a lot.

~~~
greyskull
I'd probably fail that from lack of experience, but it does seem to be a great
approach for someone a bit more seasoned. That shows a lot more about
someone's software engineering capabilities than short tricky problems.

~~~
walshemj
Some one more seasoned would not work for free at a weekend.

~~~
lostcolony
Some one more seasoned would not work for free during a weekday.

See, I can play that game too.

Besides, this was a contrived problem. If you want to say "I refuse to solve
any coding problems while interviewing for a software position", then fair
enough. Good luck finding one. I think this was a total of perhaps 3 hours of
my time, and was good enough to get an onsite? And it was time I -enjoyed-.
Unlike most phone screens, timed interviews, etc.

~~~
seanflyon
I think the grandparent is reacting to "Untimed. They gave me a weekend" which
made it sound as if the interview took all weekend which a more seasoned
programmer would be less likely to do.

~~~
walshemj
And in a lot of cases in Anglo Saxon employment law working for some one else
with out authorisation is a breech of employment law.

It was the they wanted production ready code that set alarm bells ringing.

------
jbogp
Having just had a 3h long technical interview for Google Deepmind, I cannot
agree more with a lot of points raised in this post.

Deepmind being a machine learning/statistics/maths/computer science fuelled
company, it made sense for the interview process to follow this simple
organisation.

I was however very disappointed by the questions asked for each part. Not a
single one of the ~100 questions asked during these 3h of my life demanded
some "problem solving" skills, only encyclopaedic knowledge (describe this
algorithm, what is a Jaccobian matrix, define what an artificial neural
network is, what is polymorphism, give examples of classifiers, what are the
conditions to apply a t-test...)

So what if someone doesn't remember every definition of the stats/ML/CS/Maths
respective bibles as long as they're clever enough to look it up and
understand quickly what's needed?

I mean, I get it these are very basic questions but as a highly qualified
interviewee who necessarily has other offers given this set of skills, this
fastidious, back to school, time wasting process does not reflect well on the
company and makes me consider my other options even more seriously.

~~~
koof
Do you think mathematicians start over from learning how to count everytime
they encounter a new problem? That's different from looking up the 10th digit
of pi: that's encyclopedic.

Knowing what a classifier is is simply more than encyclopedic knowledge that
you should probably know before joining an AI company.

~~~
jbogp
As I said before, yes these are very basic questions... and I'm not
complaining that the questions were too hard or anything, I'm saying that
finding someone who will be able to answer 100% of these "definition"
questions will not tell you anything about how competent that person is...

Making them face a simple stats/CS/maths/ML problem and see if he/she is able
to come up with the relevant concepts is far more interesting.

~~~
koof
Ah, I see. Personally I think I could have taken the "give an example of a
classifier" question into a deep dive of their competency (talk about how one
would build that classifier, etc.) but they might get so many completely
unqualified candidates that don't know those things that they need to have
those sorts of hurdles before they start really evaluating you for more
holistic problem solving traits.

But if you don't think they did that at all then I guess that's bad!

I generally am for the "look things up" argument, but so many people in tech
take that to an extreme of "I can fully understand an entire discipline by
looking at the Wikipedia page for 5 minutes".

~~~
jbogp
I completely agree with you. Also to be fair towards Deepmind I should mention
that this was the first round of interview. Hopefully the following steps will
prove more stimulating, in which case 3h for the first step was maybe slightly
too much.

------
yardie
My first technical interview they asked me to write a constructor and I
completely panicked. It still haunts me to this day that I think the guys in
the room thought I was some sort of idiot.

~~~
herbig
I have this issue as well. I get extremely nervous and forget very basic
things during phone and in person interviews.

I like reading these kinds of articles because they make me feel like less of
an idiot, but that's not going to help me get over the issue.

I think the next time I'm actively looking for a job I'll have to just buckle
down and practice interviewing and whiteboarding code, which unfortunately is
time I would rather be spending building real things instead.

------
seiji
Technical interviews are a glaring example of a problem we don't really
recognize: being stereotypically "male technical smart" is a disability.

Having smart people try to determine if another person matches their same
definition of "smartness" is fraught with peril. There are a dozen dimensions
to "smartness," and not everybody aligns properly.

It's similar to the quote "You have to be twice as smart to fix broken code as
you were when you wrote it." Extremely clever code with bugs is provably
unfixable. Extremely clever interviews are almost provably bad filter
criteria.

But, in the context of interviewing, evaluating _a person_ isn't the same as
evaluating _their immediate output_. Evaluating the future output of a person
isn't the same as evaluating "can this person replicate intro-to-CS exercises
they haven't touched in 12 years?"

Correct interviews require, _gasp_ , strong compassionate people skills in
addition to domain knowledge where you can challenge candidates. You've always
got to figure out what they actually know versus what they think they know
versus what they say they can do versus what they can actually do.

 _Then_ there's an entire other issue of "Smart People" versus "Capable
People." Most people in power end up being "smart," but not necessarily
capable any longer (by "failing upward" and now having magic protections).
Some people end up being decision makers with little actual creation
responsibilities (read: anything they could actually be judged against), so
they are free to just be amazing with little detriment for their decisions.
But, sometimes you need a number of "smart but not capable" people to balance
out half a company being head-down technically but not necessarily aware of
larger issues plaguing them. (Did I just invent managers?)

------
bcoates
These essays make me think I don't understand why people are doing technical
interviews. I give a (brief) technical question/coding question interview when
I'm asked to, and I suspect it's the kind the author doesn't like. But all I'm
looking for is:

If candidate lists lots of skills/experience, I do a self-assessment where I
get them to claim that they're really good at something, then ask them to do
something that requires bare-minimum skills in it. The idea is not that they
need these skills to do the job, but that if you claim a CS degree, eight
years work experience, and being a python expert, but you can't crack out a
basic, non-trick whiteboard coding exercise blindfolded, something smells bad.

If a candidate doesn't have much experience, I'm trying to figure out just how
green they are; low-experience candidates cover a _huge_ range of skill all
the way down to knowing literally nothing but a few buzzwords. If you don't
know a language or a library or a technology I can teach you, but if you don't
know _anything_ it's going to be years before I can get anything shippable out
of you.

I wish these weren't a major worry but the majority of people who come in the
door (after a resume and phone screen!) don't pass.

What do those of you who are doing intensive whiteboard-coding interviews
think you're getting out of it?

------
btipling
I've been interviewed by seldo before, when he was at awe.sm. It was a good
technical interview. His questions were relevant and forced me to think. I
like interviews in which I walk away learning something and I definitely did
learn something in this case.

As for interviewing, I cringe when I think about the poor questions I have
asked and the poor decisions I have made, and I try to learn from them. We
ended up not hiring qualified people because of bad interviews, although that
wasn't obvious to us at the time.

I was thinking for a future interview the candidate and I would try to learn
some new technology and try to build something together during the interview
or talk about it at least.

------
euphemize
I'm currently studying for multiple interviews at "established" tech
companies, and I wonder where the interviewers like the author are. I've
already finished a few, and between having to implement a linked list, analyze
a matrix, describe hashmaps and regurgitate sorting algorithms, the questions
have not had much to do with everyday coding.

I'd be really happy to discuss how to implement feature X in an app, or how to
design the moving parts of Y for a particular infrastructure and talk about
the tradeoffs each implies. But I guess I'll get back at implementing a queue
using 2 stacks, because that's what technical interviews seem to be for.

------
ionwake
This is the most god damn cathartic thread I have read on this site after
taking 1.5 years out to work on my own stuff and having failed 4 interviews
getting back into the workforce, with the most random god damn shit technical
tests.

~~~
Matt_Mickiewicz
I see this all the time... people who have CS Degrees, worked as programmers
for 5, 6 or 7 years and then they go and start interviewing and flunk over and
over again.

Unfortunately, almost everybody feels overconfident and ignores our advice to
prepare and practice for tech interviews: [http://www.quora.com/Whats-the-
best-way-to-prepare-for-a-sof...](http://www.quora.com/Whats-the-best-way-to-
prepare-for-a-software-engineering-job-interview)

~~~
vonmoltke
I know I have a minority opinion here, but I fell that if a technical
interview is such an exceptional event that I have to put that much
preparation into it, there is something wrong with the interview process.

------
benaston
Oh _how_ I agree with this post. And I've been the asshole on the hiring end
before - but I (hope) have learned my lesson.

I recently interviewed with a consultancy (role was UK-based) that is
desperate (quite literally) to hire experienced developers in the technical
area I have experience in. I came highly recommended by a recent senior hire
of theirs (he had been my manager).

The technical filter question in the face-to-face interview was (paraphrased
and simplified): "write an algorithm on the whiteboard to check for mismatched
braces".

I described an approximate solution aloud, but did not complete the solution
in the interview.

As it happens I solved it in my sleep the following night - but of course they
will never, ever know that.

Now everyone will have their pet theories as to why this is a good or bad
question, but these are frankly moot because I am confident the hiring
developer had _no idea_ what he was testing for with it. He may have a vague
"I know the answer, so he should too" mindset, but not much more.

What _does_ this question, presented verbally in an interview really test? A
recursive solution sprung to mind, but I can recall implementing recursive
solutions in production code only a few tens of times over a ten year career.
The question does not test for experience with the technology they are hiring
for. It certainly filters out developers who cannot come up with an algorithm
while standing at a whiteboard infront of a hostile audience. I could go on,
but the biases implicit in this question are as numerous as they are
irrelevant.

After a few years in the industry you typically get to work direct with these
so-called "top end" consultancies anyway, so you get to know what these kind
of questions _don 't_ filter for. I can say with confidence and from
experience they don't filter for ability to create high quality software.

The thing is, this 'clever' 'does he know recursion?!' question is almost
insultingly trivial when compared to the type of problems most development
teams face.

Like the fact that although "Joe" has an IQ of 150, he speaks in sentences so
impossibly convoluted he may as well be mute. Or "Alice" the sole domain
expert, who begrudges her position in some way and will only communicate after
you have negotiated the massive chip on her shoulder.

Hell, most organisations can't even provision accurate testing environments
and force their developers to run underpowered Windows laptops.

Let's walk before we can run yeah?

~~~
jdmichal
> A recursive solution springs to mind, but I can recall implementing
> recursive solutions in production code only a few tens of times over a ten
> year career.

If you are not comfortable with recursion, just remember: Recursion is really
just a free stack. To use your example:

Whenever you see an opening brace, push onto the stack. (Or, make a recursive
call, which goes onto the call stack...)

Whenever you see a closing brace, pop off the stack. (Or, return from the
recursive call, which then pops off the call stack...)

If the stack isn't empty at end of input, then there are mismatched opening
braces. If you ever try to pop from an empty stack, then there are mismatched
closing braces.

~~~
benaston
Thank you for the helpful comment. I can't help but feel you miss the point of
my post though. I can solve the problem. Verbally, (but admittedly less
eloquently and less complete) I approximately gave your answer in the
interview. Give me a quiet Monday morning, a pair of headphones and a
compiler/interpreter and I can solve it well.

It's the fact that it is being used as a filter when the person filtering
doesn't know what is being filtered in and out.

All in all, it's a spectacular waste of time and money.

~~~
jdmichal
I understood your post. I didn't comment on your other points because I didn't
feel I had anything to add. I was just trying to be helpful in regards to the
particular problem you posted, and for anyone having trouble with recursive
problems in general.

------
jakozaur
I disagree with author about coding question. In my experience if somebody is
not able to code quickly a simple task (like fizzbuz, reverse a string), it's
a terrible sign. It's a simple objective test that filter out a lot of
candidates.

Yeah they may be able to be productive in some specific environment (e.g. deep
in some framework, writing templates), but in general they likely don't have
solid programming foundation and will produce code of lower quality.

~~~
physPop
> In my experience

That is exactly the problem. Your experience is biased and by definition not
transferable to hiring in general. This is why thousands of academic articles
exist about research practices and good indicators for future performance, as
tokenadult's post nicely highlights.

------
tptacek
This is a really excellent post. It talks about a bunch of things that are
hobby-horses of mine (I help run recruiting for a large software security
firm), and I find myself agreeing with more of it than I disagree with.

I would go a little further than Laurie does. I think several of the goals he
sets up for his process are not in reality achievable in an interview process.

Starting axiom: job interviews are among the most hostile experiences
professionals endure in our industry. I think back to my own interviews, and
compare them to public speaking, professional disasters, death march projects
with insane deadlines, intractable politics, and it's pretty much the
interviews alone that increase my heart rate. For the past two years, I've
tried to make an effort to peek in on the interviews we do here at Matasano,
and what I've seen corroborates the belief. Several of our best hires were
_physically shaking_ during their interviews. And we try to be nice! In no
other common situation does a tech worker find themselves interrogated by a
panel of strangers whose implicit goal is to knock them out of contention.

Given that interviews are a psychologically challenging experience, and
thinking about things like the concept of "depletion" of ego or willpower,
it's straightforward to see some severe limitations to what can be
accomplished in an interview setting. If you're spending lots of energy trying
to keep from jumping out of your skin in an unpleasant situation, it's much
harder to solve problems that themselves require lots of energy.

Past that, a hypothesis, which is unpleasant for some tech workers to hear
(cold comfort: I'm 100% certain it applies to me as well). Software developers
are not, as a cohort, particularly effective intuitive psychologists.
Virtually none of us have any training in the subject. We tend sharply towards
introversion. We train our brains day-in and day-out by repeating tasks that
do nothing to developer our ability to read in-person social cues. For that
matter, we tend as a group to eschew forms of communication in which tone of
voice, body language, and emotional cues are even transmitted!

But several of the objectives Laurie sets out demand exactly that kind of
analysis. "Can the candidate intelligently discuss technology?" Well, that's
subjective, and worse, vague and abstract. Laurie tries to nail
"intelligently" down, but I think we can all see that there are other ways in
which someone can be "intelligent" about technology that evade those criteria.
Since we all intuitively know that, we substitute our own cognitive biases for
"intelligently". All of the sudden, we're gauging "confidence" and "comfort
level"... we've decided to be psychologists instead of engineers.

So, two changes I would urgently suggest Laurie consider for his process:

* Audit the whole process for tasks that could generate false positives from a nervous candidate. You aren't interviewing people to determine how good they are at interviewing, because interviewing doesn't generate money for your company. Try to build a process that is immune to discomfort and lack of confidence. It can be done! Another thing that we've found very effective: "prime" candidates early and repeatedly with non-adversarial conversations that aim to disarm them. We start our whole (multi-week) process with an hour-long version of this. We also try to innoculate our process by communicating in as much excruciating detail as we can what it will entail.

* Eliminate all subjective questions and standardize what you're left with. Engineers, in my experience, fucking hate this. But it's the right thing to do: ask every candidate, as much as possible, the same set of questions. We have a question structure that minimizes open-ended questions but has some structured "exercise" questions that give the candidate some freedom to move around --- the results of those questions can be put on a scoresheet and, to some extent, compared apples-apples to other candidates.

~~~
pastProlog
> job interviews are among the most hostile experiences professionals endure
> in our industry

I don't find being questioned during an interview hostile, although I do feel
I need to be prepared to be tested.

The most bothersome part for me is companies asking for me to list several
references, often specifically asking for previous managers. I am usually
given a standard application form to write their names, phone numbers, e-mails
etc. before I've even talked to anyone about the position.

An interview is just something I'm involved in. Since I have to hand over
contact details of several former managers/coworkers before I even talk to
anyone, I have to call up and ask a number of people for the favor of giving
me a reference before applying for a job.

Due to this, I almost never apply for one job in a one-off sense. I usually
work a few years, then decide I definitely am going to get a new job. I line
up my references then apply to jobs until I get one. Then I'm completely off
the market for another few years.

I could just not hand over the references until later in the interview
process, even though they're requesting them from me before I meet anyone, but
that starts things off on the wrong foot. As if I had something to hide.

Of course, there may be legal reasons for companies to ask for everything up
front. Asking for references too early is the biggest thing that puts me off
from looking around. I'm not going to ask several former managers/co-workers
for a favor unless I am seriously looking.

~~~
x0x0
I just don't fill in questions on the application form I don't want to answer.
Particularly references. They are unavailable -- since I can only impose upon
them so often -- until after all interviews and a salary discussion and a
serious decision that I want the job, etc.

------
TallGuyShort
>> Somebody who can intelligently discuss technology >> Somebody who knows
what they don't know

I believe in these principles especially. Most of my interview questions are
vague (and I tell the candidate this up-front, and explain why). For instance,
I'll ask them to explain how they would debug a very slow cluster, or to
explain everything happens between me hitting the keys 'google.com' to me
viewing the web page. This gives them a huge range of topics to cover in as
much detail as they want. If they know a lot about Linux administration, or
how hardware interfaces with the OS, or a lot about network protocols, they
have a chance to show it off. I'll drill deeper into wherever they take the
conversation, and a person scores major points with me if I drill down far
enough that they get lost and say "I don't know, but my guess would be... and
I would confirm that by...". I've found it to work exceptionally well.

~~~
otakucode
Oh god, a question like the Google one would have me not knowing what to
say... I mean, to give a reasonably complete answer would take hours and
hours! What do you do if someone really goes into detail and can't even get to
the part where the packet leaves the local network in a practical amount of
time? Is that viewed as inappropriate overkill or a poor understanding of the
question?

~~~
TallGuyShort
I wouldn't fault anyone for being able to go into so much breadth and depth
that we ran out of time. I would suggest they just dive into the part they
think if most interesting, or the part most relevant to the position. When
we're out of time, we're out of time, but the point is just to stimulate
enough technical conversation for me to evaluate the candidates knowledge,
passion, communication skills, strengths, interests, etc.

edit: For instance, I actually got this question asked to me in an interview
and adopted the practice several years later. It was scheduled for an hour,
they asked me this immediately, and I was talking for most of the hour. I got
through the whole process but didn't go into as much detail as I could've
along the way. Several times I was stopped and asked to clarify some details,
some of which I didn't know. That was that, and the interviewer just said
"very good" and left. Looking back, I think he was doing the exact same thing
I do now - giving me the freedom to show off what I knew, seeing if I would be
honest when I reached my limit, and probing a bit to see if I could
communicate about interesting problems.

------
tokenadult
I read the comments already posted in this thread before I read the fine blog
post. Almost everybody sucks at interviewing. Research shows that even though
job applicants think that an interview is one of the more fair procedures for
hiring a new worker for almost any kind of a job, it is one of the least
effective.

There are many discussions here on HN about company hiring procedures. Company
hiring procedures and their effectiveness is a heavily researched topic in
industrial and organizational psychology, but most hiring managers and most
job applicants haven't looked up much of the research. After reading the blog
post kindly submitted here, I'll make some comments on its tl;dr summary at
the end of the post.

1\. many interview techniques test skills that are at best irrelevant to real
working life

Yep, that's why you want your company hiring procedures to be based on
research and what really matters for finding good workers.

2\. you want somebody who knows enough to do the job right now

That's the ideal. That's why work-sample tests are, by replicated research, a
very good hiring procedure, one of the best possible hiring procedures.

3\. or somebody smart and motivated enough that they can learn the job quickly

Yep, and that's why tests of "general mental ability" are also a very
effective hiring procedure, although there are some legal requirements
surrounding use of those that you have to be careful about in the United
States.

4\. you want somebody who keeps getting better at what they do

For sure, as that is the only way your company can meet new challenges as they
come up in the company's business.

5\. your interview should be a collaborative conversations, not a combative
interrogation

I'm not sure that the author here has provided evidence for the "should"
statement in this summary, although I actually don't disagree as a matter of
how I do job interviews.

6\. you also want somebody who you will enjoy working with

Basically, almost all hiring managers fall victim to overemphasizing
likability and underemphasizing ability to get the job done, but, yeah, you
don't want to hire someone who makes the workplace miserable--that might cost
you losing other good workers.

7\. it's important to separate "enjoy working with" from "enjoy hanging out
with"

Absolutely. The best worker in your company may not be the same person you go
out with socially after work.

8\. don't hire assholes, no matter how good they are

The trick here is to figure out how much annoying behavior qualifies a person
as an "asshole" in a particular context, and that is not easy.

9\. if your team isn't diverse, your team is worse than it needed to be

There is an increasing body of research to back up this idea.

10\. accept that hiring takes a really long time and is really, really hard

Hiring is hard. It may or may not be time-consuming, depending on how
efficiently you do it.

The review article by Frank L. Schmidt and John E. Hunter, "The Validity and
Utility of Selection Models in Personnel Psychology: Practical and Theoretical
Implications of 85 Years of Research Findings,"[1] Psychological Bulletin,
Vol. 124, No. 2, 262-274 sums up, current to 1998, a meta-analysis of much of
the _huge_ peer-reviewed professional literature on the industrial and
organizational psychology devoted to business hiring procedures. There are
many kinds of hiring criteria, such as in-person interviews, telephone
interviews, resume reviews for job experience, checks for academic
credentials, personality tests, and so on. There is much published study
research on how job applicants perform after they are hired in a wide variety
of occupations.[2]

EXECUTIVE SUMMARY: If you are hiring for any kind of job in the United States,
with its legal rules about hiring, prefer a work-sample test as your hiring
procedure. If you are hiring in most other parts of the world, use a work-
sample test in combination with a general mental ability test.

The overall summary of the industrial psychology research in reliable
secondary sources is that two kinds of job screening procedures work
reasonably well. One is a general mental ability (GMA) test (an IQ-like test,
such as the Wonderlic personnel screening test). Another is a work-sample
test, where the applicant does an actual task or group of tasks like what the
applicant will do on the job if hired. (But the calculated validity of each of
the two best kinds of procedures, standing alone, is only 0.54 for work sample
tests and 0.51 for general mental ability tests.) Each of these kinds of tests
has about the same validity in screening applicants for jobs, with the general
mental ability test better predicting success for applicants who will be
trained into a new job. Neither is perfect (both miss some good performers on
the job, and select some bad performers on the job), but both are better than
any other single-factor hiring procedure that has been tested in rigorous
research, across a wide variety of occupations. So if you are hiring for your
company, it's a good idea to think about how to build a work-sample test into
all of your hiring processes.

Because of a Supreme Court decision in the United States (the decision does
not apply in other countries, which have different statutes about employment),
it is legally risky to give job applicants general mental ability tests such
as a straight-up IQ test (as was commonplace in my parents' generation) as a
routine part of hiring procedures. The Griggs v. Duke Power, 401 U.S. 424
(1971) case[3] interpreted a federal statute about employment discrimination
and held that a general intelligence test used in hiring that could have a
"disparate impact" on applicants of some protected classes must "bear a
demonstrable relationship to successful performance of the jobs for which it
was used." In other words, a company that wants to use a test like the
Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ
tests, in a hiring procedure had best conduct a specific validation study of
the test related to performance on the job in question. Some companies do the
validation study, and use IQ-like tests in hiring. Other companies use IQ-like
tests in hiring and hope that no one sues (which is not what I would advise
any company). Note that a brain-teaser-type test used in a hiring procedure
could be challenged as illegal if it can be shown to have disparate impact on
some job applicants. A company defending a brain-teaser test for hiring would
have to defend it by showing it is supported by a validation study
demonstrating that the test is related to successful performance on the job.
Such validation studies can be quite expensive. (Companies outside the United
States are regulated by different laws. One other big difference between the
United States and other countries is the relative ease with which workers may
be fired in the United States, allowing companies to correct hiring mistakes
by terminating the employment of the workers they hired mistakenly. The more
legal protections a worker has from being fired, the more reluctant companies
will be about hiring in the first place.)

The social background to the legal environment in the United States is
explained in various books about hiring procedures,[4] and some of the social
background appears to be changing in the most recent few decades, with the
prospect for further changes.[5]

Previous discussion on HN pointed out that the Schmidt & Hunter (1998) article
showed that multi-factor procedures work better than single-factor procedures,
a summary of that article we can find in the current professional literature,
for example "Reasons for being selective when choosing personnel selection
procedures"[6] (2010) by Cornelius J. König, Ute-Christine Klehe, Matthias
Berchtold, and Martin Kleinmann:

"Choosing personnel selection procedures could be so simple: Grab your copy of
Schmidt and Hunter (1998) and read their Table 1 (again). This should remind
you to use a general mental ability (GMA) test in combination with an
integrity test, a structured interview, a work sample test, and/or a
conscientiousness measure."

But the 2010 article notes, looking at actual practice of companies around the
world, "However, this idea does not seem to capture what is actually happening
in organizations, as practitioners worldwide often use procedures with low
predictive validity and regularly ignore procedures that are more valid (e.g.,
Di Milia, 2004; Lievens & De Paepe, 2004; Ryan, McFarland, Baron, & Page,
1999; Scholarios & Lockyer, 1999; Schuler, Hell, Trapmann, Schaar, & Boramir,
2007; Taylor, Keelty, & McDonnell, 2002). For example, the highly valid work
sample tests are hardly used in the US, and the potentially rather useless
procedure of graphology (Dean, 1992; Neter & Ben-Shakhar, 1989) is applied
somewhere between occasionally and often in France (Ryan et al., 1999). In
Germany, the use of GMA tests is reported to be low and to be decreasing
(i.e., only 30% of the companies surveyed by Schuler et al., 2007, now use
them)."

[1]

[http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...](http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%20Validity%20and%20Utility%20Psychological%20Bulletin.pdf)

[2]

[http://www.siop.org/workplace/employment%20testing/testtypes...](http://www.siop.org/workplace/employment%20testing/testtypes.aspx)

[3]

[http://scholar.google.com/scholar_case?case=8655598674229196...](http://scholar.google.com/scholar_case?case=8655598674229196978&q=Griggs+Duke+Power&hl=en&as_sdt=2,24)

[4]

[http://books.google.com/books?hl=en&lr=&id=SRv-
GZkw6TEC](http://books.google.com/books?hl=en&lr=&id=SRv-GZkw6TEC)

[5]

[http://intl-pss.sagepub.com/content/17/10/913.full](http://intl-
pss.sagepub.com/content/17/10/913.full)

[http://www.economics.harvard.edu/faculty/fryer/files/Fryer_R...](http://www.economics.harvard.edu/faculty/fryer/files/Fryer_Racial_Inequality.pdf)

[6]

[http://geb.uni-
giessen.de/geb/volltexte/2012/8532/pdf/prepri...](http://geb.uni-
giessen.de/geb/volltexte/2012/8532/pdf/preprint_j.1468_2389.2010.00485.x.pdf)

~~~
vosper
Thanks for posting this - some real food for thought here. Can you explain
exactly what you mean by a "work-sample test", in particular how that might
differ from a pair programming or take-home coding exercise?

~~~
tokenadult
_Can you explain exactly what you mean by a "work-sample test", in particular
how that might differ from a pair programming or take-home coding exercise?_

This HN search for comments by Thomas Ptacek on the topic of work-sample tests

[https://hn.algolia.com/?q=by%3Atptacek+work-
sample#!/comment...](https://hn.algolia.com/?q=by%3Atptacek+work-
sample#!/comment/forever/0/by%3Atptacek%20work-sample)

will lead to better descriptions of work-sample tests useful in a Hacker News
context than anything I have yet written on the topic. I'm grateful to him for
contextualizing that issue for this community.

------
aliston
I think there are some good points here, but I would argue that a lot of these
are essentially points for hiring more junior engineers. There are cases where
hiring for what someone already knows DOES matter.

I have come across too many codebases that were clearly written by folks who
were in the "get stuff done" mentality. Frankly, when you're learning a new
technology in addition to trying to do your job, your code/architecture just
isn't going to be that good off the bat. If the company had instead hired
someone who knew what they were doing from the get go, they could have started
with a more solid foundation and potentially avoided pain down the road.

~~~
jghn
IMO this is a red herring. Even ignoring shifts in language and such, I'd
imagine/hope that just about everyone would cringe a bit if they looked back
at code they wrote some time in the past. It's natural - we're constantly
learning and constantly improving.

I've changed languages multiple times in my career and yeah sure, one isn't
writing idiomatic code right away but with a little breadth of knowledge and
armed with modern tools such as stackoverflow it's really not a long
transition period. If you add in a strong team and a strong culture of code
reviews, the really egregiously non-idiomatic code will never make it in the
codebase anyways (not to mention, the newbie will learn along the way).

------
adricnet
One of the things done right at one place where I did a great many interviews
for technical candidates was at a late phase to break out the technical
interview and the management interview.

This allowed a group of potential peers to have a friendly discussion of
varying depths about not only the recent work and advertised skills of the
candidate but about current events and community engagement.

We became adept at evaluating candidates based on how they answered problems
we presented, whether we believed they actually had the skills and experience
on their CV, and how committed and engaged they were to the profession.

This is especially important for security related careers because community
engagement is vital, most people can't talk in detail about previous work and
may not have been able to publish public material, and work is highly
specialized. It is tough to evaluate someone on skills and knowledge they have
and you don't, but we got good at it.

And then the managers could talk to the candidate in a separate panel and ask
... well I have no idea to this day, actually :)

I say more about all of this a presentation I did in February, Breaking Into
Security: some InfoSec Career tips, presented at DC404, slides here:
[http://www.atlbbs.com/sharkin/breakin-
dc404.pdf](http://www.atlbbs.com/sharkin/breakin-dc404.pdf)

------
ap22213
Instead of technical interviews, I would _love_ to just pay a potentially
great candidate to come in and work for the day. Not only would it give them
an opportunity to demonstrate what they know and show off their other skills,
it would give our team a chance to see if they're a good fit. And, there would
be some real financial incentive for them to give us a try.

But, trying to get management and HR to change from the conventions is
extremely difficult. I have not been successful.

~~~
pmiller2
The biggest problem with a "work there for a day" interview IMO is that
there's no time to do anything useful. You can't get familiar with a codebase
of any size in a day.

~~~
VLM
Welcome to pair programming. Unlike real pair programming, don't let the noob
touch the keyboard for the entire day. Talk talk talk. Don't expect either to
really get much work done. But pretending to work all day is a heck of an
interview.

This works outside of programming. Over a decade ago when I was doing network
stuff, I did "pair router work" with a couple people. You don't give a noob
enable CLI on a production router on his first day, or during an interview,
either. Even if on paper, or in reality, he is better at this than you are.

If you hire them, you'll be spending the first couple days like this, anyway,
unless you go all sink or swim.

------
pmiller2
Followup question: Where can I find a list of companies (other than npm and
Matasano) that _don 't_ interview with the stupid "write code on the
whiteboard" method?

I've got some (1.5 years) experience, and I can write code. But put a
whiteboard marker in my hand and ask me questions you know the answers to, in
a high-stakes, high-pressure environment, and I fall apart.

Case in point: a recent technical interview I did, I did reasonably well until
the last 1 hour session, which was the "code on the whiteboard" problem. I got
code that functioned but wasn't fast enough. At home, after 1 google search
and a few minutes, I solved the problem.

I'm definitely in favor of the "do a small project" type of code interview. It
has its own pitfalls, but it's way better, IMO.

------
danielrm26
While I can see potential danger in hiring friends, I think it's wrong to say
outright that it's a bad idea.

Most bad hiring of friends comes from being bad at hiring in general. If you
hire someone BECAUSE they're your friend, then you're doing it wrong. The idea
is to hire people who are qualified and who you know to be quality over a long
period of time.

Used properly, friendship can just be considered a source for good references
in this case, and potentially some additional glue during tough times.

It's true there are risks to hiring friends, especially if it's done poorly or
for the wrong reasons, but it's also true that there can be extraordinary
benefits.

------
SnacksOnAPlane
My go-to question for technical interviews is "how would you write Scrabble?".
It's very open-ended. I expect them to ask me questions like "what do you
mean, write Scrabble?", to draw out some class diagrams, and to implement some
scoring. It shows how self-directed they are and how much they trust
themselves to develop architecture.

I hate questions about bit-twiddling and sorting algorithms. I wouldn't
remember that stuff because I never do it. I do expect candidates to be able
to analyze an algorithm for time and space requirements, because that's
important for pretty much any coding job.

~~~
arenaninja
I would fail this. I know of Scrabble, but I have not played it. On top of
that, from having had to build scoring for things previously, I would say that
it's non-trivial, which would compound the fact that I don't know what goes on
on a scrabble board

~~~
SnacksOnAPlane
If you didn't know Scrabble I'd ask you to code another game. The fact that
it's non-trivial is kind of the point; you have to ask questions (gather
requirements) to do it.

------
Ryel
I've interviewed with dozens and dozens of startups in NYC. I'm an entry level
Front-End Dev with passion for a lot of things I (in hindsight) probably
should have gone to college for. For example, my current side-project is a
completely client-side image metadata reader/writer that has forced me to
learn new things like parsing Binary, Endianness, Meta and a lot of things
about organizing JS because I've never built this large of a project
completely by myself even though it is very, very small. A weekend project for
most of you.

Throughout all of my interviews I've noticed one common trait.

Either I do exceptionally well on the personal questions during the interview,
or I do really well on the technical questions. There is no in-between.

Most of my interviews could be broken up into 2 sections. The first section is
usually where they open up and try to make things more comfortable. Asking me
where I'm from, how long I've been writing code, what kind of side-projects
I'm working on, etc...

The second part of the interview is when they try to figure out how much I
know. Usually we start off by just talking about technology and this is where
they try to figure out if I'm BS'ing my way through things. If you can
maintain a fluent conversation about technology; you're good-to-go. After that
they will generally try to slip in a few questions, usually about event
delegation in Javascript.

The problem for me comes from when we switch from personal questions to
technical ones, or vice-versa. I will pass either one with flying colors but I
will rarely, if ever, pass both. If that particular day I can't sell my
personal story very well, I do very well on the technical questions. If I do
well on the personal questions, I do terrible on the technical questions. I
believe there is a vast cognitive gap when you go from an emotional
conversation talking about significant things in your past (like side-
projects, old employers, hometown, etc..) and then jump to such rigid topics
like code where there is no emotion, it's purely analytical thinking.

What I would prefer is for a company to pay me $100 to come into the office
and work a half day. If I can keep up, I get the job.

~~~
Matt_Mickiewicz
Not one company has invited you to come onsite for a day-long interview to
code alongside the team?

You should be upfront with the employer. "I think the best way for me to
showcase my skills would be to come onsite for a day, and tackle whatever
projects you're working on currently. After 8 hours, you'll have a good
assessment as to whether or not it's a fit, and vice-versa."

~~~
Ryel
Not one company has asked me to write code on a computer.

Also important to note these are almost all fully funded companies with solid
dev teams. I've had great offers that just never worked out due to fit (or
rather lack of passion from the dev teams).

I've thrown the offer out there on several occasions but it's always refused
or delayed to the point where contact drops. One time the senior engineer told
me my offer to come in and code on my own dime was "nonsense" and If I knew
better I wouldnt be offering my work for free, because that would hurt the
"startup economy".

My faith in the NY Tech Scene is pretty much non-existent to the point I'd
rather make hourly wage in manual labor and hack on projects in my own time,
as opposed to burning myself out around people like noted above.

\--edit-- Which is exactly what I do

~~~
kethinov
Offering to pay for it probably puts people off.

You're suggesting a quite legitimate (if not common) method of assessing your
skills. You're already paying for it with your time, as are they.

If they think it's a novel idea and worth their time to give it a shot, they
might go for it. Or possibly give you homework as a decent compromise.

But the upfront suggestion of money exchanging hands does add a certain
ridiculousness to the idea on the face of it.

Just ask for a coding assignment or a trial day at the office on an actual
problem to work on without cash and it might work better.

------
darkstar999
> I used to ask people to write code in interviews. This is terrible.

Bull! Last time I interviewed someone, they looked good on paper and decent on
the phone. When it got down to solving a _simple_ problem on the whiteboard,
he totally flopped. This is a totally realistic situation; we get together at
least weekly and hammer out a solution on the whiteboard. Nerves could be an
issue, but a good candidate should be able to solve an easy problem in a
handful of minutes in an interview. (I'm talking _super_ easy, like joining
one SQL table to another after saying that you are proficient)

~~~
tptacek
He didn't say _coding_ was bad. He said _coding on a whiteboard_ is bad. And,
if you evaluated 100 strong developers by comparing their performance at a
terminal and a whiteboard, my guess is that you'd quickly see that he's
absolutely right about that --- not only would everyone outperform the
whiteboard with a terminal, but that the whiteboard deficit would be highly
irregular, defying any attempt to "handicap" for it.

~~~
buckbova
I would rather hire the strong developer that can code without autocomplete
and write without spellcheck.

~~~
tptacek
That'll come in handy when the inevitable days-long autocomplete and
spellcheck outage strikes.

~~~
Apocryphon
Or if the firm only uses Emacs.

~~~
tptacek
You use Emacs without autocomplete? That's not snark: I'm just here to tell
you that Emacs has fantastic autocomplete and you should definitely look into
it.

------
lazyant
This is one of the best write-ups over tech hiring I've seen in a while. For
me is about "get the job done, don't be a dick", problem is how to test for
that under time and other constraints.

------
blutoot
Not sure if this is off-topic but my concern is related to technical
interviews.

How do software engineers/devs/etc "stay in shape" for technical interviews?
Let's say I wanna switch jobs and the first hurdle to cross is inevitably
algorithms and data structures (for dev-centric roles) or something like
operating systems/networks/whatever. How does one stay sharp on those
_fundamental_ skills on a regular basis while working on a job, to avoid
cramming everything in one week before the interview?

------
darklajid
The other side to that story: I'm scared like shit whenever (and that's often
enough) I consider applying for a new thing. And back out, don't even try.

Most of the reasons are listed in the article, but let's be honest: I'd face
most of these things.. So the pipe dream atm is to find some time for personal
projects, foss contributions and .. use that to get to more reasonable
opportunities. Reaching ten years with this company here, these interview
stories might keep me here for another decade.

~~~
x0x0
If it helps: when I interview, I really really want the candidate to succeed
because he or she is solving a problem for me. I need more competent hands. I
don't know if this helps, but it's true.

------
russelluresti
I'm not sure I agree with the "team fit" thing. I get his point - most people
don't understand what team fit is or can't separate it from personal bias. But
here's my counter-argument.

Every company and team has different core values. "Team fit" means matching
company and team values. For example, I work on educational software for
teacher and students. My definition of "team fit" (for this particular
team/company) is that the person care about improving quality of education. If
you're applying just because you need a job but don't care about what it is
we're making, I don't care about any of the other criteria listed in this
article - I'm not hiring you. You may be smart, able to learn, and a great
communicator - but you don't care about what we're building so you don't have
the same core values as the rest of the company and team.

This is what "team fit" means - not what they look like, what they do in their
off time, or anything else; "team fit" is "do they personally value the same
things we value as a team"?

If you value transparency and collaboration and they prefer to work as a lone
gunman, they're not a team fit. If you value rapid iteration and user feedback
and they don't want to release anything until it's 100% perfect in their eyes,
they're not a team fit. So on and so forth with the examples.

~~~
watwut
Most people are fully able to work in both 100% done and release flawed but
soon scenarios. Most people simply adjust to company they work in just fine.
Almost all are also can work in both team and lone gunman settings - although
they will have preference there.

Unless you are very careful, you can select out very good candidates basically
based on how companies they previously worked in performed in those scenarios.

~~~
russelluresti
There's a difference between what you CAN do and what you VALUE doing. Yes,
people can put up with conditions they find undesirable, but they'll be
unhappy and cause others working with them to be unhappy as well.

------
jiggy2011
> _Variation across teams in big companies is enormous. Just because a company
> was successful doesn 't mean your candidate had anything to do with that._

This might seem to be an over generalisation. It's probably true enough at
large consulting body-shops but I'd certainly punt on a lower-end ex google
dev comparing favourably to the industry average.

------
ilyanep
All of these exact same points (including a very similar title) have been
floating around my head as an idea for an article I wanted to write. I was so
pleased to find that you had basically written my article for me. Very well
said on all accounts :D

------
yeukhon
As someone who recently graduated from college, by far the best interview I
ever had was just question all day and asked about projects I would want to
work on and asked to explain how I would go about designing my project on the
whiteboard. No I didn't get that job; I screwed up the last part (well I did
answer some questions incorrectly but I think they didn't care much about
that). For what it is worth, the position has to do with security research and
engineering. The coding part I had to go through was fairly quickly (but the
interviewer, IMO, didn't do a good job at it and caused me a lot of trouble).
Geesh, I wish I was given a offer, but looking back I wasn't ready to deliver
ANYTHING valuable as an [researcher]engineer yet.

When people just say a bunch of smart words like async and parallelism, ask
them to define them with concrete examples. Ask them further to break down
their "textbook knowledge." That's by far, IMHO, the best way to evaluate
whether a junior prospective engineer knew a lot than average or not. You want
a hard question? Just ask people to go as deep as they can to explain to you
what happen after a URL is entered into a HTTP client such as browser or curl.

I am terrible at coding under pressure. Actually, I am terrible at coding. I
constantly read books and github projects and I end up asking a lot of
questions: is this a good practice to do such. You see, I am all for style and
I want to be better writing code close to what people prefer to write. I went
to talks and learned tips how to write better code and I will immediately
employe those new techniques in my new projects.

I will end up with questions and I always wish someone could be around and
help. I usually go to IRC or stackoverflow for opinion but I hope when I do
get a job soon, I will be able to ask my senior colleague and hope they could
provide feedback.

I haven't done everything like some superstar already have at my age. I suck
and I am sorry, but I think I am one smart ass person capable of delivering a
project with some guidance and day-to-day meeting and mentorship.

In fact there are senior engineers I've met or worked very very briefly during
hackathon and when I asked them certain design question they will blur about
their ideas and don't really know how to contribute, or up to me to explore
what could be done. In essence, neither of us know everything. Senior knows
more because they have done more, almost certainly something repetitive. It is
like asking me to write hello world every day and I will have no problem
responding to it. Sometimes when you look at code written by senior engineers
their code smell even worse than what I write (but there are definitely the
good one I can always use as reference).

Conclusion is: when you want to hire a new engineer, especially a junior
engineer, ask yourself: are you ready to be a mentor and is there anyone ready
to believe they can be a good mentor. Actually, when you hire someone, ask
your co-workers whether they are ready to be each other's mentor. I don't want
to work at a company where everyone is hiding in a cave and think everyone is
smart and autonomous. There are times you need to be there to guide someone
through his or her obstacle. A company encourages brownbag and training will
be ideal for me.

I am serious; don't ask me to code quick sort. I am not going to do it. I know
I am just going to memorize it and after I've written one a dozen time it will
just become hello world. No, I am kidding. I need a job so I will do it. I
will, but feeling a time bomb ticking next to me. I will either fail or go on
with another few rounds writing more quick sort.

------
bcantrill
I think a more apt title is "technical interviews suck", and speaking
personally, I have entirely given up on them. (I would say that I gave up on
them after two decades, but the truth is that I gave up on them a decade ago
-- and I really tried hard in that first decade to develop the perfect
technical interview.)

My belief has become that the only way to hire the traits that I'm looking for
(high technical ability, sufficient education, predisposition to rigor, and --
most importantly -- indefatigable persistence) is by judging a candidate on
their work, not their performance in an interview. (After all, software
engineering isn't done via pop quiz -- and it's not even a timed test.)

The problem then becomes: how do you judge the works of someone you've never
worked with before? Three ways that have worked for me:

(1) Rolodex. This is an easy way to hire reliably: someone on the team has
worked with the person in the past, and vouches for them as one of the tribe.
Assuming that you trust the person vouching for them, the interview consists
of you selling them, not them selling you. This method has high fidelity
(though is still fallible), but also suffers from obvious limitations in terms
of scale and breadth.

(2) Known curricula. There are some schools where I know (or someone on the
team knows) the computer science curriculum very well, and can assess a
student simply by the courses that they have taken (or, better, TA'd), the
labs they have worked in, or the professors that they have worked for. The
fidelity of this method will depend on the school, how well one knows the
curriculum, etc. -- and it has all of the strengths and weaknesses of hiring
university grads. (It also suffers from serious scale problems and runs the
risk of creating monoculture.)

(3) Open source. If you lead open source projects that attract community
contributors, you may find it surprisingly easy to coax the best of those
contributors into working for you full-time. While I know that this isn't a
fit for every company, it has become my preferred way to hire: you are hiring
someone who can obviously do the work (because, um, they're already doing it)
and who has demonstrated interested in the problem space (especially if their
contributions have come during their free time). Importantly, you are also not
limiting yourself to a particular geographic area or school or company history
or whatever; the people I have hired via open source are people I never would
have found otherwise. And, it must be said, they have proven (without
exception, actually) to be great hires. There are obvious problems with this
as well in terms of scale and (especially) predictability, but I view open
source as the farm system for software engineering: it's a way to find and
develop talent at lowest opportunity cost.

Edit: I forgot a fourth method that has actually worked for me.

(4) Homework. When interviewing someone who I don't know and who is otherwise
a stranger, I will ask them some exceedingly basic questions to assess
technical viability, and then proceed to discuss the problems that we're
solving and why I personally find them exciting. If they are sufficiently
interested at the end of this conversation (which is really more me selling
them than them selling me), I will assign homework -- which is generally to
take some fixed amount of time (I have usually said no more than eight total
hours) to build something fun in one of the technologies that we have built or
otherwise use. (When node.js was new, this was to build something in node.js
-- but more recently, it's been to build something fun with Manta.[1]) If a
candidate comes back with completed homework, you each know something about
the other: they were sufficiently interested in you to actually play around
(and they like playing around to begin with -- which is essential for us) and
you now have some works by which to judge them.

[1] [http://www.joyent.com/blog/introducing-kartlytics-mario-
kart...](http://www.joyent.com/blog/introducing-kartlytics-mario-
kart-64-analytics)

~~~
hackinthebochs
Homework is great, but only if a job offer is guaranteed if submitting a
sufficient project. It is hubris to think someone will care enough about your
company to spend 8 hours of their time for the _small chance_ to be hired.
Presumably you are giving the same assignment to a handful of other
candidates. You are simply outsourcing much of the investment of hiring to the
candidates. This is simply unethical (one of many tests for ethics is "if
everyone did X, would the scenario be tenable"\--in this case the answer is
obviously no).

~~~
bcantrill
It's not a small chance at all -- and it only comes after a conversation where
I have done basic vetting and where I have explained why the problems that
we're solving are exciting to me. (That is, if you're not ginned up to do the
homework, I haven't done my job -- or it's not a fit.)

And as an aside, your test for ethics is entirely asinine, as it renders
anything of sub-infinite scale unethical. (Among its more obvious failings,
anyone who chooses not to have children is behaving unethically -- as is
anyone who has children, for that matter.)

~~~
defen
> And as an aside, your test for ethics is entirely asinine, as it renders
> anything of sub-infinite scale unethical.

That's not his test, that's Kant's Categorical Imperative :)

------
Todoed
Mostly do!

~~~
dang
If anyone can suggest a non-baity title, we'll happily change it beyond
s/you/people/.

~~~
kybernetikos
Actually I would have preferred the original title because then I would have
known that I didn't want to click the link.

~~~
dang
That's a pretty good argument that I don't think I've heard before.

------
lnanek2
> The famous fizzbuzz test simply asks "are you aware of the modulo operator?

No it doesn't. It could be implemented with counters you reset when needed.
Didn't really read any further, I don't think this person should really be
giving technical interviews anyway.

~~~
allensb
Yeah, but not knowing the mod operator would get you dinged.

~~~
jghn
I thought the point of the FizzBuzz was that the asker doesn't care about
'clever' (for some definition of clever) answers, but rather, "Can this person
actually write something which solves the problem, no matter how brain dead a
solution"

------
shaunrussell
From my experience this is how everyone is hiring... sounds like you've had
some bad experiences.

