
Why we don't hire programmers based on puzzles and tricks - mapleoin
https://37signals.com/svn/posts/3071
======
csmatt
I remember when I was prepping for my Microsoft interview. There was so much
lore online as to the types of questions they'd ask and how to prepare. I even
found 'How Would You Move Mount Fuji?' in the library and started reading
through it. It was an absolute waste of time as I, fortunately, learned weeks
before the interview. As an example, one of the questions I was asked was 'how
would you write the C function strcmp() without using any libraries?'

My gripe these days is that places I've interviewed with now that I have a few
years of experience are still concerned about big-O notation and a lot of the
more academic questions. I feel like I shouldn't have to go back to my college
notes to prep for an interview for a job that I'd really be using my current
experience for. I think it comes down to other developers not spending time
really thinking about the quality of their interview questions. They just do a
search for 'programming interview questions' and call it a day.

My most recent interview was with Amazon. I was asked the dumb big-O runtime
stuff as well as how I'd count all of the stars in the universe. It pissed me
off, but I did like the question I was asked about how I'd design an elevator
system and then asked how my solution would scale. That's not too far off from
what I'd be doing, albeit in an abstract form.

~~~
andrewvc
I generally agree, most logic questions are useless, but Big-O notation I use
every day. Programmers who don't understand the performance difference between
Array#include? and Set#include? in ruby are dangerous!

~~~
randomdata
I'm curious to know where you use the notation in daily programming. In code
comments? Discussions with other developers? Other documentation?

I use the spirt of Big-O almost daily, but do the actual calculations rarely,
and basically never use the notation. With some experience, you quickly get an
intuitive feel for execution time, which is good enough in most cases.

~~~
kamaal
Actually using something like a math notation to solve a programming problem
is asking for too much. What are those general problems which can't be solved
by standard libraries, frameworks and IDE's these days?

And even if you are facing problems that mandate you to roll out your stuff
how much of that requires these math tricks.

Given all this, if you truly want to hire a math geek to get some magic done.
I guess you are looking at a very small pool of people to hire from and it
should hardly be a problem to hire one.

~~~
to3m
Big-O notation is hardly "math geek" stuff. I'd not call this comment
ridiculous, because I might have misunderstood it, but catch me in my cups and
ask me for my true opinion and maybe I'd say that.

Big-O notation is pretty basic, and very useful. Combined with a memorized
table of powers of two (you don't need all of them - I know up to 2^20, and
this has proven sufficient - just enough so you can guess log N for a given
value) you have a good chance at being able to make quick calculations about
time and space requirements for things. Which, since we don't have infinitely
fast computers with infinite amounts of memory, often comes in handy.

~~~
kamaal
I'm not talking about Big-O. I'm talking about the general theme of these
interviews.

Besides that I still don't understand this. I don't even know the last time
anybody around me ever had a problem or a show stopper problem anything to do
with Big-O or an algorithm.

Its always how the programmer can take pressure, how he can contribute to
making the project happen, getting things done, how he can collaborate with
fellow team mates, how he responds under times of crisis, how good he is with
tools, what are his everyday practices, does he write unit tests, does he do
code reviews, does he document the API's he writes, how does he package his
code, how update he is with software world(tools, framework and techniques)
etc etc.

Lets be frank, Its always these things that are relevant today in practical
everyday software projects today. Unfortunately people hardly ask questions on
these and then complain of bad software practices among people these days.

~~~
pnathan
> I don't even know the last time anybody around me ever had a problem or a
> show stopper problem anything to do with Big-O or an algorithm.

Either your team does horrifically easy work, or no one has told you about
their abstract work. I am sorry, but even writing CRUD apps I have to think
about the informatics of the system to make it work reasonably well.

------
kamaal
This is something to think about.

The interview process is so badly gamed that these days its kind of a
ceremony. In a recent interview, I gave a candidate a practical problem. He
was allowed to use the sed man page, and then I gave him a file and
search/replace problem. Nothing much! The candidate fails! Instead he asks me
if there is going to be a algorithm/data structure interview. I told him, I
don't know but as far as I was concerned I was only going to test him on
practical everyday problems.

It was almost like I had committed a sin. He was like, every company has those
standard one's so why was I doing like a practical test. These days all
candidates do is go through most common puzzles/algorithms/data structure
stuff. Its actually quite easy to get access to those. Just spend some time on
interview forums and you can practically game any interview, in any company
any where around the world today.

My approach is straight simple. Throw a manual/documentation at the person.
Give him a problem common enough to test a practical everyday situation. Check
if he can pass the the test of reading the documentation and figuring out the
solution to the problem. Or give him/her their favorite IDE/text editor +
documentation + problem and let them solve it.

If they can do it, or even get the approach right they pass the test.

Else if all they are going to do is vomit some mathematical theory from
college text books, which isn't even remotely relevant to our everyday work-
I'm hardly interested in hiring such candidates.

~~~
mamcx
I do something like that. My first test is ask to reverse a string (not using
the reverse function!) in any language and/or pseudocode. Is _incredible_ how
badly the test goes with the people! I interview for myself or others more
than 100 people (of any background, including university, tech schols...) and
I think only 10/12 people do it correctly - barely-

To make it more fair, I always let them alone. And give internet and a
computer, and remember them they can use internet and search. Not even with
that amount of help the thing work well (I truly have find someone smart
enough to copy-paste something and return to me in 1 minute or less).

Then i do the puzzle question, but know I retire it some years ago and some
more question about real stuff...

~~~
doktrin
Reversing a string is actually an incredibly common interview question. It's a
bit like FizzBuzz (IME even more common, but I can't really speak about the
world at large).

~~~
mmorett
It might be an incredibly common interview question, but it's not an
incredibly common day-to-day task, particularly when there are methods built
in to classes that perform this function. An interesting task? Perhaps. But
not reflection of the dev's skills, per se.

You're not asking about syntax with this. It's a problem solving question and
at the end of the day, the only real metric here is not whether a dev could
code this up, but whether he can code it up quickly, under time constraints,
while being stared at, under pressure, with rent or a mortgage on the line,
and maybe kids to feed.

I am hard pressed to think of even _one_ real life dev task I had in my entire
career where I had to solve a problem under those circumstances. For most
devs, we take ownership of the problem and solve it at our desk under
reasonable time frames without the spotlight. For bigger problems, we might
think about it in the car on the way home. We might sleep on it over the
weekend.

That doesn't apply to reversing a string, of course. But you have to consider
that, depending on the dev's background and skills, he may be spending time in
jQuery, maybe Spring and Hibernate (for Java guys), maybe even some CSS. Maybe
he's a GWT guy? Maybe he was assigned to a team dedicated to (God forbid)
EJBs. In other words, he may be multifaceted and spends more time working with
an API than "just" raw language coding. The language is just a means for
working with the API.

And the one thing you want to measure is not something he does daily. Now, if
he can't use a for loop, feel free to mock him. But reversing a string? That's
clearly a puzzle.

EDIT: I can't get this part out of my mind: "Is incredible how badly the test
goes with the people! I interview for myself or others more than 100 people
(of any background, including university, tech schols...) and I think only
10/12 people do it correctly - barely-"

So you continue to use that question? Despite the results? So that when you do
hire a guy that passes, he won't ever do this on the job because you know you
damn well he should use the reverse() method? Because you won't pay him to
code it raw and waste time and money when there is a 1.5 second solution to
this? That's sadistic. :-)

~~~
suresk
I don't see how reversing a string is a "puzzle" - I'd put its difficulty on
par (or maybe slightly harder than) fizzbuzz. In most mainstream languages, it
requires 2 pieces of knowledge:

1) That a string is implemented as an array of characters.

2) How to reverse an array.

The implementation is straightforward and should be trivial for just about
anyone who's done much development.

I don't know that the question is that great, but it certainly isn't bad as a
fizzbuzz-type question to use as a filter.

~~~
mmorett
Fair enough.

Here's my solution:

def reverse() {

    
    
        def a = "Can a fella get a job?"
        StringBuilder b = new StringBuilder()
    
        for (int i = a.length(); i > 0; i--) {
            b.append(a.charAt(i-1))
        }
        print b.toString()

}

But it doesn't use arrays because quite frankly, I haven't used an array in
years. I despise the noise of the brackets. :-)

Seriously, I use ArrayLists instead of arrays. But even on this FizzBuzz lite
exercise, I got tripped up. My "i > 0" was off by one. And I started down the
path of a.substring, but that was a bad move. I quickly switched to charAt.

But about the only relevant piece of this: on an IDE, I was able to place a
breakpoint, see where I'm at, what my variables look like, and make
adjustments.

That's real life. My original cut of this would have disqualified me.
Actually, even the cut above would disqualify me since I didn't use arrays.

But I need to confess. What you see above is literally the first time I've
ever reversed a string in this manner. It's just not something I do. And it
didn't roll off the tongue so to speak. I knew I had to process the string
from the end and work backwards, but it still felt like trivia and not a
substantive inquisition into my skills or experience.

Trivial? A bit. It took me about 5 min. The off by one was killing me. It hits
you in two places: the i > 0 and the charAt(i-1).

~~~
mseebach
Your solution is fine, you've passed, flying colours. I might poke a bit
around the edges to see if I can get you to say the word "array", but mostly
out of interest.

The important bit is that this is not an assessment of your ability as a
programmer, it's a negative filter: Are you one of these people who simply
can't program, because if you can't, let's not waste any more of each others
time.

~~~
mmorett
Arrays. (Uhhg...I said it but I feel dirty.) ;-~

I've never liked arrays due to the brackets. But in a twist of irony, I now
spend all day in Groovy and I use maps, thus brackets all the time. Yet I
don't even notice.

So after carefully thinking about it, I now realize its not the brackets. I
can't explain it. There's something about arrays I just despise despite having
no similar ill will towards ArrayLists. They just feel so 70s.

~~~
qu4z-2
Personally I much prefer a[i - 1] syntax over a.charAt(i - 1). The extra
"charAt" seems noisy to me. But that's probably just my C background showing.

------
potatolicious
Damn right. Even worse than whiteboarding random algorithms are logic puzzles.

I can _sort of_ see how implementing my own hash table fits into a dev job,
but I why manhole covers are round, or light bulbs in a room has absolutely
_nothing_ to do with anything relevant in our field.

People who use logic puzzles as proxies for _anything_ in an interview have my
everlasting scorn.

~~~
tiziano88
I think that question about why manhole covers are round is more of a urban
legend than anything else. Or it might be a question asked to someone applying
for a non-technical role, to see how they reason about the issue (though it is
most definitely not the best question to achieve so).

~~~
ghurlman
I was asked that question during an interview for a technical role at
Microsoft. Granted, it was 13 years ago at this point, but they did really ask
stuff like that, and probably still do.

~~~
AdamTReineke
I haven't heard of those silly questions in the intern interview circuit. (2
at-school, 4 on campus)

~~~
potatolicious
My impression is that they are no longer in vogue - for a few years when I was
in college (2004-ish) they were _all_ over the place. Our industry is
extremely faddish, even when it comes to interviewing...

You can still see them in some places I believe. I've heard anecdotally that
Microsoft is still a fan of them.

~~~
BenOfTomorrow
When I was trained as an interviewer at Microsoft (circa 2006), they
explicitly warned people against asking questions like that (and had been
doing so for some time). There may be older employees who stubbornly persist,
though.

I should add that my personal experience contrasts with yours - I did a lot of
interviews around 2004 and got asked riddle-type questions by one, maybe two,
people. My memory is that they mostly died with the original dot-com boom.

------
pdeuchler
Interviewers ask puzzles and give you tricks because they want to see your
problem solving process. Obviously, anyone who hires simply off a litmus test
of such questions is not doing their job, but that doesn't preclude the use of
such brain teasers/problems at all.

Asking a candidate a difficult brainteaser that may be even close to
impossible can be extremely revealing. There isn't a canned response, you get
to see them work under pressure, you experience live problem solving and you
can understand how they move through their work better. All of these factors
are just as important as competency when hiring.

A great programmer doesn't "fizzle" in front of a difficult interview
question. He/she may or may not get the correct answer, but as long as they
have the requisite problem solving skills and communicate with their
interviewer well they will come out fine. I'd say that showing _how you fail_
can be quite beneficial when trying to get a job, especially if you happen to
fail gracefully :)

It seems like 37signals is just echoing platitudes to gain karma/publicity at
the expense of cheapening the discussion.

~~~
gizzlon
People can be nervous on job interview and therefore "shut down" and fail
simple tests. This does not reveal a persons real problem solving skills.

I don't consider myself especially nervous in job interviews and when taking
tests and exams. But I still had a much harder time solving simple programming
tasks than I would "back at my desk" with no-one watching. YMMW.

It would suck to want a job so bad you completely freeze up on simple
interviewing tests :(

~~~
pwny
On the other hand, if I'm hiring someone for a high pressure job I don't want
them to fail when under pressure.

I understand your point but I don't really buy it. I train lifeguards as a
hobby/side-job (I'm in uni right now) and pressure is the number 1 reason they
give us for failing their final pratical test. I can't give a kid a permit to
work as a lifeguard if the pressure of an exam makes him screw up because the
real life pressure of saving somebody's life is even greater.

The exact same applies here for all jobs where you expect the engineer to work
in stressful situations.

~~~
LockeWatts
"On the other hand, if I'm hiring someone for a high pressure job I don't want
them to fail when under pressure."

But those are in no way the same kinds of pressure. The pressure of being able
to meet a release date, design incredibly safety sensitive algorithms, etc,
are completely different from the social pressure of having someone evaluate
work you normally do by yourself, in real time.

This kind of interviewing would work well for the kind of stress a salesman
gets, not the kind of stress a software engineer gets.

~~~
pwny
I completely agree and that's why that part of the interview shouldn't be the
only one. As someone said earlier, it should be a negative filter of
incredibly unsuitable candidates.

In a perfect setting the interviewer wouldn't evaluate the candidate's ability
to solve the problem, but his/her ability to approach it, explore it, ask
questions and take steps in the appropriate direction.

------
Osiris
I'm a self-taught programmer. These stories about programming puzzles in
interviews always make me nervous to look for a job.

My first job wasn't a programming job, but I did a lot of programming because
the need arose and I was the only one capable of doing it.

I always felt too inexperienced to look for a real programming job. A friend
pushed me to interview for the position I'm in right now (full time web
developer). They didn't make me do any whiteboarding or puzzles. What I did do
is give them a code sample and we discussed how what I could have done better
and why.

I got the job and I'm doing quite well, despite my lack of a CS background.

Since I don't have a formal CS background, and most of my development
experience is web development, I have basically no experience dealing with
complex algorithms, but I've found I can solve a hard problem with enough
effort (Google, co-workers, etc).

For web developers, wouldn't more appropriate questions relate to "please
design a database for a website that needs to track the following", or "What
would your process be for designing and implementing a REST API that
would...". These would allow the developer to show you their thought process
of working through a large problem without there being a necessarily right or
wrong answer.

------
rdtsc
Exactly. I never ask those questions. I ask practical problems that we solve
every day here. Some are simplified of course.

Also I don't try to be confrontational, I take the "let's solve this
together". That often puts them at ease and makes for a better experience and
a more productive interview.

Now it is still an interview and the selection process could be brutal but
this way I get the most _relevant_ knowledge and information about the
applicant in the shortest amount of time.

~~~
elliottcarlson
The "lets solve this together" approach is always great - I've had people
simply give up and not ask any further questions - those are generally "pass"
situations. I rather see someone be genuinely interested in how to solve the
question - the more questions they ask, the more I will be willing to help
them, and the better they will actually do in the interview.

~~~
LockeWatts
The problem with this though is the meta-strategy of interviewing. Unless your
interviewer tells you up front whether or not it's okay for you to ask an
unlimited number of questions, you're basically guessing at it.

I know several companies that interview with a "hint" system, where every time
you ask a question, they just give you a hint to the problem, generally
regardless of the question asked, and then mark it down. Too many hints and
you're passed over.

~~~
elliottcarlson
Interesting - while I never said it was OK to ask unlimited questions I have
generally gauged how someone is doing and then let them know it's OK to ask
questions or need hints. Some people have accepted the offer, and some people
have simply given up - which just astonishes me.

------
cletus
Sigh. This is the issue that will just never die.

Let's just summarize the points:

\- Not everyone can produce real-world code. Most of what we do for a living
belongs to our employer. Side projects, particularly in the US, can be an
issue for IP reasons (California is somewhat of an exception here);

\- Side projects, even open source projects, can be of questionable "real
world" value;

\- I've personally encountered many "programmers" who can talk a good game but
can't code a for loop;

\- tests like the FizzBuzz test [1] are, in my experience, remarkably
effective as an early _negative filter_. This is an important point. If
someone blows you away at FizzBuzz, it doesn't mean they're an awesome
engineer. But if they can't do it, it almost certainly means they _aren't_.
The idea here is to spend the most time with candidates who might potentially
work out and wasting as little time as possible on those that probably won't;

\- the problem with these kinds of whiteboard coding problems is that the
tendency is for interviewers to think the problem needs to be "hard". It
doesn't. In making it too hard (IMHO) you risk destroying the value of your
filter;

\- pop-quizzes of obscure language features, the kind that might appear in
certification exams, are a waste of time. I have no argument with that;

\- whiteboarding code by itself is not a great filter. It should be used in
conjunction with a multi-faceted interviewing approach that involves testing
fundamentals, the ability to construct a relatively simple algorithm, the
issues of working on a team and on a production code base and systems design.

\- the problem with simply talking about "real world" code, as the author
suggests, is you're no longer finding a good engineer, you're finding someone
you like, someone who thinks like you. This falls under the umbrella of
cultural fit, which is of course important, but don't mistake that for
engineering skill.

\- I think we can all agree that "logic" puzzles like "how would you move
Mount Fuji?" or "if you shrunk to 1cm in size and dropped in a blender, what
would you do?" are stupid.

\- testing "back of the envelope" estimation however can be useful. I mean
things like "how much storage is required to store satellite images for Google
Maps?" The idea isn't to get an accurate answer. It's to see what assumptions
the candidate states and, based ont hose assumptions, to come up with a
reasonable ballpark number.

The problem here is that there are many engineers who can't comprehend the
possibility that there is someone being paid to be a programmer who can't
code. But I assure you this is the case. It's shocking but true. Simple coding
tests largely filter these people out so if you're offended by such simple
tests, just do it and move on. I assure you there's a reason why they exist.

One final prediction: There's some guy here on HN who always posts the exact
same huge comment on any hiring thread. I'm sure it'll pop up any moment now.

EDIT: let me add a point about trial periods and take home assignments.

Both of these are guaranteed recipes for mediocrity. Truly outstanding
candidates need to justify the time investment for either option and very few
companies have the kind of gravitas that would justify it.

Anything written without supervision will be of questionable provenance at
best.

As for whether or not someone will work out in your organization, bringing
them in for a day is (IMHO) of questionable value. Many engineers are
introverts. I include myself in this. It's _incredibly_ awkward as is to be in
a new company or even a new team in the same company. I question the value of
any such assessment over what you learn in 1-4 hours of interviewing.

[1]: [http://www.codinghorror.com/blog/2007/02/why-cant-
programmer...](http://www.codinghorror.com/blog/2007/02/why-cant-programmers-
program.html)

~~~
grey-area
I think the point of the article is this:

Trivial puzzlers like:write a fizzbuzz program, swap two variable contents
without using a temp variable, how do you count the stars in the universe

Just encourage people to game the system. It's impossible to get real insight
from them, because you can't differentiate between those who worked it out,
and those who have read about it. If you're in an interview by now and someone
asks one of these cookie-cutter questions like: how much storage is required
to store satellite images for Google Maps? or How would you count the starts
in the universe?

If you're good at problem solving but haven't though it through before you
might come up with a general ballpark solution, or you might freeze under
pressure of the interview of course. If you're a slick liar and have been
reading codinghorror, you'll have an answer down pat for this category of
questions before you even enter the room, one which you cribbed off the
internet, as you will for all the other common categories. If you're so
incompetent that you're neither, I guess you'd be weeded out - not a huge win
- I wouldn't expect such people even to get to interview, which is a vastly
expensive process (in time and money).

In contrast a real world problem, if the discussion is led properly by the
interviewer, offers the opportunity for real in-depth discussion about the
sort of problems you're going to be expecting this programmer to tackle every
day - what trade-offs should they make when designing data structures to
balance normalisation and speed, which libraries would they use to solve a
particular problem, which problems they have seen in the past which are
similar and how did they solve them, etc etc. _and_ it also deals with
cultural fit. There is not a clear dichotomy here between logic (puzzles), and
wishy-washy cultural fit (a chat about code) - an informal chat, if held
properly, can provide far more insight than a set of questions.

This depends entirely on the sort of job you're hiring for of course, but if
you're hiring a web developer then a good background in interview teasers is
not really on the list of desirable attributes, but a good understanding of
real code is. I guess if you don't control the entire interview process and/or
work at a large organisation, this sort of filter might become appealing
because of the standard of interviewees you receive, but that's a reaction to
a failed process.

One hiring project which I really admire is the stripe CTF as it selects for a
certain type of person that they're obviously keen to find. It's a puzzle, but
one so original and unique that it's very hard to game, and also one which
records the users' attempts in real time (if they keep logs).

~~~
zeteo
Come to think of it, swapping two variables in place is a horrible question -
because you can't really do it! A statement like

a = a + b

requires the (invisible) presence of a register where a + b (or xor or
whatever) is placed right before being written back to a. But then you might
as well use this register directly for the swap instead of dabbling around
with one-to-one functions.

~~~
steveklabnik
Also, it depends on which language you're using...

    
    
        $ irb
        irb(main):001:0> a = 1
        => 1
        irb(main):002:0> b = 2
        => 2
        irb(main):003:0> b, a = a, b
        => [1, 2]
        irb(main):004:0> puts "#{a} #{b}"
        2 1
        => nil
    

(obviously this has the same flaw you mention, the interpreter is doing stuff
for you. It follows the letter of the 'no intermediate variables' only.)

~~~
michaelfairley
You can also follow the letter in C, Java, etc.:

    
    
      a ^= b
      b ^= a
      a ^= b

~~~
jervisfm
What does that do?

~~~
steveklabnik
<http://en.wikipedia.org/wiki/XOR_swap_algorithm>

~~~
jervisfm
Thanks for the pointer.

------
jmduke
Eight minutes, 21 points, zero specifics.

I don't understand the 37signals love sometimes.

~~~
RyanZAG
I think people are up voting this one based off just the message, rather than
the content. The message is important, I guess. Plus a lot of 37signals
staffer's and friends use HN - they will likely be up voting this based off a
Facebook post or similar. ~10 of the 21 points being users affiliated with
37signals is very likely.

~~~
LockeWatts
Doesn't this violate the HN etiquette?

~~~
pseut
The discussion's been useful, though.

------
buro9
The question I always like to ask and reserve a good chunk of time for is
along the lines of "What non-technical hobby or interest do you have?" and
then "If you had all the technical resources you could dream of, how would you
now bring something new to your hobby?" and once we've gone through that the
real question is:

"Given that when you're duck hunting the key is to shoot ahead of where the
duck is, and that a lot of the suggestions have been the next iteration
applied to the hobby/pastime... what's the third or fourth iteration?".

Basically I want to see the excitement and passion about technology, their
pastime, and how they think, how they identify problems, solve them in
numerous ways, go beyond just today's solution. And I get wrapped up in it
too, I also get excited by some of the stuff that comes up.

With a great candidate... the interview ends with us both inspired and a
thousand new thoughts floating around the room.

The vast majority of candidates cannot leap beyond the first iteration, and
some struggle to even see how anything within their hobby could ever change.

~~~
papsosouid
>What non-technical hobby or interest do you have?

That seems like a poor choice. Are you trying to eliminate people who don't
have non-technical hobbies for some reason?

~~~
buro9
It's fairly irrelevant, the last part is the important part... very rarely I
do find someone who stubbornly declare that they have no other interests at
all, but then I just propose for them some pastime or other that may fascinate
them.

~~~
papsosouid
>very rarely I do find someone who stubbornly declare that they have no other
interests at all

Of course, but your wording rules out a large number of likely interests as
they are technical.

~~~
buro9
Not at all.

One interview was someone into rally driving and who was dreaming up HUD units
in his car, to allow him to see his progress in real-time compared to those
who had already raced, and to report tons of sensor based info back to the
press hut.

Then there was the rock climber who wondered whether rope tension could reveal
the likelihood of the tether point giving way.

Good hires raise interesting questions and then set to figuring out the many
ways to answer them.

The majority of hires I've made are technical, but their pastimes are not
generally "When I go home I code".

------
lifeisstillgood
About a year ago I travelled to the south west (of UK) for a job - A full day
of interviews, product ideas, I nailed the whole lot. Apart from the
obligatory coding test. I have 15 years in this game and I have coded OSS or
commercial code every day or week in Python, the language of the test, since
96.

It was a codility.com test - I had not actually been expecting it but, kerpow.
I rewrote my own abs() because I forgot such a thing existed, the problem
itself is still blanked from my brain - and every passing minute it got worse.

After an hour and a half, the interviewer took pity on me and drove me to the
station.

I actually sat on the train took the codility demo test. 100%, 3 minutes.
Signed up, took more. I could do them. Just impossible to know what went wrong
but it went badly wrong.

Ultimately it worked out well. I would have had to live weekdays down there
and my marriage probably would not have survived it. Now I work 15 minutes
walk away and give my kids breakfast each day after we sit on the sofa and
watch Mister Maker.

For me that test was a wake up call.

Firstly, I run my own business - being at the mercy of one boss is rubbish.

Secondly, these tests are rubbish for deciding who to hire - but they are OK
for programmers as a form of continuing education.

Thirdly, the exercises at the end of each chapter of SICP are much much better
form of education.

Forthly, I was once asked how a large corporate IT shop should handle a
reduction in workforce. I said march everyone into a room and those who cant
code FizzBuzz get a pink slip

Even given my experiences above - I still think that is the best option.
Sometimes a guy just dies on that day. Its unlucky. But sometimes it works out
for the best.

edit: clarity

~~~
mmorett
I never heard of codility so I went to take the demo test. I couldn't pass
that in _English_ , much less Java. Lol. That's not a programming test. That's
a math test.

That is the perfect example of what not to do.

------
louthy
I do tests for prospective candidates, but my approach is slightly different.

After the interview I email them a small unfinished program. I then give them
a set of instructions on what I would like them to do to finish it off, and
they can then do it at home on their own dev setup; with the internet and any
other resource they would normally use for development.

I give a coding style guide, and in the areas where some specific technique is
required I make that explicit.

The app has a few bugs in it, and some of the techniques asked for are
slightly leftfield.

I instruct them that if at any point they have any questions then feel free to
email me - and they won't be marked down for asking questions.

In fact I mark candidates up if they ask questions.

The task shouldn't take more than 30-45 mins for a decent coder.

I find this to be a really valuable approach to seeing how a coder works. It
shows how they work with others' code, it shows they can follow instruction,
it shows they can communicate when unsure rather than hiding away and it gives
a good guide to their style of coding.

It's a far less confrontational approach, and gives the candidate the best
chance of showing what they can do.

In the interview I tend to be much more interested in the person, because I'd
rather have a coder who is slightly less capable that I can train, than a
coder who is amazing at everything but won't fit in. I'll still discuss past
projects and dig deeper where necessary, but I like to find out how they
influenced the outcomes of projects rather than get too deep on algorithms and
the like.

So far it's worked very well :)

------
RyanZAG
I think in addition to the measures he says, if you're hiring for a web
position, it's good to make sure they understand how the stuff they're using
functions. If they don't have a good grasp of how http works / url parameters
/ that kind of thing, then they can have some nice looking code which seems to
work, but has faulty assumptions that can be security and bug nightmares down
the road.

Of course, you could always take smart people and train them - but seriously,
who does that anymore?

~~~
hippee-lee
> Of course, you could always take smart people and train them - but
> seriously, who does that anymore?

The companies who get tired of whining that there is no tech talent to hire.

~~~
RyanZAG
I've never actually seen that happen. This is what happens:

1) Large companies will import H1Bs and/or open development offices in India
and similar.

2) Startups will have the founders learn to code (this often ends well), or
they will outsource the development work (this often ends badly).

3) Companies will hang outside universities at graduation/trade shows and try
to grab promising students (this is mainly restricted to top tier
universities)

You may have good examples of it happening though?

~~~
josh2600
We hire folks who have either a CompSci background or "skillz". If you can
walk the walk a piece of paper is just signaling.

At 2600hz we have folks with Masters Degrees and folks who learned to code the
hard way, but it's important to understand that it's easier to start with a
foundation and build than it is to start from scratch. That being said, old
habits die hard and it can be difficult to unlearn them.

Ultimately, referrals are almost always the best source of candidates. The
best way to interview is with a conversation. You can learn just about
everything you want to know if you ask the right questions.

------
jwwest
The best interview I ever had, I was given a problem to solve with a system, a
whiteboard and 20 minutes.

I sketched out a quick ERD, and explained the high level architecture of the
system. No code was written -- it was assumed that I could implement said
system if I could design it.

The worst job interview grilled me on low level algorithms (I'm not a CS grad)
and wanted me to sketch out a quick sort. Of course, I can describe a quick
sort now as a result of said interview, but since it was all C# I doubt we
were going to have to implement our own sort algorithms.

Another terrible interviewer asked me about projects I did in college. I
finished grad school 3 years ago, and have 8 years professional experience...

~~~
rdtsc
Exactly. Same here.

Now I give interviews, and I do the former not the later. Make sure you do the
same if you get to interview others.

This puzzle and brainteaser shit needs to stop.

~~~
mmorett
A. Friggin. Men. I just wish they'd make it easier for us to weed _ourselves_
out. Just come straight out and ask for CS grads only. None of that "or
equivalent experience" nonsense. I also don't have a CS degree. I have the
bastard stepchild degree (CIS). :-)

Just ask for CS ONLY and we can easily avoid you and make this easier on all
of us.

------
elliottcarlson
I've been on both sides of the hiring process - and most recently was going
through the interview process again as I sought out my next opportunity. One
thing that stood out to me was the way companies would approach the white
boarding or coding challenges. While I agree with the articles issue with
coding up a specific question (especially when it is yet another fizz-buzz -
even if it has a twist to it) - the puzzle style interview can provide some
kind of insight in to the thought process behind someones work. One company in
particular really stood out by having a great white boarding exercise that not
only was entertaining, but offered a decent amount of a challenge that it
really made them a frontrunner in my final decision making process. A good
coding interview can be beneficial not only to the hiring company, but can
also give decent insight in to how the company works for the interviewee - and
that is something that should be equally important if you are trying to
attract talent.

~~~
notimetorelax
Do you think you can elaborate a little more on the exercise that you find
interesting?

~~~
elliottcarlson
On one hand I would love to - and on the other hand I don't want to give this
companies interview question away out in the open. If you want, you can email
me (email is in my profile) and I would be more than happy to share it
directly!

~~~
notimetorelax
Fair enough. It's OK I asked only out of general curiosity and to compare my
own experience with yours.

------
dmbaggett
At ITA Software, we found in-person quizzing to be a lot less useful an
indicator than looking at puzzle (or other) code someone had written
"offline".

We also found puzzles to be a useful talent magnet. If you're Google (or
perhaps 37Signals) you don't need a talent magnet. If you're a small company
nobody's heard of, one can be quite helpful.

~~~
mds
Not that it matters anymore since this was pre-Google, but I swear to god,
someone actually asked me the manhole cover question when I interviewed at ITA
in 2010. This was for a DevOps type role, so maybe they were less well
organized on that side of the house.

At first I laughed because I thought he was joking until he I saw he was
serious. He followed that up with something about stacking weights on a
seesaw.

~~~
dmbaggett
Not my fault -- that was after I was gone. :)

------
at-fates-hands
This topic keeps coming around and it always makes me wonder why people have
such a hard time hiring good programmers.

I say because the best places I've worked at already have a "system" in place
for their front-end people. You can still be a marginal developer and do well
if you follow the system. It takes a few months, but once you know how they
write their code and the standards they use, it's pretty easy to write
sustainable, reusable code.

This also makes it considerable easier to think and work outside the box. You
can take a few of the better programmers, tell them about a new framework or
approach you want to use on a project and let them loose. If it's a success,
you merge these new ideas into the existing ecosystem. Simply repeat as
necessary.

The industry really puts a lot of hard work into finding and hiring good
candidates, but it's not the programmers who are important. It's the system
you're plugging them into which matters.

------
some_googler
When I came to Google for my onsite interview, in the group of people
scheduled for that day there was this guy with an alpha-nerd attitude
including a Rubik cube that he kept playing effortlessly -- he could
completely scramble the thing and then solve it in seconds almost without
looking at it. I was never able to solve more than two faces of Rubik in my
life :) fortunately no stupid puzzles at all in the interviews (at least not
in mine), I got in. The Rubik guy, never saw again. FWIW.

------
libria
Key word here was "front-end". Algorithms aren't usually featured as often
there. Before we break out the broad brushes, lets not paint all programming
jobs as requiring the same skills.

[1] Eric Lippert summed this up quite nicely:

>> Does not having knowledge in Data Structures really affect one's career in
programming?

> Well it certainly will prevent you from getting a job on my team. But like I
> said before, programming is a huge field. There are lots of kinds of
> computer programming that don't require knowledge of data structures.

[1] <http://programmers.stackexchange.com/a/102123>

------
bramcohen
Over time I've made interview challenge problems progressively easier, as I've
found that the trickier ones don't give any more information than the easier
ones. My challenge now basically amounts to 'read this doubly nested for loop
and tell me what it does', equivalent in difficulty to fizzbuzz.

If you can't solve that, even in the pressure of an interview, then I'm sorry,
but you have no business working as a programmer, and I'm offended that you
even applied.

And yes, people flunk it all the time.

------
mcphilip
The last interview I gave for a senior dev I tried making up some custom
discussion questions to give after the basic tell me about yourself stuff. The
questions boiled down to:

1\. Junior developer code review where a save method included a call to a list
method "because it's more efficient" than making a separate ajax call. Explain
to the junior developer why that's not necessarily a good idea.

2\. UI wants a tree view of categories where each category has a name and a
parent category. The number of categories can be arbitrarily large. Discuss
problems you might run into using a database agnositic approach (e.g.
Hibernate) and what solutions you'd propose.

3\. Prototyping a new app, two potential clients in the same domain have
widely different ideas of how those domains should be represented (i.e.
obviously similar domain class names, very different properties). Discuss ways
of modeling domain classes such that each client sees the data in their
preferred format.

All in all, I think the interview went pretty well because the discussion
questions led to discussing all sorts of related topics that gave me a good
feel for the types of problems the candidate had run into and if he had
success in dealing with them.

It was great giving an interview without arcane things like explain the yield
keyword in C# and how it might be used.

------
tokenadult
As Cletus wrote in his detailed comment, "this is the issue that will just
never die." We discuss company hiring procedures here on Hacker News over and
over and over, because most of us have applied for a job at least once in our
life, and those of us who are building up a start-up have to think about whom
we would hire.

The point in the submitted blog post by 37 Signals, "The only reliable gauge
I’ve found for future programmer success is looking at real code they’ve
written, talking through bigger picture issues, and, if all that is swell,
trying them out for size," basically says that you should do a work-sample
test to hire a programmer. And that's what research says. The more a hiring
manager understands what a worker will do on the job, and the better the
manager appreciates what a new hire may grow into doing after a few years on
the job, the better the manager can devise a work-sample test to identify a
good worker. There is a research base for this. Work-sample tests aren't
perfect hiring procedures either--nothing is foolproof, and every hiring
procedure goes wrong with both "false positives" and "false negatives"--but
you can improve your odds of hiring a good worker by using a work-sample test
routinely whenever you are hiring. As a job applicant, you can select better
rather than worse bosses and co-workers by aiming most of your applications at
companies that explicitly use work-sample tests as part of the hiring process.

With the help of several Hacker News participants, I have written a FAQ about
company hiring procedures, revised over a few months of edits to fit the
typical recurrent HN discussion of this issue. See

<http://news.ycombinator.com/item?id=5227923>

for the latest posting of that, with full references to the research
literature and legal cases about hiring workers in today's world. Feel free to
contact me through my HN profile

<http://news.ycombinator.com/user?id=tokenadult>

if you have suggestions for improving that FAQ before I post it to my personal
website.

P.S. Even before I saw the prediction at the end of Cletus's comment, I
planned to make the prediction untrue.

EDIT TO RESPOND TO FIRST REPLY ABOUT PUZZLE QUESTIONS: Yes, if a company in
the United States insists on using puzzle questions as a hiring procedure, and
justifies using those puzzle questions by saying that they want to see which
applicants are "good problem-solvers," or "able to think on their feet," a
rejected job applicant just might be able to subject the company to a very
expensive lawsuit based on employment discrimination, unless the company has
prepared beforehand a validation study showing that those puzzle questions
have a bona-fide relationship to work requirements. I would not advise a
company to take that risk, especially when the legally safer alternative of
doing a straight-up work-sample test is available. The law is different in
other countries, and as a reply in this thread points out, in the EU it is
generally legal to use straight-up IQ-type tests in hiring processes, although
those are underused by private companies in Europe, according to the sources
in my FAQ post.

ANOTHER EDIT, TO LINK TO AN UNBELIEVABLE ANECDOTE ABOUT HIRING:

In August 2012, I heard a story from a hiring manager of programmers about the
hiring procedure he uses as an initial screen for applicants who have degrees
in computer science: "Write a loop that displays the numbers 1 to 100." That
sounds awfully easy, even to me, but he says that the great majority of his
applicants with accredited CS degrees fail that screening test. My earlier
telling of the full anecdote

<http://news.ycombinator.com/item?id=4603414>

and his

<http://news.ycombinator.com/item?id=4919749>

seem pretty nearly unbelievable in what they imply about how clueless many CS
grads are, and yet I think the anecdote is a true description of reality.
Cletus too mentions in this thread, with considerable agreement from reply
comments, that many people hired as programmers cannot actually program.

~~~
hackinthebochs
All this teeth-gnashing regarding programming and logic puzzles are simply to
cover up the simple fact that most people (even around here) are insecure
about their programming skill, and more-so their intelligence. The fact is
that these types of questions are indeed backed up by the very research you
cite, namely that a work-sample test and general mental ability are the only
reliable predictors of job performance. But this is exactly what programming
puzzles are (and to a lesser extent logic puzzles). A programming puzzle is
simply the intersection of a work-sample test and an IQ test. And of course,
logic skills and programming skills go hand-in-hand. Instead of writing
endless blog posts and internet comments about how useless these tests are
(trying to convince a potential future hiring manager not to use them on you),
just get better at them! You'll be better off for it.

Of course, not all programming jobs (most, in fact) need top 10% programmers
with gifted IQs. It is important that jobs are honest with themselves
regarding the level of talent they need for their next sexting iphone app. But
this too is a part of the problem. Everyone believes their shitty app or
website is going to change the world, and thus need world-class developers to
help them realize their vision. Egos will be the death of us all.

~~~
derefr
> A programming puzzle is simply the intersection of a work-sample test and an
> IQ test.

No: a programming puzzle is the intersection of a work-sample test with _one
question from_ an IQ test. IQ tests get a good handle on your intelligence by
testing you over a wide range of near-trivial questions, each known to
similarly measure intelligence, and then adding the scores up. By giving you
so many opportunities to succeed or fail, they don't fall victim to the "I
just couldn't figure this one out" problem. Whereas programming puzzles do
that all the time.

Now, if your interview consisted of 200 programming puzzles, each of which
should take roughly 20 seconds to solve, _then_ you could say it's getting a
fair handle on someone's IQ as measured through the lens of programming. But
throwing two 10-minute puzzles into the interview does nothing to help you
measure anything. The "scores" you get are black-and-white threshold filters
on a set of numbers (IQ) that are almost totally middle-grey.

~~~
hackinthebochs
> a programming puzzle is the intersection of a work-sample test with one
> question from an IQ test

In the worst case this is true. But ideally the question will have various
components, and various levels of difficulty. I'm reminded of a blog posted
here some time ago that had the candidate split a string based on a provided
dictionary of words (or something to that effect). The best candidates
produced maximally efficient code including memoization or dynamic
programming. The worst failed to implement a basic "split on the space"
functionality. Besides these two extremes there was a whole spectrum in
between. The point is to not give questions that are Pass/Fail, but a question
that has a spectrum of possible answers allowing one to find the limits of the
candidates knowledge.

~~~
derefr
That still doesn't help the candidates who just don't see the trick.

What if you know memoization and dynamic programming, but it just doesn't
occur to you that day, that to correctly split on spaces (not adding empty
tokens at the beginning and end of the string, etc.) you need to use a state
machine?

You could give the candidate that "step" of the solution and continue on to
see if they know the rest, but the candidate will likely be feeling a loss of
confidence. They'll think "hey, if I couldn't answer the 'easier' part, why
should I be able to answer the 'hard' part?" and then their mind will lock up
even harder when they search it for the next step.

This is why IQ tests are organized as a series of _orthogonal_ questions. No
question is dependent on any other question, so you can just move on quickly
from a failure and prove your worth on another question.

In fact, on a usual IQ test, you're even given five or six "presentations" of
each question, each slightly different--like math homework questions with
different numbers inserted. It happens quite often that you'll get stuck on,
say, recognizing how one specific pattern of shapes go together, but will be
able to see a different pattern instantly.

Interviews do the opposite of this; they are presented as if intelligence is a
linear quantity, and if you fail on any part of a puzzle, you couldn't
possibly be able to solve a "harder" part. In other words, interviewers are
using the "error" theory of intelligence, rather than the "bugs" theory[1].

[1] <http://news.ycombinator.com/item?id=5233497>

~~~
hackinthebochs
If your point is that these sorts of tests don't have any statistical power, I
would whole-heartedly agree. Certainly you have to be careful with the types
of conclusions one draws from the results of such tests. But they are not
useless. The probability that a candidate will work out after having solved
the problem with the best and most efficient solution is certainly much higher
than if she only gave a middle-of-the-road solution. They are useful as
positive filters.

And just to be a unnecessarily technical: if you accept the premise that the
combination of appropriate CS knowledge and intelligence increases the
likelihood that a candidate will successfully answer the question, then the
fact that they answered the question successfully increases proportionally the
probability that the candidate has sufficient CS knowledge and IQ (good ol'
Bayes' theorem). This then increases the probability that the candidate will
be successful on the job (citation in ancestor comment). That was a long
winded way of justifying these types of questions as a positive filter.

~~~
derefr
I didn't say they're not a positive filter; just that they're a _black and
white_ positive filter.

Usually, hiring isn't about "passing or failing" candidates; it's about
_ranking_ candidates relative to one-another, and hiring the "best" ones
(under whatever utility-function you think will help your business succeed.)

And putting people into "really impressed me in solving a puzzle" and "did not
really impress me in solving a puzzle" does not much help in ranking people.
It just gives you two buckets, one full of people who were both intelligent
and _lucky_ enough to get that particular trick that day; and one full of
people who might or might not be intelligent--more intelligent than the people
in the other bucket even--but who didn't get the trick that day. Personally,
I'd rather not allow "luck" into my hiring process if at all possible.[1]

And this isn't so hard! I don't see why there is such a strong digging-in
against the notion of changing away from "one or two long puzzles, that each
give a single bit of informational entropy about the candidate's capabilities"
to "hundreds of trivial, short puzzles, that together give a strong,
statistically-sound measure of the candidate's capabilities _relative to the
other candidates measured._ "

I imagine the answer is that "the long-form questions give me a chance to get
to know the candidate better, and see their [cognitive, and social] approach
to problem-solving." But the OP (this guy:
<http://news.ycombinator.com/item?id=5264735>) never mentioned _knowing
someone's approach to problem-solving_ as a dependent variable for their
workplace success. IQ tests--relatively ranking someone's general problem-
solving ability--is a dependent variable for workplace success. Work samples--
showing, technically, that you can _actually do the job as asked_ \--is a
dependent variable for workplace success. That's it. Rely on those, not your
gut.

\---

But, if I may go on a bit of a rant here...

This implies that convincing your interviewer that you'll be a nice guy who
stays on-task, doesn't complain about problems, works long hours because
they're "dedicated to the company", will talk down their successes (that is,
be "humble") and won't demand fair compensation for their abilities, and is
otherwise the "Agreeable, Conscientious"[2] kind of person that schools are
expected to produce[3] and HR departments want to consume... is _not a
dependent variable_ for workplace success. There's no correlation. People can
be great workers and produce their best work with their boss hating them and
their coworkers thinking they're an arrogant jerk. And people will not just
put up with them--but even approve of their behavior--if they get the job done
better than someone who isn't like this. (Anyone who's ever watched House will
probably grasp the concept intuitively.)

In fact, in software development at least, I suspect that this "not submissive
to workplace domination" attitude can even be _positively correlated_ with
success--it's what gives people a sense of ownership and responsibility over
project components, makes them passionate about making things _right_ instead
of _just working_ , getting coworkers to refactor their own APIs so it doesn't
hurt to consume them, etc. And I think that at least some software shops know
this--it's, as far as I can tell, the original meaning (before it got diluted
to meaninglessness) of looking for a "rockstar" developer. A clearer term
might be _developer primadonna_.

But these types of people--unless they're lying--interview _very badly_ ,
since, implicitly, an interview is an hour-long test of meaningless workplace
submission. This probably isn't a bad thing--if you have the type of
organization heavily infected with stress addiction[4], then this type of
person--who will avoid the gametalk[5] that powers reciprocal dopamine
transactions--won't fit in well anyway.

But if you're looking to build an organization full of people who care more
about winning[6] than about "being a company", my advice is: skip the
interviews. Just hand them the IQ+work sample test, send them into a room, and
get them to do it. Maybe we could even turn this process into some sort of
widely-recognized certification [a developer's journeyman certificate,
basically] so the interviewed would only have to put up with it once. That
would reduce the hiring process, simply, to "let's go out for a pint." And
that sounds, in a healthy industry, like how things should be.

\---

[1] For the same reason, I don't want to know what school you went to; luck
[of parentage] significantly determines what schools you'll naturally end up
at, with or without trying, but when people _know_ what school you went to, a
peculiar kind of nepotism/cronyism appears, with Harvard interviewers hiring
Harvard graduates even if the Ohio State University candidate is otherwise
equivalent, etc.

[2] <http://en.wikipedia.org/wiki/Big_Five_personality_traits>

[3] <http://www.overcomingbias.com/2010/06/school-attitudes.html>

[4] [http://the-programmers-stone.com/about/the-dreaded-
jungian-b...](http://the-programmers-stone.com/about/the-dreaded-jungian-
backlash/)

[5] [http://www.ribbonfarm.com/2009/11/11/the-gervais-
principle-i...](http://www.ribbonfarm.com/2009/11/11/the-gervais-principle-ii-
posturetalk-powertalk-babytalk-and-gametalk/)

[6] <http://lesswrong.com/lw/nb/something_to_protect/>

~~~
hackinthebochs
I appreciate your comment. I didn't realize you were actually _advocating_
giving a bunch of small programming questions in place of one or two involved
programming questions. I totally agree that this is a far superior way to test
programming ability. Actually developing such questions would be rather hard I
think, it seems it would be very easy to degenerate to trivia. Perhaps a set
of questions similiar to that Quixey one minute programming challenge that was
posted here a while ago (<http://challenge.quixey.com/>)?

I actually have a very ambitious idea to completely disrupt tech hiring. You
have echoed much of my rationale. I don't want to go into detail, but the
premise is basically to remove ego from the hiring process and make it far
more statistically sound. Of course, this is also the very reason would very
likely fail. Everyone loves to employ their pet "interviewing hack" to find
world-class developers (who happen to be a mirror image of themselves). Most
people will not give that up lightly.

------
zdw
The best questions to ask are the ones that pertain to actual work.

Think of a problem you had before and solved or are attempting to solve. Ask
the person being interviewed how they'd handle the problem. Do they have good
process, intuition, and problem solving skills? Are they clear about
communicating their intentions and explaining their decisions?

This isn't a trick - it's what the person is going to be doing there, in and
out, every day.

~~~
pwny
Completely agree. I recently had an interview where I was asked to solve a job
scheduling type problem. So I see the potential graph, remember topological
sort and bingo!

Right after that the interviewer told me they have a lot of code that manages
assets that have priorities between each other and that this is the kind of
problem they solve regularly.

It was a good question: it wasn't too hard as to take hours to solve, it was a
decent test of how I was able to choose an appropriate representation for the
data and it applied to a real world problem they solve.

------
blparker
I consider myself a proficient developer, however, in situations where I'm
required to solve an obscure puzzle while someone is staring at me, generally
makes me uncomfortable...no matter how easy the question. I interviewed for an
internship position at Microsoft, where I was required to write some bizarre
code on graph paper while the interviewer literally stood over my shoulder.
Horribly uncomfortable, I barely made any progress, and was subsequently
denied a position (obviously). After the interview, I went home and solved the
question in a matter of minutes, wrote the code, optimized, and fully tested
it. While I choked during the interview, I saw fellow classmates fly off to
Redmond...classmates that I _knew_ weren't better developers than I was,
simply because I had worked with many of them on projects several times.
That's when I knew this whole puzzle interview stuff was nonsense.

~~~
jdmichal
I have a similar story with Microsoft when I interviewed with them my senior
year also. I feel like I nailed everything except for the puzzles, and also
got denied. My favorite one I "failed" on was:

Given a node of a linked list, delete that node. The expected answer: Copy the
next node's contents into the given node. Then delete the next node. Just
simply never occurred to me to use a memory copy in a linked list. And this is
despite the fact that I could immediately reason about the solution he gave me
and point out a flaw in it (deleting the last node is impossible).

~~~
TheCoelacanth
In addition to not being able to delete the last node, it also invalidates any
pointers to the node after the one you're deleting.

~~~
jdmichal
Great point, and not something I had thought of. Just makes the "expected"
solution even more odd... If you're exposing nodes enough to get an operation
to delete a particular node with no context, then the given solution just
invalidated a completely different node, but not the one you asked it to
delete. No, the one you asked it to delete is still valid, just with different
data associated.

------
chadhietala
As someone that just went through a whole slew of interviews I would say I
agree with this to some extent. I'm a frontend developer and while CS
principals are important, asking questions that are not practical to everyday
development, are counter-productive and do not give you a view into how that
person tackles a real problem.

Things like "write a function that performs merge sort", are bullshit because
you would never do this in real life, largely because languages have sort
methods that are already extremely performant. Plus a lot of languages will
already be using some variant of merge sort and exposing it as a "sort"
method.

If you are given a problem like "create a tabbed news component" or you're
asked to do a small project (nothing longer than an hour), I think you can
still get a good understanding of how people solve problems. This is less
stressful, less "gotcha" mentality, and a fair approach.

~~~
mark-r
Merge sort is a bad example - it can actually be useful in real life, because
the library sort functions are limited to what will fit in memory while merge
sort is not. It's also simple enough to keep in your head without needing to
look it up. I've had to code it up on the job more than once.

------
paf31
Shameless plug: I've been working on this : <http://initialround.com> to allow
applicants to complete an interview at home and in a browser, to eliminate the
pressure of the "quizzing cage". I'd be interested to hear people's take on
this approach.

~~~
wavefunction
My current position did something similar with an existing FiddleJS they had.
When it was time for the initial screening, we spoke on the phone while I
forked the Fiddle and "whiteboarded" in real-time. I thought it was great and
I urge you to pursue your project further.

~~~
paf31
Thanks! I've done similar things with online text editors, and wanted the
ability to compare applicants side-by-side. I was also thinking this might be
useful for recruiters who don't necessarily know the code, but just want to
pick questions from a preset list to get started.

------
ctdonath
_[I] got asked how to do something in JavaScript on a white board. The
specifics are vague, but it’s crystal clear how stupid it made me feel and how
little it had to do with the actual job._

So did I. (Well, it was C then, but same idea.) The prospect of scrawling code
on an inapplicable medium while people stared at every move is, of course,
irritating.

Instead of going to the whiteboard, I pulled out my notebook computer, fired
up a code editor, and told the several guys present to continue the interview
_while_ I worked on the challenge. Time was saved by multitasking, I did the
work in a sensible normal manner, everyone was comfortable with the situation,
and they were happy with the resulting code.

I got the job. And turned it down.

------
j45
"Clever architecture increases possibility. Clever code increases complexity."
- @edw519 the other day -
<https://twitter.com/edw519/status/301393887174483968>

Testing with excessively clever tests and puzzles will attract a mindset of
creating more complex code and than needed. Architecture a problem in a way
that solve the problem elegantly but remains open and flexible is far more a
tougher challenge than an algorithmic challenge.

There's a fine line between testing a hire to make sure someone's behind the
steering wheel vs. communicating that you want things solved in interesting
ways.

------
freework
I've been interviewing pretty much non-stop since last April. I feel like I've
become somewhat of an expert when it comes to interviewing processes.

First off, as I've become a better programmer, I feel this fact is best shown
in my opinions. When I first started out, if you had asked me my opinion of
PHP vs Python, I would not have been able to say anything coherent. Now that
I've worked with both technologies, my opinion is much more coherent.

Instead of asking puzzle questions, ask candidates their opinion. "What is
your opinion of Mongodb?" I've spent enough time in the trenches with Mongo to
have quite an opinion of that technology. Same with Postgres, Django,
Javascript, Coffeescript, etc.

I think hands down, the worse type of interview questions are the ones where
you're told to solve a problem, but _you can only solve the problem in a
certain way_.

Its become a interview idiom for me. One recently was to reverse the words in
a string. In comes "This is a string", out comes "string a is This". No
problem. In python this can be done with one line:

the_string.split()[::-1]

It takes me 5 seconds to write that out. The interviewer telle me "good job",
then he asks me to do it again, but this time to do it without using any
standard library functions such as split or reverse.

At this point, I can almost with 100% certainty that this company will be a
terrible place to work. It wouldn't piss me off it it wasn't for the fact that
like 90% of all interviews I do end up like this.

I've through about this a lot, and what I've come up with for an explanation
as to why I can;t do these types of questions is because my algorithm writing
process is very subconscious. When I'm writing code and I come to a problem
that requires a lot of thinking, I usually stop, do something else and let the
problem float around for a bit in my subconscious. I got my best ideas when
I'm in the shower, on my bike riding around time, even while reading a news
article. When people are watching me (especially strange people I don't even
know) I don't do my best thinking. At this point I'd do anything to be able to
think in front of people. Because its keeping me from being able to get jobs.

~~~
mattstreet
I think you're wrong about the 100% bad place to work just because someone
asked you to reverse a string without using library functions.

I would think your computer science education would have prepared you to plan
out such an algorithm. Sadly, its pretty had for a place to confirm you have
subconscious skills by asking a question and then waiting for you to come back
the next day with the answer once it popped into your head.

~~~
freework
First off, I'm self taught, I don't have any computer science education.
Secondly, it wouldn't be too hard to give me an _actual_ problem that the
company is _actually_ having, and letting me think about it for a few days,
then come back with my solution. Why must all interview problems be made up?

------
soseng
To change it up a little bit, has anyone here been asked simple debugging
questions? I've interviewed at numerous places and I've never been asked the
question why a certain piece of code is breaking or how to fix it. The
questions could be designed to be self-contained and just complex enough to
get a feel for how the candidate thinks. My consulting job usually takes me to
places where a project is in trouble or failing. It would seem that quickly
understanding broken foreign code would be a huge asset. The closest question
I've been asked is: "Here's a Singleton implementation in Java. What can go
wrong?"

~~~
suresk
I tend to think debugging is a pretty useful indicator of how well someone
will do - the best developers I've known were really good at debugging, and
I've seen very few people be good at debugging and bad at programming - so I
usually try to incorporate a debugging question or two into my interviews.

I like to ask debugging type questions in sort of a role-play format. Usually
I'll start with a symptom that an end-user might see and then allow the
candidate to ask me for any additional information (as either the end user or
a system admin giving them log files and other things) to help them narrow
down what the problem might be.

It is a little tricky to make it possible for them to make progress without
knowing anything about the hypothetical code base, but I like it because I
think troubleshooting problems is a decent indicator of someone's skill and
knowledge: Do they have a good approach to narrowing down where the problem
is? Do they know what kind of information to ask for (for Java devs, I like to
ask questions that lead them towards asking for thread dumps or heap dumps),
which shows mastery of a language/platform? Do they ask good questions in
general?

I've also seen people use printed stack traces and ask the candidate walk
through what the stack trace is telling them and what the possible root causes
might be - this is more useful when you want to test someone's knowledge of a
specific framework.

Handing someone an application in a broken state and watching them work
through the process of debugging it would seem like it would be interesting,
but I've never had the time to prepare this for an interview.

------
ambiate
Basically, most of the questions I see asked in interviews are contained in
Levitin's algorithms book. The greatest potential for someone to excel at
those trick interviews is to be fresh out of college.

I walked out of my first interviews, because 'I do not like programming
games.' The first was a 4d matrix puzzle with more mathematical/theory roots
than programming/logic.

My third interview started with the above quote and I was hired on the spot --
I believe they took it as a rebellion of every fresh out of college kid
wanting to program video games.

------
agentultra
I have a long-winded blog post about this very subject awaiting some polish.

I've done my fair share of interviews. I still don't like them. I've narrowed
it down to basically two things:

1\. There is a lot of highly subjective, mystical advice about what "works,"
in a technical interview and what doesn't. Big problems, small problems,
trivia, puzzles, white-boards, no white-boards... hardly any of it is
thoroughly studied and built on solid theory. It's all conjecture and personal
bias.

2\. How the candidate is feeling and where their head is at on the day of the
interview _matters_. I can go on about cache alignments, bus errors,
segmentation faults, C++'s lack of order in evaluating function arguments, why
template class declarations must exist in the same compilation unit as the
definition, why CLOS is amazing and the computation model of recursion is
brilliant (or at least why I think so). Yet on a bad day I can (and have)
botched perfectly trivial, benign problems.

However at the end of the day you need some sort of process. I just think that
the current practices are not good enough. It seems to me that companies are
probably turning away perfectly fine candidates without realizing it. There's
pros and cons to every approach I suppose. But I still think trivial problems
are dumb.

~~~
suresk
I agree that the hiring situation in our industry is less than ideal. The
irony is that, for all of the data we use and tools we make for others, our
hiring practices often rely on few tools, limited data, and lots of gut
feelings about what is important.

Take the process of advertising a position and moving candidates through the
hiring process - the tools there are almost universally crappy and haven't
made a lot of progress in the last decade.

Also, as far as I can tell, very few companies attempt to record any kind of
data about how candidates are hired and how they subsequently performed on the
job (even tracking job performance is a tough problem) - and it certainly
doesn't exist industry-wide.

There is a ton of money in this area - why hasn't there been much progress? Is
it because it is an unsexy problem, or is it because it is genuinely
impossible?

------
DigitalSea
I agree wholeheartedly here. I know many developers (myself included) who
would fail a FizzBuzz test or any instance where they're asked to write code
on a whiteboard and yet can produce some really clean, efficient and above all
well-written code. How about instead of asking people to write on a whiteboard
or do Computer Science 101 tests, how about you give them a computer, a
problem and ask them to solve it?

Don't forget it's not always how smart they are, it's their work ethic and
whether or not they'll be a good fit for your team. Personalities that meld
well with already hired employees are an essential must, no point hiring an
introvert for a position when the team are extremely social people who have
Friday afternoon beers.

Here is a tip: hiring a developer? Ask them to write a blog application, a
task management system or a real world problem they will encounter if hired by
your company not some ridiculous test that could stop you hiring a great
developer. I consider myself to be pretty good, but I don't have a CS degree,
I suck at mathematics and can only do simple math, yet I am able to produce
exceptionally great code especially in instances where deadlines are tight and
sometimes unreasonable.

~~~
suresk
I'm sympathetic to this - I don't have a CS degree and most of my last jobs
have been through references from people I've worked with before, so I haven't
had to answer any tough programming questions in an interview in quite a
while. I'm sure if I tried to interview at Google, Amazon, Microsoft, etc, it
would be a little tough for me.

At the same time - any reasonably experienced developer should be able to
whiteboard something as simple as fizzbuzz without a problem. The more complex
the problems, the more you worry about the whiteboard effect, but you should
be able to at least walk through your thought process and come up with some
reasonable attempt.

Having a candidate build something outside of the interview process probably
isn't bad (as long as you follow up by going through the code with them), but
I think something like a blog application is both too big (unless you really
constrain the feature set) and too simple to work in some cases. Something
that requires a bit more thought and a bit less time would be better, IMHO.

------
feliperibeiro
This is a complicated issue, but in my opinion the google/facebook style of
interview is the one that gives you less false positives, even though it gives
tons of false negatives. In other words, very few people who can do it are bad
coders (false positives), but many good coders can't do it (false negatives).

So, as companies like Google and Facebook have so many applications, they're
much more concerned about false positives than false negatives.

------
russelluresti
Just my 2 cents on this issue - writing code on a whiteboard isn't always
about the code you write.

It's not about the actual code you produce in the interview, it's about
talking through your thought process, your ability to reason while under
pressure, and your attitude of whether or not "this is beneath me - wah!".

I've been in several interviews where I had to write code. I made some
mistakes, some things I plain out didn't know (for my first job, I had no clue
how to write any JavaScript, and I explained that it was just something I
didn't know how to do, but that I'd be willing to learn it as required). I
still got hired from all of them. So, if you think coding tests during
interviews are about producing flawless code, they're not. They're to see how
you think - and that is profoundly important as a developer.

I would also argue that a developer's existing skills are the least important
factor when deciding whether or not to hire someone. Every developer works as
part of a team and has to collaborate and communicate with others in that
team. If you want to hire a developer, you need to see how well they
communicate complex technical problems to non-technical people.

------
rayiner
I used to give interview candidates a little background lecture on our problem
domain then ask them for input on whatever I was working on that week. E.g. If
you were trying to parallelize this code, how would you structure it? I never
asked fizzbuzz style coding questions, because I think they're stupid and I
probably couldn't code a linked list on a white board (I rarely code on
whiteboards you see).

------
mattbillenstein
Most eye-opening thing about programmer interviewing I've read:
[https://sites.google.com/site/steveyegge2/five-essential-
pho...](https://sites.google.com/site/steveyegge2/five-essential-phone-screen-
questions)

I probe for weakness in 5-6 different areas of computer science in the phone
screen - keeps me from bringing in weak candidates.

Then in the on-site, I give them my Macbook and ask them to solve a few simple
problems with some flat files I conjured in a language of their choice that's
installed on that machine (C/C++/Java/Python/Ruby/PHP/Perl/shell/etc). You get
an idea of how they solve problems, think about solutions, fix bugs, avoid
common errors, and what they look up on the web (I let them use a browser to
lookup apis and so forth - it's a good way to weed out "cut and paste"
programmers). The people who do well finish quickly and we chat about the
company with the remaining time - the people who do poorly use almost all the
time on the first problem, make a bunch of mistakes, and usually don't
understand the proper data structure for the job.

------
SGemployee
Employer told me "We do hardcore stuff!" come on really hardcore?! I would
really love to post their apps here but I'm not that kind of person.

~~~
Dirlewanger
Brogrammers, man. They are not a myth.

------
GhotiFish
If I can have a word on FizzBuzz?

There are a few programmers that have a proclivity towards "clean, perfect"
solutions, but FizzBuzz is ugly.

Making FizzBuzz work at all is just stupidly simple; but the moment you ask
yourself about the little details, you notice that you either have to make an
if tree, a bunch of if-elseif's, or some other hinky thing that ruins the
"purity" of this simple little program.

Alot of people take the perspective that if you go

    
    
        if(i % 15)     print "fizzbuzz";
        else if(i % 5) print "fizz"; //or was it buzz?
        else if(i % 3) print "buzz"; //or was it fizz?
        else           print i;
    

then you've found an optimal solution and you can wash your hands of it, until
you think to yourself: "but an if tree would use less comparisons! Should I be
readable? Or take the efficient route? But if I'm thinking about efficiency.
wouldn't a lookup be faster and simpler?"

I suspect when presenting a problem like this, the interviewer filters out two
kinds of people: the people who "can't code", and the people who are
obsessive.

~~~
run4yourlives
The entire point of FizzBuzz is to filter out the people that say they can
code but clearly cannot. This is a surprisingly high number of applicants,
unfortunately.

~~~
GhotiFish
I agree, I think FizzBuzz has this property, but I think it also has the
property of filtering obsessive people as well.

You might not want obsessive people either, in which case fizzbuzz is clearly
a good choice.

~~~
kragen
I have sometimes in the past been obsessive to the degree you're describing,
and the consequence was that, practically speaking, I couldn't do the thing I
was obsessing about.

------
fiblye
I was invited to a company for a game development position. One part of the
interview consisted of a fairly challenging mathematical/algorithmic problem.
It wasn't impossible, but it's certainly not something I can do in ten minutes
at a white board with some people taking notes about my every movement.

The other part was being shown intentionally messy and broken code. Not code
that'd fail to compile (although it sure looked like it would), but something
thrown together in a completely insane manner that I couldn't possibly make
sense of in 15 minutes. Much of it boiled down to, "What would happen if you
call this function and use this variable before you actually define them and
then do x, y and z?", and the way this language did it was quite different
from other languages. I mean, yes, I did learn a lot about the language from
that interview, but I'd really hope that's not the kind of code they'd be
throwing at me or expecting me to write at their company.

~~~
jes5199
Plenty of the gigs I've had have been mostly trying to unscramble really
confusing code that mostly-works. If my codebase has a bunch of strange crap
in it that I need fixed, I certainly will bring examples of it to the
interview.

------
Tekker
When I interview others, I use a couple of basic whiteboard show-me pseudocode
questions (how would you reverse a string, how does a linked list work, give
me an example of a callback). It seems to me that if they know their stuff,
they can answer those. If someone can't get through any of them, or at least
have a valient stab at it, chances are they don't have the background I want.

By the same token, I've been interviewed myself where I've been given weird
shit questions like others have mentioned; not the blender one, but shit along
that line. That's nothing I personally do without some serious contemplation,
not to be achieved in 5 minutes in an interview. Anyone who does so likely
knew the answer ahead of time or is extremely puzzle-oriented. I think the
real test is to examine the puzzle-solving processes of the candidate, but as
a rule I don't think much of those kinds of questions.

------
thesis
Ok... I was under the impression that 37Signals hired mainly remote workers.
Could this account for the lack of a white board?

Personally, I use a whiteboard when hiring. It's not the deciding factor...
but it's a part of the bigger picture. As part of a small start up I need to
know right then and there if they know what I need them to know.

~~~
blktiger
I think using a whiteboard to test someone's design knowledge is probably a
good idea. I like to draw diagrams on a whiteboard when I'm designing a
solution to something. However, using a whiteboard for a CODING exercise is
stupid. When was the last time you wrote any significant amount of code on a
whiteboard? Give someone a laptop if you want to see their coding skills. Even
better, have them pair-program with one of your developers.

------
KeepTalking
Coming up with good questions to ask in an interview is an effort that the
hiring team needs to invest in. This involves a top down reflection of the job
along with a vision for the job. It requires to ask him/her self before hand
"What sort of tasks is this hire going to be doing" , " What language skills
does he need" , " Does he need to reinvent a sorting algorithm to get this job
" .

Well this rarely happens , to start of folks (generally) tasked with the
hiring process are not the right "vision" folks. They are highly task oriented
individuals aka MS Project specialists.

A real test would have to encompass among many things. \- Do they are ask the
right questions ? \- Are they capable of picking up new languages, ideas ,
technologies while defending or questioning changes in an articulate yet
respectable demeanor. \- how they handle things when they get to a wall.

------
saman_b
That's what I was discussing with my friend a few days ago. He had an
interview with Google last month and it took him a month or two to prepare for
that. he did not pass the interview, but he got an interview with Amazon this
month. He was busy with other things and he could not study for the interview.
He mentioned that he cannot remember most of the things he prepared and read
for the last interview. I am forgetful too, and I guess 90% of people forget
details about algorithms and how to answer a puzzle very fast if they don't
use it everyday.

So the question is what is the value of a skill or knowledge that fades away
after a month or two? I know people who got hired by Google or other big
companies that forgot all the information they acquired before the interview.
Cause most of it is useless for the actual job. Many logic questions or
puzzles are well known and people tend to know the answer before the
interview.

If I had to interview someone; I would assign him a project from the company
that is part the of the job he is going to do, leave him alone in a room with
a computer for a time period I expect someone to finish that specific project.
Finally, I come back to see the out come.

It is good for the applicant, as there is no pressure on him (except some time
constraints maybe which is not a big deal), I do not put him on the spot and
he does not have to deal with his boss while answering to a question. He does
not have to dig unrelated information before the interview, and he can show
off his programming skills if he has any.

It is good for the company, as I do not have to spend a day interviewing and
prepare for that, I will know right away from the code if the guy is a good
fit or not, I can see his programming skills, I know ahead of time how long in
average it takes for an internal person to do the job, so I can measure his
performance relatively. I do not even have to evaluate the code in front of
him, I can invite multiple candidates at the same time, put them in separate
rooms. Get the results, evaluate them and invite them for the second round if
I liked their code.

------
ianstallings
Well the good news is this problem should be solved soon since we spend so
much time focused on it.

I don't care for puzzles. Just conversations. I skim resumes looking for
glimpses into personality more than looking for skill sets. I have a guy that
went to Yale on my team. I didn't even know that until he mentioned it the
other day, yet it was on his resume. I missed it because I don't care where
you went. I don't care how fast you can solve a puzzle. It's put up or shut up
and show me what you've built when you come here. If you can do that in an
interview there's a great chance I'll hire you. The last two hires whipped out
apps they'd built and then we went over how they accomplished it. To me that's
more valuable than any other process, having a portfolio or sample ready when
you show up.

------
webo
I was interviewing with Microsoft last semester for a SE internship. At the
end of the interview with a senior engineer, we were just chatting about
various topics. This is one topic that we talked about. His response was
something along these lines:

"Of course we do not write code on whiteboard everyday. Of course we do not
write code to detect if a graph is acyclic (my interview question) on a daily
basis. Of course we do not put our engineers on the spot with these types of
questions. However, when the time comes, may it be only once in 10 years, we
know that we have hired the right people because these questions show your
thinking abilities - not if you know the answer or not."

Btw, I didn't get an offer, but I'm going to Amazon where I had very similar
interview process as Microsoft.

------
bjoe_lewis
Ok, this is true story and has nothing to do with my credibility. Few days ago
glenwood systems, a core product developing company conducted a recruitement
drive in my college. There was me, a coding geek with all my passion for one
thing, and there was him, who is a puzzle freak. They selected him past the
first two rounds full of puzzles and math, finally rejected him knowing his
programming incompetence. And they missed someone who really could help. me.
There are a ton of books out there teaching you how to crack interview puzzles
and programming riddles. There's no book telling you to get your hands dirty
with projects and code. The author is right. Hire the one who did things, not
the one who learnt to crack puzzles.

------
alan_cx
Not employed people for years, but I never put much value on ultimate skills.
I used to think long term. So, give me the the right human being, with a
reasonable level of knowledge, and very quickly they will be more than good
enough. A little more time and for me, they become pretty much perfect.

I want the right personality. The ultimate skills will follow. So, I just chat
to candidates and employ those who I feel I and my people can work with. I
never left it so late that we needed instant skills. To me that would be poor
management.

Prior to that, I have tested and done many of the things people cite here, but
most of the time I got awful employees who turned out to be good at doing
interviews.

Yeah, a bit hippy dippy, but it worked for me.

------
andrew_wc_brown
I won't present a résumé, a github link, a linked in profile, or do nonsense
tests. I always send a full web-application I built for code review, do a
technical demonstration and provide sample work on their codebase. Any other
route is a waste of time.

------
spiralpolitik
Programming is about problem solving. What you want in a programmer is someone
who knows how to approach solving a problem in a structured repeatable manner.

What getting someone to solve small problems like FizzBuzz in the interview
gives you an idea of how then approach the problem. Can they decompose the
problem into smaller chunks ? Can they then refine their solution into a
better solution ? Do they know when to stop refining before the solution
becomes difficult to understand ? These are the skills you want to asses from
a potential programmer.

Programming languages, platforms, frameworks come and go but if you don't know
how to approach solving a problem then that knowledge is ultimately worthless.

~~~
dpcan
I totally agree, but only in the FizzBuzz case.

It's the perfect, super-simple puzzle that can be solved quickly, but let's
you see how a programmer might approach a problem, how they like to code, and
their go-to language, etc.

It's not like you are trying to stump the programmer. Any programmer with a
lick of skill should be able to knock out the FizzBuzz problem in their
language of choice.

It's actually kind-of fun the first time you do it.

Personally, I'd watch the programmer's face. If they smiled after realizing
the problem is not as clear-cut as it seems, they probably enjoy programming.

------
sytelus
Some of the best hiring strategy I've came across is indeed "test driving".
Take a challenging problem your team has, slim it down and ask candidate to do
some quick prototype during a weekend. No interviews, puzzles or whiteboarding
:). If you were going to fly down a candidate on-site for a day of interviews
instead, it's going to take same time anyway considering logistical effort on
candidate's part.

Test-driving, however, is unfortunately neither scalable nor "leak proof". As
soon as one candidate gives away your test drive question, the next candidate
could easily cheat away inflating their apparent awesomeness compared to other
better candidates.

However it would be incorrect to write down all whiteboarding interviews as
"evil". Like any other interviewing techniques, it really depends on how you
do it. A good whiteboarding question that allows candidate to use CS
fundamentals to solve fairly non-trivial problem that you also needed to solve
for your current work is a great question. There are 3 major reason why this
doesn't work as expected at some large companies:

1\. Interviewers can't think of good whiteboarding question and fall back to
commonly asked popular puzzles. These interviewers are also often the ones who
have one "favorite" question that they would ask everyone. This is absolutely
#1 problem why whiteboarding interviews devolves in to secret puzzle marathon.
The best questions that interviewer could ask are actually the ones that they
needed to solve themselves as part of their work recently. This keeps
questions relevant and refreshed regularly. It also allows to compare
candidate with themselves and follow the philosophy of hiring people smarter
than you. It's however hard because interviewers themselves needs to continue
doing something interesting regularly.

2\. Interviewers provide feedback to each other during the loop. It's not
coincidence that if 1st interviewer gives "hire", all rest tend to do same at
companies where there is no clear policy of not communicating feedback before
end of the interview.

3\. Interviewers don't follow up candidate response by extending question,
building on to next level perhaps open ended scenario, asking auxiliary
details such as test cases, complexity etc.

------
btipling
Eh, being able to understand stacks and queues, big-O, etc and problem solving
are important things, but otherwise agree that software construction skills
are important as well as understanding tradeoffs bottlenecks, networking etc.

------
eranki
Why do we always feel the need to upvote posts like these as if we don't
discuss them every 2 weeks?

Next up in the topic cycle: Discrimination in tech. Why working more than 40
hours a week is killing your productivity.

------
ricardobeat
Aaand HN has just written a 10-hour book on the subject of hiring.

------
jdavis703
At my old company we'd tell interviewees to bring a laptop loaded with there
usual dev tools. After a couple of typical interview questions they were then
instructed to create a small program using whatever language/tools they'd
like. They had "unlimited" time and sat at an empty cubicle. Candidates also
has had full access to the Internet. It was amazing how several people could
not even complete the simplest program without needing extensive assistance.

------
megaframe
I wish this were the case more. I graduated with and electrical engineering
degree, but over the years I've spent the bulk of my time coding up DB, big
data stats, ML, etc. to help me do my job. Trying to explain I'm more than
capable while I falter with simple puzzle based question is an impossible
gauntlet that has kept me from even considering a career shift. (Granted I'm a
much better analog designer than I am a java code.)

------
army
I think the skills you need to throw together a nice RoR app are different
from other positions.

I wouldn't want on critical infrastructure code alongside someone who can't
reason through the performance implications of their code on the fly. You can
waste a hell of a lot of time diagnosing and fixing performance problems
because someone built an entire module around an inappropriate data structure.

------
zenbowman
It's one thing to ask people to right syntactically correct code. It's another
thing to ask them to write pseudocode in whatever language they please and
understanding their process.

At the end of the day, as Hal Abelson said, computer science is not about
computers, or science. It's about being able to formalize process.

------
maigret
Kent Beck tweeted a very fitting comment: 'Programming contest problems
shouldn't be algorithms, they should be like "set up continuous deployment of
a multi-region Django app on AWS"'

<https://twitter.com/KentBeck/status/299528659746840576>

~~~
papsosouid
Why should programming contest problems be tests of practical sysadmin skills?
That makes no sense at all.

~~~
maigret
Because this is much more than sysadmin skills involved. You've got to get a
scalable application written first. It doesn't need to be complicated, but it
needs to be robust and keep data consistent... etc.

Also, this type of skills is badly missing to fresh grads.

~~~
papsosouid
No you don't, it says setup a django app. That involves writing no code, and
is entirely a standard sysadmin task.

~~~
maigret
Script is code too. But I don't think you get there with plain sysadmin
skills. How do you sync up the data between the regions? I have worked on a
similar case not long ago, and there were definitely more than just "sysadmin"
skills involved. I suppose Kent Beck also went through the same exercise.
Pointing to the deficiency of usual contests is a good thing. In any case, if
all sysadmins can do continuous deployment, cloud management and worldwide
distributed apps, then I've definitely met the wrong ones before.

------
garraeth
Comments are closed on the article and I wanted to thank David.

I'm one of those who freeze up like a deer in headlights when asked to do a
quiz. But have a degree in CS, have made several enterprise level packages by
myself, and have been programming for 31 years -- so am fairly confident that
I can program.

------
asmosoinio
Really liked the text at the bottom:

(If you need help posting a comment, feel free to use any of these samples:
“You make todo lists, you don’t need real software engineers”, “Math is
actually really important, you know!”, “Google is worth one gajillion dollars
and they use quizzes, so there!”)

------
SeanDav
All great and well spending 20-40 hours trying someone out to see if they are
a good fit, but this is only practical once you are down to a very short list
of final candidates.

Nothing is said about how you actually get to this final list and that of
course is where the real challenge lies.

~~~
bcoates
Whatever happened to picking at random? It's optimal unless you know some
other method of narrowing down the list isn't pathological.

------
dschiptsov
Because no one will hire a writer based on his knowledge of long words.)

Excellence in the grammar is also irrelevant.

~~~
petercooper
And a doctor doesn't necessarily have to be fit and healthy.

~~~
dschiptsov
That's different.))

------
monksy
On Coding Portfolios:
[http://programmers.stackexchange.com/questions/102490/requir...](http://programmers.stackexchange.com/questions/102490/requiring-
programming-portfolios-in-interviews-recruiting)

------
bromang
is there any well known data/studies on selection methods for programmers?

I can understand the reasons why not to use algo problems as a filter, but I
would have thought simpler forms of intelligence testing would still be quite
useful.

~~~
fnordfnordfnord
I think Token has the longest list of that sort of thing. I'm sure he'll be
along shortly to post it ITT. Dig through his comments here if you can't wait.
<http://news.ycombinator.com/threads?id=tokenadult>

------
pjmlp
What is this continuous reference to FizzBuzz from American developers?

Being born in the other side of the Atlantic if it wasn't for HN, I would
never heard of FizzBuzz in my 14 years of software development.

~~~
TheCoelacanth
I think it's just because of well-known blog posts[1][2]. I don't think it's a
particularly common question for interviewers to ask here either.

[1] [http://www.codinghorror.com/blog/2007/02/why-cant-
programmer...](http://www.codinghorror.com/blog/2007/02/why-cant-programmers-
program.html)

[2] [http://www.codinghorror.com/blog/2007/02/fizzbuzz-the-
progra...](http://www.codinghorror.com/blog/2007/02/fizzbuzz-the-programmers-
stairway-to-heaven.html)

------
sippndipp
Agreed - but I guess if you're 37signals you just attract the right people and
you don't have to worry at all. But if you're not then maybe some of these
riddles are just good heuristics.

------
codex_irl
I like this: [http://cdnnew.theinspiration.com/wp-
content/uploads/tattoo_Q...](http://cdnnew.theinspiration.com/wp-
content/uploads/tattoo_QR.jpg)

------
fnordfnordfnord
It's too easy to prepare for the common ones. I have a list of them which I
share with my students before they graduate. Sometimes we work through them
for fun.

------
aosmith
Couldn't agree more. Every time I've agreed to a "programming test" I find
myself wonder why I agreed to it in the first place.

------
laureny
Congratulations to 37signals for adopting a hiring practice that most
companies have been using for more than a decade.

------
flexie
These puzzles and IQ tests are as pervasive as they are irrelevant. Nowadays,
you even need to pass one to get some government jobs. Here's the one to pass
to get a job in the EU agencies:

[https://europa.eu/epso/application/passport/quiz.cfm?lang=EN...](https://europa.eu/epso/application/passport/quiz.cfm?lang=EN&comp_id=1&quizid=10&f_sub=+OK)

------
geebee
One of my career goals is to get a programming job without having to show that
I can add a leaf to a binary tree through recursion (again).

I don't mean this as flippantly as it might sound. I used to think it was kind
of fun and exciting to be at the whiteboard, thinking on my feat, dealing with
various curveball data structures and algorithms questions. I certainly always
read up on this stuff prior to interviews, and from what I've heard, I'd be
advised to do it again if I get an interview at google or something.

But the last time I did this left me depressed. I realized that I've done so
much of my work in obscurity that people are still asking me about binary
trees. I want to be very clear that _I don't blame them_! I also started to
realize that I have a lot more to offer than my ability to traverse various
tree structures, and that the interviewers seem unaware of this. But it's up
to me to show them, not on them to just take me at my word.

The last time I went through a massive, all day technical interview, I didn't
get the job, partly because I didn't review my data structures and algorithms
book (again), but probably also because I came off as cranky and irritated. I
was busy that week, probably should have put it off.

Just earlier that week, I had taken several clients from pharma and
semiconductor companies through some beta testing of our software, refining
the model, getting feedback on use, optimizing the code base so it could give
answers more quickly. The company I was interviewing for created supply
planning, forecasting, and inventory management software for various large
businesses. As a math major with an MS in Industrial Engineering and a
background writing software, it was right up my alley.

My interviewers showed no interest in anything other than my ability to code
or a couple branches of math. In fact, some of them seemed so inexperienced
that I'm not sure they were aware that this kind of experience even exists to
be asked about (Oh, you work with clients? Our project managers do that). At
lunch, one guy asked me how to swap two integers without creating a third
integer. This is after a full morning of technical grilling on math and
programming, with a lengthy afternoon to follow.

Fortunately, I'm now in a job where I am encouraged, not just allowed, to
contribute to open source, do presentations at conferences, try to build
communities, and so forth. I would not consider any job that did not have this
element (and now that I have a job like this, I'm not looking anyway). But if
I were to look around, I would vastly prefer to be hired for my known and
widely used code base than for my ability to detect cycles in linked lists,
implement a hashing function, or print out all possible permutations of a
string (with no duplicates!).

I'm not there yet, but that's my career goal.

~~~
mmorett
"The last time I went through a massive, all day technical interview, I didn't
get the job, partly because I didn't review my data structures and algorithms
book (again)"

Which means you're not using that stuff on a daily basis. Which is fine.
You're probably doing other things. Development is more than just data
structures and algorithms. Yet, that's all many interviewers want to ask
about.

Here's my guess: you probably reviewed that stuff prior to job 1. You got that
job, due to short term memory, then you went to work. Now you want to go to
job 2 (or 3 or 4). And you find yourself having to "review" material that
doesn't stick to short term memory, but I'm sure you know damn well how to
find it and interpret it if you needed to. No doubt at all.

------
neeraj_r
Thats good. So we don't need anonymous to crack your system.. :)

------
pjungwir
I interviewed at several companies about a year ago, and one had a practice
that really stood out. I was impressed by how organized it was, how well it
seemed to be a good test of a job candidate, and how it gave me many
opportunities to show my qualifications. It was a whole day, but they flew me
out and put me up in a nice hotel nearby.

The day was broken up into sessions of about 1 hour each (maybe a bit more).
In each session it was me and 3-4 other people, but the people rotated, so I
got to meet lots of engineers, a PM or two, the hiring manager, etc., and they
all got to meet me. A few people showed up more than once.

The first session was basically an introduction. I think it was shorter than
the others. No quiz-style questions, but stuff about my background, etc., and
opportunities for me to ask questions also.

Next I did a presentation at the whiteboard describing some project I'd worked
on. They had told me this would be a part of the day, so I could have prepared
(though I didn't). I talked about my recent startup, describing some of the
data model and machine learning tricks I'd developed. They asked lots of
questions, some to clarify, others about my reasoning, how it would scale, if
I'd considered technology X, etc.

In the next session was me at the whiteboard again, but here they described a
new feature they'd like to add to their site, and my job was to sketch out a
rough solution. So partially this was about my ability to ask clarification
questions, and also about my big-picture design skills. Again they asked lots
of questions, and once we even got into a specific SQL query (where I opted to
use an EXISTS with a correlated sub-query). So this session seemed a lot like
the first, but more impromptu on my part and probably more practiced on
theirs.

Then they took the whole team out to lunch at a nice Thai restaurant (~15 devs
plus the director of engineering and the PM). This part was relaxing and fun.
Of course this was still part of the interview, and wouldn't be relaxing for
everyone, but still it was a good way to feel out personality fit for both
sides. Also there were too many people for me to be the focus of attention, so
that was a nice break.

Finally there was one afternoon session where they gave me a laptop with a toy
Rails app, and had me add a feature (change a relationship from one-to-one to
one-to-many, or something like that). There were two devs watching me code,
and I had to do the whole MVC thing: write a migration, tweak the controller,
edit the view. I got to talk through the changes as I did them, so they knew
my reasoning. It made me smile to slip in a few fancy vim moves; I don't know
if they noticed. But it made me realize this was also a test about some basic
mechanics, too. They let me choose my editor/IDE, so it was a chance for them
to watch whether I knew my tools.

Then there was a final session for follow-up questions etc. This was fairly
short I believe.

All along the way I had lots of opportunities to ask questions of my own,
which I appreciated.

There were no puzzles, no trivia quizzes. Personally I'd add a FizzBuzz test
to the initial phone screen (like Han Solo "They hardly asked me any
questions."), but otherwise I plan to completely rip off this template the
next time I hire someone myself. Maybe it can help some of you!

------
known
interview != quiz

------
papsosouid
>I remember the first time I interviewed for a front-end programming position
and got asked how to do something in JavaScript on a white board

>how little it had to do with the actual job.

That doesn't make any sense. How is solving a problem with javascript
irrelevant to the job of a front end developer? That's what they do.

~~~
Ziomislaw
oh! you write your code on a whiteboard? do you have to type it into a
computer by yourself or do you have people doing this for you? ;)

~~~
papsosouid
Writing it on a whiteboard instead of in a text editor takes it from 100%
completely relevant to 100% completely irrelevant?

------
edwardunknown
I'd never work for a place that gave me a fucking puzzle to solve. I'd
probably smack the interviewer upside the head for wasting my time.

The best way to do it is pay the person per task as an independent contractor
until you're sure a full time position is what you're both looking for. If
they're helpful to you then you keep calling them and if not everybody keeps
their dignity, no big deal. Don't humiliate potential employees right off the
bat with stupid games.

------
michaelochurch
I'm going to avoid the cultural flamewar around whether companies _should_ do
this.

I don't mind it. I've interviewed for enough jobs and had enough acceptances
and rejections to learn something: it's not a big deal. If you don't know an
API function, you don't know it. The likelihood that you won't get the job
because you didn't know that one question is unlikely.

In fact, part of the evaluation is whether you can handle _not_ knowing the
answer with grace. The fail-outs are the ones who use their cool or seem to
think the question is stupid, not the ones who get it wrong.

Once you realize that you don't need to get all of the answers to pass an
interview, it gets easier.

------
huhsamovar
No need to read. The author of this post has clearly missed the point.

~~~
tanel
word, to be honest I don't get it why it's on hn.

------
mnemosyne51
I hope someone from Goldman Sucks reads this.

------
misiti3780
I have heard google is a big violator of this?

