
Coding Horror: The Hardest Interview Puzzle Question Ever - Anon84
http://www.codinghorror.com/blog/archives/001243.html
======
lbrandy
This is a false dichotomy. Having the interviewee give a brief talk and
following with a dialogue about something they've worked on in the past is
absolutely vital. An interview consisting of that exclusively is probably
better than an interview consisting solely of brain teasers. However, that's a
silly comparison.

Way too many questions get labeled (on the internets) as "puzzles" and "brain
teasers" when they are simply abstract algorithm questions. Crossing rivers
with chickens and wolves, and/or figuring out the prisoners with the one room
and the light-bulbs and all that crap are brain teasers and are totally
useless. Impossible questions are not brain teasers. If I asked you how many
tires are in the city of Chicago, and you said 1000, what should I think? What
does that tell me?

Even still, way too many programmers decide to get intellectually offended (at
least on the internet) when they hear about some problem that may involve,
say, dropping light bulbs off a building to find the highest floor from which
they can be safely dropped. This is nothing but an algorithm development
question for a problem that doesn't have an off-the-shelf solution. This is
testing your algorithmic thinking skills.

And as always I hear that phrasing such and such algorithm question "in real
terms" is better than phrasing it abstractly. I've never seen a compelling
argument that this is true except for this faux outrage that people have about
puzzle questions. Phrasing it abstractly is quicker, simpler, and clearer. It
removes all the hemming and hawing about implementation details. It removes
answers like "doesn't intel have a library that does that?". It gets to the
point, and quickly.

There is a second problem with that angst, and comments like: "I just can't
muster any enthusiasm for completely random arbitrary puzzles in the face of
so many actual, real world problems.". It is that we are computer scientists
and abstract problem solving is something we should care about. Abstraction
shouldn't be some barrier that makes it hard for you to think about a simple
problem. Abstracting the real-world details away from you shouldn't get you
emotional.

Besides, if I had an interview that was only a water cooler talk about
something in the past, Jeff Atwood would smack that out of the park. And then
I'd end up with a shop full of Jeff Atwoods. wink.

~~~
jerf
I think this is another "In theory, theory and practice are the same. In
practice, they are not." issue.

In theory, theoretical questions are great. In practice, they are fidgety
little things that must be asked precisely correctly to have the same
"correct" answer that the interviewer has memorized and are frequently
botched, almost invariably involve a "gotcha" answer in such a way that
honestly I do not want to see a coder actually code, and have a laser-like
focus on an aspect of the job that in most cases will involve less than 1% of
your job, no exaggeration.

I've done some algorithms work at my job, but in all my years it has always
been replacing O(n^2) or O(n^3) solutions with O(n) or O(n log n) in very
straightforward ways. Had I chosen an O(n log log n) answer, it would have
been wrong for being radically more complicated.

Basically, my problem with the puzzle questions is they show a misplaced
priority. You _could_ be asking gotcha puzzle questions (that, by the way,
I've probably already read and memorized the answer...), _or_ you could ask
the interviewee to write a script in their favorite language to count the
words or whatever, and learn _both_ everything the puzzle question would have
taught you _and_ whether they can code.

This may be a side effect of the fact that I refuse to spend eight hours
interviewing someone. I want it done in less than half an hour unless there's
a good reason to make it go longer (which does happen). Puzzle questions are a
waste of time in that context. I guess if you're planning on wasting a
candidate's entire day, you can afford wasting time on puzzles. (I have read
the science that says we make decisions on the hiring issue far faster than in
eight hours, which matches my experience, so why sit there and fiddle around
all day when the decision has been made?)

~~~
raganwald
_I guess if you're planning on wasting a candidate's entire day, you can
afford wasting time on puzzles. (I have read the science that says we make
decisions on the hiring issue far faster than in eight hours, which matches my
experience, so why sit there and fiddle around all day when the decision has
been made?)_

I find it easy to believe that many people make a decision in far less than
eight hours of interviewing, however I do not believe that all interview
processes lasting eight hours are a waste of time for the candidate and for
the interviewers.

To pick an example, what about a process involving twelve different
interviewers, each of whom takes between a half hour and an hour, with the
process spread over a certain amount of elapsed time? Can we really assert
that we cannot make a better decision with this process than with having one
interviewer spend thirty minutes with the candidate?

~~~
jwilliams
All the research I've seen seems to draw the conclusion that the only strong
predictor of performance in a job -- is past performance.

Puzzles and psychometric testing are useful, but have a lot of limitations.
However, you may have a job that involves a high level of responsibility (e.g.
people's lives) and you want to assess their response to stress/crisis. Again,
it's not perfect -- you don't truely know how someone will respond in a unique
situation -- but in those circumstances you might want all the information
possible.

References, by and large, are useless.

Cultural fit is incredibly important, but there are very few interviewers or
interview techniques that do this adequately. There is so much bias in this
it's hard to assess very well.

So what are the keys for me? Validating what they have on their CV. If they
have experience with a particularly technology, I ask about it. I propose
problems that they are likely to encounter every day in their job, and see how
they go about fixing it. From what I've seen, abstract problems do not
exercise this sufficiently.

What is it from there? Well, if what they have been doing is a fit for the
job, then that's great. If there is a gap, there is an assessment on what it
will take to bridge. How a person has conducted their career is a good
indicator on how achievable this is.

~~~
jganetsk
_References, by and large, are useless._

Marissa Mayer says, in all their studies of the Google interviewing process,
references are the best indicator of future performance.

~~~
jwilliams
Yeah. That's why I left it slightly qualified - references are only useful if
there is a strong network that can recommend and validate people.

This was probably certainly the case for Google, but probably not for many
others.

------
crescendo
This kind of practice really irks me. I'm not sure what information these
companies are going after, but what they're actually getting is "people who
read lots of brain teaser books". Somehow I doubt that's what they're looking
for.

~~~
Anon84
Brain teasers are kind of extreme problem solving. Being able to tackle
apparently impossible problems is a useful skill to have in most business
situations. Think of it as a way to see if you can "think outside the box
(tm)"

~~~
mixmax
I would say that being good at brainteaser problems only proves that you're
good at brainteaser problems. Nothing else.

~~~
raganwald
Yes. And no.

Take yourself back to 1989 or so when very few companies asked brain teaser
questions so therefore there were very few people who studied brain teasers
specifically to ace job interviews. The question for an interviewer was
whether being good at brain teasers _correlated_ to being good at a
programming job in a statistically significant manner. Now combine that with a
number of other interview techniques, each of which correlates well and you
may have had something.

If you must search for reasons and meanings, it was probably because the kind
of people who studied brain teasers for fun also happened to have a certain
knack for the math-y problem-solving part of programming. Or they actually
enjoyed solving problems for fun. Or both. I am not suggesting that we need to
prove that the skill of solving a brain teaser was an important programming
skill, it could be there was a root talent which expressed itself in
programming and brain teaser abilities simultaneously. Or that some event in
their past opened them up to learning two different and unrelated skills. All
that mattered was whether you could measure a correlation.

But that was then. Today, I avoid such questions entirely. However, that is
because IMO it no longer correlates well. Mostly because it has become popular
enough that interviewees game the system by swotting up on brain teasers, so
being good at them no longer means you have innate talent for math-y problem
solving, nor does it mean you enjoy solving problems for fun.

So in summary I say that being good at brainteaser problems in a job interview
no longer even proves that you're good at brain teaser problems, or like
solving them for fun, but I disagree that it has never conveyed useful
information to an interviewer.

~~~
mixmax
The post was primarily based on my (limited) experence with good hackers I
know and their ability to solve brain teasers. Most of them, incidentally, are
not very good at it. The sample base is small though, so it might be skewed.

Also, in my opinion a lot of them are "aha" questions - once you get them
they're obvious. They don't require deductive thinking, but rather that you
"get" this particular question. A good example of this might be "Why are
manhole covers round?" The answer is that round covers are bigger than the
hole no matter how you turn them, and thus can't fall in. It's hard to deduct
your way to this conclusion, yet once you get it it's obvious.

Questions such as "how many pianos are there in New York?" require a bit more
deductive thinking, but I'm sceptical as to how much they say about someones
ability to code.

But, as you point out, if there's a correlation, hoowever small, and you get
other loosely correlated data it adds up.

~~~
harpastum
I agree that this type of question isn't helpful because it's so easy to
memorize the 'tricks' instead of thinking about the solution, but I find it
fairly simple to deduct the reason for manhole cover shapes:

First, determine the requirements of a manhole cover (strong enough for semis
to roll over it, movable for access, big enough for most people to fit in the
hole) and the possible issues (theft, wear/tear/destruction, falling into the
hole).

With these requirements, it's pretty easy to design an acceptable manhole
cover:

1\. The manhole cover must be at least 2' in diameter to allow people to go
into the hole.

2\. The manhole cover should be thick metal, both to increase life and to
prevent theft (some manhole covers weigh more than 80 lbs).

3\. The manhole cover should have a hole/catch mechanism in it to help open
it.

4\. The manhole cover should not be able to fall into the hole, so should be a
shape that can't fall into itself (circles are the only shape I know of that
follow this).

In the real world, many manhole covers do not follow all of these requirements
(I've seen triangle-shaped ones), but this design outline describes the
manhole covers near me pretty accurately.

I would argue the 'spirit' of the question (determining whether or not the
applicant can use deduction to find an answer to a tricky problem) is still
extremely important--it's just that this type of problem (and especially the
manhole cover example) has been gamed.

~~~
raganwald
I love discussing the manhole question because the alleged answer is not the
point. The point of such a question is for the candidate to think aloud and
walk you through their problem exploration process. Such questions select for
people who are a little meta about themselves, which is why they can think
about something and talk about their thinking at the same time.

The infamous retort is this parody:
<http://hebig.org/blogs/archives/main/000962.php>

But that really illustrates the trap of almost all interview questions. The
interviewer is obsessed with the correct answer, when in reality it's the
process of arriving at any answer that matters.

My personal beef with "Aha!" questions is that if given one that I actually
solve on the spot, I can't tell you much about how I got it. It just came to
me.

The only exception I can remember is a question about finding whether a linked
list has a cycle in it. The "proper" answer involves a pair of cursors, one of
which operates at double the speed of the other. I came up with a much less
efficient but equivalent solution because the problem reminded me of something
we did with Turing machines in college back in the 80s.

So in that particular case I could actually explain how I arrived at an
answer. Now that I think about it, I think I'm a pattern-matching machine. My
algorithm for solving problems is to perturb them until they resemble
something I've seen before.

------
ewanmcteagle
This is a rough correlate with intelligence testing. There's nothing wrong
with that. The problem most people keep bringing up with these things is that
you will miss some good people that they have worked with who couldn't have
done these problems. That may be so but the real fear should be of people you
hire not of people you pass up. Hiring people is much easier than firing
people. If you can get someone who's good at puzzles and is good in other ways
you've only gained something not lost. I think the other reason people are
offended is because they themselves are not good at these problems and take it
as an insult.

On another point, you can't just get 'good at puzzles.' It's really not
possible. Even if you are learning tricks and techniques for piercing these
things that type of thinking can be applicable elsewhere. It doesn't even have
to be algorithms. Working your brain is a reasonable thing to do. Just
signaling that you are willing to do so can be of interest to an employer.

------
lacker
It's too easy to fake skill if you let the interview candidate talk about
whatever they want. A lot of PhDs who can't really program will sound great if
you let them control the topic, but ask them to write code to find the most
frequent word in a page of text and they will be stumped.

------
Eliezer
The actual answer to the "hardest interview puzzle question ever" is 17, and
the pirate starts from the left.

------
jwilliams
Here is a hard interview question - "Here is a problem we encounter every day
at X company. Have you encountered this before in a previous project? How
would you go about solving it?"

~~~
iigs
I've had a hard time getting that prototype to work as a valid question
because I've always found I'm leading candidates through the thought process.
I have yet to find a way to ask it in an open ended manner and receive a
satisfactory (meaning it has walked enough of the path to prove a candidate
can engineer) answer.

~~~
jwilliams
Yeah, this is a confound - It's not unrealistic though - as working through a
problem together is likely a part of your ongoing work relationship.

This is where you need to start injecting things like "Don't you think?..." or
"I hadn't thought about it that way, tell me more...". That kind of
interaction will be come critical day to day, it also helps test the depth of
the candidate.

I guess that's why I don't like puzzles - it feels too much like a trick
(also, I'm terrible at them). What I usually look for are teammates - people I
can work together with to crack problems.

I guess it just comes down to picking something that draws that out -
something realistic, but without too many preconceptions on the interviewer's
part.

------
robotron
I would walk out of the interview because, obviously, the interviewer (or
crafter of the questions) is a special kind of douche bag.

------
binarycheese
That's why Microsoft lost to Google

~~~
mkuhn
I think this is a bit premature and oversimplified...

