
Enough with the Programming Puzzles - roncox
https://www.linkedin.com/pulse/enough-programming-puzzles-ron-cox
======
kinkdr
Us, software engineers, start to sound like spoiled kids when it comes to
interviewing.

We don't like writing code on the whiteboard, we don't like mind teasers, we
don't like programming puzzles, and so on. We complain about everything.

Well, hiring is a difficult problem, and nobody has a universal solution yet.
But the truth is that if I want to get a job, I will need eat the humble pie
and get out there and try to perform as good as I can to whatever I am asked.
If I don't, somebody else will.

However, remember that interview is a two way process, if I don't like the
employer's process, then maybe I will not even interview for them. In this
case though, I cannot expect them to give me an offer, just because.

~~~
pandaman
It seems that other professions have solved this. Nobody is asking lawyers to
run a trial on a whiteboard or solve a whodunit mystery in 2 hours. And I
don't think that licensing process for lawyers guarantees anything other than
the right to practice law (there are pretty bad licensed lawyers out there).
Do construction workers draw diagrams of installing drywall in their hiring
process? Do plumbers get challenged with the back of cereal box trace-the-pipe
puzzle?

There are jobs, which have a similar hiring process. Performance artists
actually need to perform to get hired but they do perform their actual work.
I.e. they are not playing the anthem of Cambodia on a toy recorder in order to
be hired as a death metal band drummer.

~~~
wanderer2323
>I.e. they are not playing the anthem of Cambodia on a toy recorder in order
to be hired as a death metal band drummer.

Think about it this way: they are not playing a full playlist on an actual
stage with a soundcheck and live audience either. Compared to the real gig
they are performing a smaller task in an artificially simplified environment.
It is actually very similar to a coding interview.

People often forget that puzzles are not a thing into itself, it's a tool for
discovering candidate's style and a springboard for further discussion. I'm
not interested in a solution itself as much as in an ability to discuss it,
explain it & improve it -- because those are the things that are being done in
a workplace on a daily basis.

~~~
pandaman
People also forget the goal of the hiring process - finding somebody who can
get the work done. I am in the game industry and, luckily, not everybody is
doing whiteboard coding and puzzles here, but, out of my anecdotal experience,
studios that do it also release pretty crappy games. It could be that
springboarding discussion is not selecting very good programmers. It could
also be that the management who enforces these practices is not competent to
run the studio. I cannot tell either way, just my experience.

~~~
wanderer2323
Out of curiosity, what do the studios that don't release crappy games do?

~~~
pandaman
Math and general CS (in the sense of "What is IEEE representation of 1.f?" not
"The difference between Red-Black and B- trees?") questions. Past experience
questions.

------
wturner
Those puzzles might serve an ancillary goal of finding people willing to
conform, are good at doing what they are told and have the ability to do
unconventional programming tasks on a whim.

All of those are characteristics that the company might be looking for.

In short those tests aren't checking for just aptitude, they are checking for
the personality characteristics of people willing to do them and tolerate the
absurdity - like a cult.

I once applied as a JavaScript developer to a local company and they gave me a
test in Cake PHP. This was after a face-to-face interview with the owner.

I finished most of it but their was one thing I couldn't figure out. His
response via email was along the lines of " You really should learn this
stuff". This was after answering a bunch of granular questions about
JavaScript and passing those preliminary questions.

But I guess its my fault and I "really should learn this stuff" (j/k)

:/

~~~
simula67
These tests are probably designed to check your motivation, not your skill.

If Google is asking questions about manhole covers, you better study about
them before an interview. A candidate who can answer such a question would
more likely want to badly work for Google. That is probably a better predictor
for a good workplace performance than raw skill.

~~~
TheOtherHobbes
It's a good predictor for wanting to work at Google.

It's in no way a good predictor for being able to do good work at Google - or
anywhere else.

------
dclowd9901
Isn't it difficult to reject these puzzles on principle without sounding
either bitter or like you simply can't solve them? I can't imagine a single
case where this is going to cause a potential employer to re examine these
exercises. They'll simply chock you up to being unqualified.

I reckon most jobs simply require you to not be a knob for an hour while
someone asks you tricky questions about your worst behaviors. How often, day
to day, are you required to explain something bad about yourself in a
constructive way? Yet it is as utterly common as programming puzzles.

My supposition is that if that's where the wind is blowing, I'd had damn
better face my sail that way. I hate silly problem solving puzzles as much as
anyone, but not a shit lot you can do about it when they're the ones holding
the keys.

~~~
efaref
Well, I guess you could start your own company. You could even hire anyone who
comes in the door claiming to be able to code, although what you'll do with a
dozen people who can't even FizzBuzz is beyond me.

------
lewisl9029
I'm actually in the middle of a bunch of interviews right now. The most
ridiculous interview process I encountered so far was one that was supposed to
be 7 (!!) steps long (that usually takes 3-4 weeks by their own estimates),
beginning with a 45 minute HackerRank challenge, then a technical project that
was said to take 2-5 hours on average (but the time given was a whole week, so
I have my doubts about that estimate), followed by up to 4 Skype calls, and
finally an onsite.

The HackerRank challenge consisted of two relatively trivial multiple choice
problems, a simple FizzBuzz-level coding problem, and a programming puzzle.
And I do mean programming _puzzle_. I consider it a puzzle in the sense that
it had very little to do with any classical CS domain I'm aware of (although
that may just be an artifact of my own ignorance, but to someone unaware of
that domain like myself, it as might as well just be a puzzle) and certainly
wasn't anything one would ever encounter in front-end development (I was
applying as a front-end engineer). I later found the problem online and here's
a thread discussing it for those interested (the company I interviewed with
wasn't Google though): [http://stackoverflow.com/questions/28967020/check-if-
there-e...](http://stackoverflow.com/questions/28967020/check-if-there-exists-
a-circle)

I finished the 2 MC questions and first coding problem within the first 5
minutes, but I wasn't able to finish the programming puzzle in the time
remaining (For what it's worth, I was on the right track with my approach, but
it took me a lot of trial and error to arrive at that approach, so I didn't
have enough time left to finish the implementation). I got my rejection email
two days later.

I really question the wisdom of using a programming puzzle like this as a
filter in the first step of your interview process (or really, in any step of
the interview process for that matter). You're effectively discarding
candidates solely based on their inability to solve an esoteric programming
puzzle under some arbitrary time constraint.

Although at the end of the day, I'm not that disappointed in this early
rejection. If this programming puzzle was a sign of things to come in the next
6 steps of their interview process, I think I may have dodged a bullet. That
could have been so many hours of my life that I'd never get back. Good
riddance I say.

~~~
penguinduck
You dismiss this as "esoteric", but that's the whole point, the purpose of a
puzzle like this is for you to be unable to map it to something you've done
previously or to something you learned in school, and instead to force you to
invent an algorithm on your own.

It is the opposite of what the author of the article complains about -
"academic", "targeted at cs grads" \- it goes against that. It is an attempt
to test problem-solving ability by removing knowledge from the equation.
Knowledge can then be tested separately. Fresh CS grads are not going to do
better on this problem than a guy who graduated 10 years ago.

In fact, this is the kind of problem given to children (who can't be expected
to have CS knowledge, and you don't want to give a big advantage to those who
happen to have some) in programming competitions. I would put the difficulty
at either easy problem for high schoolers or difficult problem for elementary
schoolers (this doesn't mean it's super-easy, these problems have to be
relatively hard or the smartest kids would all solve everything).

I am not sure why so many developers dismiss the value of this and are in fact
often actively hostile to any suggestion that it might be ONE of the
indicators of programming ability.

I do agree that the time constraint was probably too strict and favors people
who've solved stuff like this before, mainly due to them knowing they should
try to solve this with a pen and paper or in their head, then just code the
solution once they've found it. Trial and error through code is a huge waste
of time and this was probably your problem.

~~~
lewisl9029
> I am not sure why so many developers dismiss the value of this and are in
> fact often actively hostile to any suggestion that it might be ONE of the
> indicators of programming ability.

No disagreement there. I certainly wasn't trying to dismiss the ability to
come up with novel solutions to problems never encountered as an indicator for
programming ability and general intelligence _completely_. However, I
certainly don't believe it's a _good enough_ indicator to use as the sole
basis for simply rejecting otherwise perfectly capable candidates before
you've even talked to them.

I'm also of the opinion that interviews should strive to assess how well
candidates can perform on the specific position you're hiring for (that is the
main purpose of interviews, is it not?). In my own case (I was applying for a
front-end engineer), and in the case of most other engineering positions, I
don't believe puzzle solving is a very _useful_ indicator for that purpose,
because an overwhelming majority of engineering tasks involve applying the
scientific method and engineering knowledge to solve variations of well-known
engineering problems in their specific fields.

EDIT: Another point that I forgot to mention: If you are in fact interested in
a candidate's ability to solve novel problems they've never encountered, at
least do a better job of ensuring the problem you give them is _actually
novel_. I found the StackOverflow thread for the problem they gave me in about
30 seconds of Googling. By not ensuring the problem novel enough that
searching for it won't do you any good, you're going to be disproportionately
passing dishonest candidates who would simply cheat to get through this part
of your process. Certainly that can't be the result you're looking for.

~~~
penguinduck
Regarding the first, I think they basically do this to test how "programming
smart" someone is. Like my boss tells me when I interview, "determine
potential". However, I agree that this shouldn't disqualify someone for a more
technical kind of job. For me it is more of a "if he is really smart we should
hire him despite the fact he is perhaps unqualified, he will easily learn"
thing. The disqualifying questions are easier than this one.

Regarding the second, I agree completely. This problem is good because it
doesn't give an advantage to rote memorization of basic CS from college
(unlike asking about red-black trees etc), but when stolen from the internet
and reused becomes bad because it gives an advantage to something worse -
googling questions and memorizing answers.

I assume (or hope) Google switches up the problems they use all the time and
hopefully doesn't use automated testing (because I care how someone thinks,
not just whether they produce the solution in X minutes), but since you
interviewed at a different company perhaps it is a cargo cult effect: Google
has some of the smartest programmers and they asked this, so obviously we
should also ask the same thing! Cargo culting is extremely common in IT in
less-established companies, my workplace is not immune to it either. Pick
things a super-successful company does that require almost zero investment,
effort or true change, and copy those. It is ridiculous.

Of course, the things these companies really do right, they are too much
effort, let's just copy them in superficial ways and hope for the best!

------
philbarr
One of the problems with coding challenges is that I suspect you miss out on a
lot of potential candidates. I did a round of interviews recently and one
wanted me to do an online test and then a puzzle.

I did the online test, received the puzzle, and in the intervening period I
was interviewed and offered a job by someone else. When I rang the first place
to tell them the guy was very upset because it had taken so long to find
someone who could pass the test!

Seems to me if you're that desperate for candidates then don't assume you're
the only company in the world.

~~~
pmyjavec
I would have to largely agree with this based on my own experience, and I've
been on both sides of the fence.

The best interviews I've had was when I was invited to come work with the team
after a few meet and greets. It allows all parties to relax a little and get a
taste of what working together would be like.

------
m0nty
Do people applying for other roles get treated to tests like this? Seems
peculiar to IT afaict. Having worked in IT for > 25 years, I would have
thought my track record would speak for itself, but still I'm treated as
though I've been lying and cheating my way through my career. It's not like
just asking questions, either: it's a test of basic aptitude and (like the
author) I'm at the stage of refusing to do them.

~~~
kriro
There's assessment centers for most decent entry level management jobs which
usually involve grouping tasks by importance and team oriented tasks. They can
last a day, sometimes two. Typical consulting gigs have case studies and
market size guesstimating (the dreaded how many lawnmowers in NYC etc.)

Typical office secretary jobs here require a work sample so you'll get a
typical task for the job and have to do it on the spot (write a letter to
someone, translate a text). My current job (academia) required a somewhat
simple test to see if I have a decent understanding of method/statistics and
the programming test was answering a couple of questions :P

In logistics you usually have to reason through a couple of typical work
situations.

After entry level...it's usually just an interview and references.

~~~
gaius
_After entry level...it 's usually just an interview and references._

Except it's not, that's the problem. But this is not a matter of incompetence
on the part of the employer, but of deliberate and calculated policy.

~~~
kriro
Just to clarify. I was referring to the non-programming jobs. So for an entry
level management job you'll do the assessment center and jump through all
sorts of hoops. For your next job it's usually a basic discussion. It'll get
more rigid again higher up as there will be a vetting process etc. But beyond
entry level there's usually no skill tests in the domains I know outside of
IT. I think that's the main point. Most people could accept all sorts of
programming and possibly even logic tests for entry level IT jobs but after
you've shipped software it becomes a bit ridiculous in many cases.

------
WatchDog
I don't know what the solution to hiring is, but I don't think it is as simple
as just talking to people. While I have not been involved in too many new
hires, I have known many people that come across as knowledgeable in
conversation but produce pretty terrible code.

I think coding challenges provide some value, but it can be a but much to
expect candidates to spend hour's of their time early in the process

------
vacri
Surely there's a middle-ground between puzzles designed to trip you up, and
puzzles designed to show you have some baseline chops. There's a lot of people
who can't walk their talk.

At one place I work at, we're trying to get some senior-level PHP devs... and
we're getting some applicants that can't even conceptually do FizzBuzz...

~~~
collyw
Its not usually that difficult to work out if people know what they are
talking about.

~~~
efaref
Some people are _really_ good at bullshitting their way through stuff. You
need to make them actually do stuff to discover they flounder.

And since you can't have them do anything meaningful in an interview (there
isn't enough time), anything you ask them to do will be a silly "programming
puzzle".

------
hartror
As an interviewer I am upfront with our hiring process and that part of it is
we ask candidates to do a "exercise" after the first face to face meeting.
This exercise is crafted to be representative of the job we are trying to fill
and is followed up by a second face to face code review session of the code.

The key of the exercise isn't necessarily to complete the whole task (though
often people do as the exercises are interesting) but to provide a discussion
piece for the code review session. Candidates are told this so that they don't
feel like they have to put too much work into it (2 hours is suggested).

The code review session is key, walking through someones code and discussing
decisions and trade offs. It provides a layered insight into their smarts,
communication skills and thought processes. It is the closest I've gotten to
actually working with someone in the interview process.

And isn't that what both parties are looking for?

------
sandworm101
So I guess we know the answer to the "How would you react if assigned a task
you didn't enjoy?" question. A: Rather than spend two hours getting the job
done, an hour is spent doing a write-up on why the task wasn't done.

A job, any job, will involve thousands of hours with at least a few hundred of
them on tasks you really do not appreciate. A candidate not willing to suffer
for a couple hours during the interview probably wouldn't have lasted very
long anyway.

~~~
philbarr
> A job, any job, will involve thousands of hours with at least a few hundred
> of them on tasks you really do not appreciate.

But you get paid to do those tasks. And remember that those 2 hours are an
estimate, which means it'll probably take longer. Then multiply that by the
number of interviews you have and you can quickly see why it becomes unfair
rather than 'lazy'.

~~~
efaref
Sometimes you have to work on spec.

------
gaius
Typical LinkedIn "thought leader", didn't Google et al abandon brainteasers
over a year ago?

~~~
joshvm
The article isn't talking about brain teasers. Google did indeed banish them,
but now they've switched to 4-5 interviews where you're asked a short coding
problem.

The problems themselves aren't difficult and given the time constraint, you
can fit the answer in the height of one whiteboard. The trick is that you have
to be fast, optimal and correct (i.e. syntax counts). From my experience, you
should be able to complete each problem in about 20 minutes - i.e. a good
enough solution that the interviewer moves on.

The article is right: there are books and books on doing these toy problems
with array manipulation, tree traversals, sort/search and so on. If you did a
TopCoder SRM every night for a few months, you would probably ace most of
these interviews.

But this is the issue: the interviews basically test how well you prepared.
They don't simulate a work environment and I don't think I was ever asked any
serious questions about my work experience/research beyond chit-chat. Given
that I was applying for a specialist computer vision role, that was a bit
surprising.

------
asimpletune
I wonder if the author didn't know enough about the company to know if he
wanted to do an online coding test, then how was he certain that the content
wasn't relevant to what the company was looking for?

------
divkakwani
I think a candidate needs to have both algorithmic and software design skills.
Companies put a lot of emphasis on the former skill, ignoring the latter,
which imo should equally be tested.

------
th0ma5
I think it is a very fair point that if you have to use a puzzle maybe you're
not attracting the right candidates.

~~~
collyw
You are certainly putting off some candidates.

------
collyw
Well done for replying back with a reasoned response for not participating. I
have done this at a few places. I hope if enough people do it people will get
rid of these interview techniques.

------
jarcane
If you want to find out if a programmer can actually do the job, then it seems
to me that the way forward is to have them do something that actually
resembles the job.

In the high-end cooking world, it's not uncommon to do one of two things when
looking to hire a new cook:

1) Make me a dish. Present the chef with the kitchen, and possibly a specific
main ingredient(s), and have them make the best dish they can with that
ingredient. Sometimes this is also under some kind of time constraint, to test
pressure under fire, but doesn't have to be (great chefs might take weeks to
develop a new recipe after all).

2) The "stage". The kitchen takes on the chef temporarily for a short period,
where they will actually work on different portions of the line, so that the
employer can see both how they perform in real time and where their strengths
lie. This can be as little as a single shift, or up to a month or more at some
very, very high end kitchens like Thomas Keller's. Crucially as well, this is
generally _paid_ (it'd actually be illegal otherwise), so the prospective
hiree can actually afford to dedicate that kind of time.

Now some of this is similar to stuff that some tech companies do already. But
I think they often miss crucial elements that makes this actually work in a
way that's not painful for both parties.

The "make me a dish" method for instance, superficially resembles the infamous
"sit down at our computer and code a thing" tests I've read of many times. But
there are key differences. Tooling is a pain point in programming that doesn't
so much exist in cooking (in fact part of this test can be making sure the
prospective cook is familiar with all the basic tools). Sitting in at a
foreign editor/IDE/language etc. is likely to be more stressful than is really
accurate to how one can expect a new coder to perform in the real world. The
simple solution is "bring your own knives." Let the coder use their own
machine, or at least give them time to set up a basic version of their
favorite tools.

The other thing about "make me a dish" that's missing from programming
equivalents that is I think most important is _creativity_. "Make me a dish"
isn't about just proving you can cook, it's about proving you can _create_.
And isn't that something we want from great programmers as well? Generally
when I see and read about these tasks their very proscribed: there's a
specific spec/demand/puzzle, and often with the expectation it will be solved
in a particular way. One test task I was given once actually even specified
what data format I was supposed to use to communicate between back and front
ends. What about a mini-hackathon/jam approach instead? Give me a subject or a
data source and a bit of time, and see what I can come with?

The "stage" as well is something I think is underconsidered with tech
companies. Companies spend thousands of dollars on hiring practices and
recruiters and advertising, but they can't just do a paid trial period
instead? It makes no sense. If you think you've got a promising candidate,
just take them on for a couple weeks, point them at a relatively low-hanging
fruit task on the issues board, and see how they do.

Alternately, maybe just accept a fucking risk now and again. If the candidate
seems to know what they're talking about in the interview, maybe just hire the
fucking person, and if they suck, fire them. This is the same stricture and
risk 90% of hiring practices are under, and somehow the economy hasn't
collapsed yet so I am gonna tentatively suggest it seems to work.

Sometimes the path of least resistance is the easier, cheaper way.

~~~
laichzeit0
> If the candidate seems to know what they're talking about in the interview,
> maybe just hire the fucking person, and if they suck, fire them.

In some countries, employment laws are so crazy, that it's almost impossible
to fire someone willy nilly because they "suck". You have to give them written
warnings, re-"skill" them, move them to different departments, etc. It's such
a pain in the ass and then they still take you to court for unfair dismissal.

~~~
seren
Even in countries with labor laws, people usually have a 3 to 6 months trial
periods where they can be fired at will.

I think the true deterrent is not the law, but that it is rather expensive to
train someone for 2 months then have to fire him/her, and find someone else.

------
laichzeit0
Do you really need to test whether a candidate is technically competent if
they have a degree from a well respected university like MIT, etc? You already
know he has the discipline and academic requisites.

Interviewing someone at that point, I'd expect the employer would be more
interested to see if they're a good fit for the company culture and not waste
time on academic trivialities.

~~~
greggman
Well, Laszlo Bock claims Google looked at the data and found zero correlation
between having a degree from a well respected university and being useful at
Google

[http://mobile.nytimes.com/2014/02/23/opinion/sunday/friedman...](http://mobile.nytimes.com/2014/02/23/opinion/sunday/friedman-
how-to-get-a-job-at-google.html)

So no, using a degree isn't useful for hiring engineers apparently

~~~
laichzeit0
That's pretty weird though.

Is using a degree useful for hiring medical doctors, mechanical engineers,
actuaries? It's really hard to believe that only Computer Science as a field
has a problem where it's churning out people who are technically incapable of
programming to such a degree that you need to explicitly test them in
interviews through "puzzles".

~~~
greggman
Maybe they all have that problem and no one has ever bother to actually check
the data?

I know for me and interviewing programmers my personal experience is the
degree doesn't matter. I've met plenty of degreed "engineers" who couldn't
program for shit from schools like MIT and Harvard. I've also met some amazing
programmers from those same places. I don't know what make the difference
except possible love of the activity. Not sure how to asses that in an
interview or whether it correlates better with outcomes.

