
Ask HN: What % of your job interviewees pass FizzBuzz type questions? - tomx
I'm sent 100 resumes a month, then interview about ~10 people. Of those, about 50% can do FizzBuzz style questions. On paper, all of these ~10 people would be qualified to do the work, however (I think) some have become rusty, or focused more on management or other areas.<p>What percentage of your job interview candidates pass FizzBuzz style questions? Any tips to improve the signal to noise ratio?
======
patio11
For folks who doubt the numbers people are quoting here: remember, if half the
pool can't program to save their lives, guess which half of the pool will
still be sending out job applications next week.

At a previous company where, for cultural reasons, lack of programming skill
was not a barrier to being hired as a software engineer, approximately half of
our software engineers could FizzBuzz. Of our outsourced coders, I'd put the
number at one of the twenty I knew, and he would need extensive coaching to
make it happen.

Some of these folks were at least moderately productive at tasks which you and
I do every day which theoretically happen in an IDE but do not require much
abstract thinking, such as changing labels on UI elements, adding new columns
to tables (by copy/pasting a line which worked and tweaking it until output
matched expectations), and the like.

------
pixeloution
Just curious if you expect perfect code on a sheet of paper or give them an
IDE and say go for it.

As I remember fizzbuzz (print numbers 1 thru x, then fizz if divisible by 3,
buzz if divisible by 5, fizzbuzz if divisible by both)

I did it in a text editor in under a minute. Got two errors because I did it
without thinking, fixed it, and had a working solution in 90 seconds. It's
taking me longer to write this response.

I can't imagine anyone who writes code daily who couldn't get this right in
under 5 minutes given a text editor and a way to run the code, but I could
imagine plenty of people who trying to do it on a sheet of paper who would
make goofs. And most of those would make good employees.

~~~
hga
Back in the days when I did this (1990s) we did the C/C++ reverse a linked
list and compute the factorial of a number. And, yeah, people made mistakes on
a dry erase board under interview pressure, but it was very clear who has a
clue vs. who couldn't program their way out of a wet paper bag.

These sorts of tests are low pass filters, they're not designed to find good
or great programmers, just confirm that the person can program _at all_ (as we
define it). Since most people who call themselves programmers really aren't,
they're highly useful. Anyone who's looking for perfection of this nature in
an interview and who fails a candidate for lacking that is doing the latter a
favor by not hiring them.

~~~
dlikhten
Give em a laptop. Those are a dime a dozen (we got a ton sitting around the
office). Have em code in front of you. Hell I'd install any interpreter within
reason to let em solve it.

~~~
wladh
It's not about how expensive a laptop is or isn't. If somebody can't master
the basic syntax of the language so that they can write a very basic algorithm
without syntax errors and in such a way that it would run on the first try,
they can't be very productive. And fizzbuzz is really the rock-bottom of the
simple algorithms. Even a linked-list library would be a very reasonable thing
to ask (if we're talking about C for instance). If somebody has to go through
a few edit/(compile)/debug/change cycles for each simple thing, it doesn't
sound very productive. Now, obviously if we're talking about an application
that involves some baroque frameworks and class hierarchies, things are
different, but simple things should be doable on paper.

~~~
lotharbot
Writing on a whiteboard while explaining yourself is different from typing
alone into an IDE. Conventions you might use to avoid syntax errors (like
typing a pair of braces and then filling the middle) may not work on a
whiteboard. Pacing is different, and muscle memory doesn't help. You're
looking at a blank whiteboard rather than your terminal with other parts of
the codebase. The first few minutes of coding are often "getting up to speed";
many programmers are orders of magnitude more productive once they're "in the
zone".

In short, I wouldn't consider "your whiteboard code is missing a semicolon" to
indicate a lack of productivity overall, only a lack of practice coding on a
whiteboard. (It surprised me how much better my whiteboard code got over the
course of a quarter of teaching.)

~~~
rmc
FizzBuzz is not aimed to find the best, the kind of people who fail fizzbuzz
don't fail it for syntax errors or semicolons. People fail fizzbuzz by staring
blankly at the screen/whiteboard acnd not nothing where to start, you fail
fizzbuzz by not knowing how to do a loop. You fail fizzbuzz by nkt knowing how
to do an if statement, you fail fizzbuzz by not knowing about modulo
arithmetic. Fizzbuzz is designed to get rid of those people.

------
ericb
In the actual interview, I find that most give up after spitting out some
pseudocode. About 40% have reasonable psuedo code that makes me think they'd
get there (but these half-answers don't give me enough confidence to give them
a thumbs up).

Actual code that works (and running it from a terminal), we are seeing only
about 15% tops maybe lower.

We have started sending a fizzbuzz-ish question, a relatively easy css
question, and a word-problem about performance as pre-interview questions
through recruiters. This has dropped our resume inflow dramatically and saved
a lot of time, but that's depressing in a way.

We are looking for a Rails or PHP dev in waltham (near boston) currently
without a lot of luck. The job has a lot of pros, but probably doesn't do
itself justice on-paper.

~~~
dhimes
_The job has a lot of pros, but probably doesn't do itself justice on-paper_

Keep in mind that you may be getting a lot of good people with a lot of pros,
but that don't do themselves justice on paper.

~~~
ericb
Possible, but the painful disappointments have all been the reverse. Good on
paper, but the paper is, to be generous, "embellished."

~~~
dhimes
Sure, but it's really hard to try it the other way: "Well, he looks like a
dipshit on paper, but I've just got a feeling."

I haven't hired for my own company yet, but when I did if for previous
employers I found it quite hard.

------
wavded
I failed FizzBuzz, I admit it. I was asked that as an interview question. I
studied hard for my interview but I forgot about the mod operator and totally
fumbled my way through the interview. I am now the Senior Web Developer and
turned out to be a great asset to the company. But you wouldn't have known
that from my FizzBuzz results. I also didn't have 'Computer Science' as a
Major. Couple strikes. However they took a chance on me.

Since then I've been able to interview others and I look for different things
than FizzBuzz compliance. I want to see how they solve problems in general. I
want to see if they have any passion for what they do. I want to see things
they've developed.

------
elliottcarlson
I have used various interview questions, including FizzBuzz, factorials and
other questions, and I would say the success rate I saw was about 20%. My
personal favorite question is to have someone write a shuffle function without
using any built in randomization functions - however most people give up, and
others can't follow simple directions and wrap their head around the problem.
This particular question had more of a 5 to 10% success rate.

~~~
iron_ball
Do you meant that they can't use Array.shuffle() or Array.sort(<some built-in
random function>)? Or that they don't get _any_ external source of randomness,
even a 0.0 - 1.0 float?

~~~
elliottcarlson
No built in functions such as the ones mentioned, and preferably no quick
entropy generating methods that are obvious but that's not as important as not
using the built in functions. Another requirement is that each run of the
function has to truly be a reshuffle of the array so that it can't be
predictable.

It's harder than it sounds, but I also don't expect a perfect answer -
generally I just want to see how people think out of thee box to try and solve
an odd problem.

------
corin_
Can anyone explain how it is possible for people applying for these jobs to
fail?

I dabble in code, but am no where near the level I would have to be to do
_any_ job in this area, I mean seriously. Took my two minutes to do it with a
pen and paper in PHP, same with in JS, same in bash scripting.

How can anyone who isn't able to do this pretend to even have an interest, yet
alone the ability to do the job?

~~~
hga
Good questions. I'm not sure myself, but I suspect the lure of $$$. Someone
who's good at playing a sort of game I loathe can prey on companies that don't
do good screening and keep a job for as long as a few months and sometimes
indefinitely.

Heck, look at how much cargo cult programming goes on at companies from top to
bottom. PG pointed out that one of the main reasons for dot.bomb failure was
not being able to technically execute (sure, the company may still have been
hopeless, but they didn't even get to test their sales proposition).

And a lot if not most companies simply don't focus on technical excellence or
even simple competance. E.g. look at how the powers that be at Friendser let
it wither on the vine because they assumes the tech was in the bag while their
persued big deals, all the while ignoring for years (I think) that the system
had gotten so overloaded it was essentially useless.

And, heck, if I had the time to mentor you and you were interested, I might
take you on even at your current level. You "know your own limitations" today,
but you're still, say, better than 70% plus or minus of the people who call
themselves programmers. If you can execute the little stuff, I can teach you
the big picture (with the aid of a book or two and some hard work).

------
buro9
I think that the lower the percentage, the more you need to improve your
screening prior to getting the candidate in. It's just wasting your time and
theirs.

I would love to say that you can tell from a CV whether or not they could pass
FizzBuzz, but it's not true. I've interviewed MScs and PhDs that could not do
FizzBuzz. Seriously.

The phone screener is your friend.

------
ajuc
I have question to recruiters:

if job requires "Hibernate" and I've used hibernate in my previous job, but
have never configured it from scratch, only tweaked some models, wrote some
EJBQL queries - does this count as "knowing Hibernate"? I've also never used
Hibernate annotations, becasue we use hbm files, and we have templates to make
the, so I'd have problems writing such file from scratch.

Do you check knowledge of required libraries on the blackboard? Do you assume
people should know all the corners of such libraries, or do knowing some
things and wanting to learn more if it will be needed suffices?

I use at work jboss, hibernate, jbpm, and many other technologies that are
often mentioned in job offers, but I don't feel I can say I know them - only
the parts that I needed to do the job. Is this considered not enough?

~~~
ig1
Generally speaking you don't need to match a job description 100%, if you
match 60-70% that's fine. Depending on the job description you can tell which
things are key to the job (programming language, and specialist skills like
machine learnings which are core to the job) but everything else is mostly
optional.

The more of the description you can match the better it is, but 60-70% will
get you an interview in most places.

------
cantastoria
I'm curious what candidates are getting stuck on. Is it the testing for
divisibility that's tripping them up?

~~~
DrJokepu
Mostly they just don't know how to start. They can't break down the problem
into subproblems. Once a candidate managed to break down the problem to
subproblems, they can always do the rest easily (solving the subproblems,
composing the solutions to solve the main problem).

~~~
cantastoria
Ok in that case then they're a no hire as that's just a fundamental part of
developing software. Although I must admit that this problem surprises me more
than people's inability to code. I mean if you can't break down problems
you're just ill suited to the field.

------
DrJokepu
In my experience, about one in five (20%) people I get to interview can solve
basic programming problems on a whiteboard (in their language of choice). It's
really depressing.

------
ColinWright
We now insist on resumes being accompanied by the answers to set "homework."
No answers, no interview.

We get about half our interviewees unable to solve problems that are similar
or easier during interview. Whether that's nerves/stress or simply an
indication that they got someone else to do the homework for them we don't
know.

One candidate even phoned a friend during the coding part of the interview to
get some answers. For some jobs he'd be hired, but not for most.

------
aplusbi
I've never asked FizzBuzz but I do regularly ask coding/algorithm questions.
They are usually more difficult than FizzBuzz. I'd say around 80-90% can at
least come up with _a_ solution in 45 minutes with some help. Probably less
than 10% can come up with a good solution entirely on their own.

I attribute this to two things, first I think our phone screenings work well
enough to keep out people who really can't do FizzBuzz, and second that I'm
fairly generous during interviews. I often don't expect real code, sometimes
I'm satisfied with just a discussion of the algorithm (no white board coding
at all). I don't expect code to compile and I even let candidates use
undefined "helper" functions (although I usually only allow that if I get the
feeling that they could implement them if asked).

* For those that are curious I have two favorite questions - print out all the permutations of a given (ASCII) string and describe a search algorithm for a sorted array that has been split in two and the two pieces have been swapped (i.e. - 4,5,6,7,8,1,2,3).

~~~
pp13
@aplusbi

Let's say I interviewed you. I had asked to implement the fastest algorithm to
give back the largest palindrome of words in English dictionary and compare it
the largest palindrome of the french language. What the best solution you can
come up with in 45 minutes.

After, that I asked you, write a simple ftp server, in the language of your
choice. With your first solution, I asked you implement a SSL library and add
it to the ftp server to make it sftp.

Based on that I can judge how good of a programmer you really are.

Sorry for being sarcastic, but I think most interviewers are on a power trip.
They ask questions that if they heard for the first time, they couldn't come
up with answers either.

I think your better off really talking about and going in depth with the
programmers experience. If your experienced yourself, you should have no
problem.

~~~
eropple
His first question is very straightforward if you've ever encountered
permutations in a math class (and you should have, if you're a programmer).
Even if you haven't, it shouldn't be that difficult.

His second is trickier, but solvable. And, no, I've never seen that
(particular) question before.

~~~
pp13
My point was, his questions, doesn't really indicate if the candidate is a
good programmer or not.

It could be the person studied up on all sorts of puzzles and famous
algorithms online but hasn't really written or programmed anything.

Not all jobs require that much expertise. You could be doing some really
simple programming work.

I think most interviewers ask these type of questions 1.) make themselves feel
smart 2.) they get a kick out it

~~~
pp13
@eropple

So do you have data to back up your claims that; answering those questions
determine if your a good programmer.

Yes, there are tons of programming jobs out there, that don't require much of
those skills. Not all jobs are that innovative.

Most of the development jobs I have seen require you to come up to speed with
the code base fast. So it's that code reading comprehension that I think is
most prevalent.

~~~
eropple
I did not in any way, shape, or form claim that such questions determine
whether or not a candidate is a good programmer. If you are going to respond
to my posts, please respond to my posts and not what you think or wish I said
in those posts.

No interview or interview question can determine conclusively that you are a
good programmer. But they reduce the likelihood that you're not, and comments
like your strident and hysterical analogy drawn between these really very
simple problems and "write a SSL library!" are doing little to change my mind.

~~~
pp13
Ok, I didn't mean to offend you, I am sorry if I have.

That being said these are my thoughts on the subject of interviewing
programmers.

When I just graduated college, I was all about puzzles and programming
puzzles. My thoughts were that only people who could do these puzzles deserved
programming jobs.

I was so wrong. Puzzles are something that you get or don't get. If your in an
interview, and you haven't faced those combinatorial problems for a while, you
could bomb, if someone asked you those questions.

Understood "fizzbuzz" problems are suppose to be the the lowest level of
programming problems that any competent programmer can solve. If you want to
fast screen candidates, go ahead and ask them.

When you get into problems, that require "tricks", either you get it or don't.
It could be easy for some people and hard for others. There is no correlation
of them getting it, to being good on the job, unless the problems are such
that, the programmer will face them every day on the job.

Another good permutation problem, is subset sum problem. Come up with a
program that determines if a array or list, can be partitioned into 2 non
overlapping subsets, such that the the sum of both subsets equals each other.
Print out both subsets if they are equal or state that it cannot be done. Is
it possible to get better than O(n^2) for the optimal solution. That is fun
one, but I don't think asking that would give you much insight into
determining whether the programmer is fit or not.

I think interviewing is a really important task. It a really hard job to get
right. If your a bad interviewer, the company you work for will eventually get
a bad reputation.

Many software companies do not train people how to interview. That in our
industry, is a huge problem.

There are many other things besides raw technical skills that will determine,
how well the candidate will do at their job and at at the company.

Going over the candidate's experience in detail. Getting a feel for what they
really want and expect from the job. Do you like the candidate's personality?
What does your gut instinct say. What technical things do you need to ask that
are relevant for that job, that you couldn't determine from the their
experience? How would you design questions, relevant to the job at hand.

Just my random thoughts.

~~~
Someone
"Another good permutation problem, is subset sum problem. [...] Is it possible
to get better than O(n^2) for the optimal solution."

AFAIK subset sum is NP complete, so I guess you meant O(2^n). Even so, I would
not expect a programmer to tell me much about it.

------
spamizbad
4 candidates, 2 passed FizzBuzz. Oddly, the two that failed had a Masters in
CS (albeit no BS in CS).

Of those that passed: one had Masters in Library Science looking to change
careers. The other was a fresh out of college CS major from Illinois State.

Next batch, I think we'll add another trivial question: count the number of
vowels (a, e, i, o, u) in a string.

~~~
pixeloution

      <?php
    
      $string = "aeiouxabcd";
      $checkfor = array('a','e','i','o','u');
      $count = 0;
    
      for($x = 0; $x < strlen($string); $x++) {
        if(in_array($string[$x], $checkfor)) {
      	 $count++;
        }
      }
    
      echo $count . "\n";
    

Am I hired? Or do I lose cool points for PHP?

~~~
patio11
Every time FizzBuzz problems come up among engineers, people race to solve
them and post their answers, then compete to see who can write increasingly
more nifty answers for a question which does not seek niftiness at all.

I'm all for intellectual gamesmanship, but these are our professional
equivalent of a doctor being asked to identify the difference between blood
and water. You can do it. _We know_. Demonstrating that you can do it is not
the point of the exercise. We do it to have a cheap-to-administer test to
exclude people-who-cannot-actually-program-despite-previous-job-titles from
the expensive portions of the hiring process.

------
wpeterson
The percentage of people who can complete a basic coding challenge during
interview is more a testament to your phone screening than to the population
of candidates.

If you're not weeding out these people with a 10-15 minute phone call you'll
waste a lot of your and their time.

------
albedoa
This post by Jeff Atwood might be of interest to you if you haven't read it:

[http://www.codinghorror.com/blog/2007/02/why-cant-
programmer...](http://www.codinghorror.com/blog/2007/02/why-cant-programmers-
program.html)

~~~
tomwalsham
The fascinating thing about that post is the number of sneering comments, with
their own (broken) implementations of FizzBuzz. At a glance only maybe 20-30%
of solutions in the comments there actually work, and many are horribly
suboptimal.

------
chollida1
I've actually asked this question as a gentle warm up for people, and we
achieve around a 90% success rate.

On average it takes people around 5 minutes to do.

We have people do it on a whiteboard to get them standing up and moving.

------
petercooper
_Of those, about 50% can do FizzBuzz style questions. On paper, all of these
~10 people would be qualified to do the work_

If it's for a programming position, the "paper" is lying in this case. If
someone is claiming to have experience or qualification in certain areas that
they _can't even perform basic operations in_ , they're fraudsters.

It'd like trying to hire a surgeon and have someone turn up who doesn't even
know what lymph nodes are. Dangerous and unhireable, but sadly a lot of
employers put up with this sort of nonsense.

------
eftpotrm
That sort of thing specifically, never actually tested it.

The last test I did help administer was for a VB+SQL job, and the first
question was to write an example of a valid INNER JOIN. I'd say at maximum 25%
of the candidates could do this.

Improving SNR? I did once have a potential employer get me to do a time-
limited online test. If you wanted you could always stick your questions into
one of them, so you can at least do the fizzbuzz-level screening without
calling them in and sitting them down.

~~~
tomjen3
I think the reason you got so bad responses on the first question wasn't that
they couldn't write the join statement but that they had heard the distinction
between the different joins once in college and then never considered them
again.

I bet that if you had asked for an example of a sql code which would list all
the employees born before 1980 along with the department they worked for and
the name of the head of that department, you would have gotten a much more
useful result out of that.

That query is, incidentally, much more difficult to write.

~~~
eftpotrm
But with a normalised database you couldn't the above without two inner joins
(and it's still a trivial query that any competent dev for that sort of
position should be able to dictate while driving, it's that easy).

Really, if you can't remember the difference between inner, outer, full and
cross joins then you shouldn't be working with databases. Which was rather
what the test showed us, frankly.

------
lrm242
How are you asking the questions? Good interviewers try to adapt to the person
they are interviewing. Depending on how you're asking the programming
question, you might want to think about changing it. If, for example, you sit
back in your chair and ask someone to go to a whiteboard to write code you
should consider that some people simply are not going to respond to that sort
of method of answering, even if they are a brilliant programmer.

------
minikomi
Wow.. This is pretty astonishing. I know this is a majorly skewed, opinionated
audience, but do people really not have that much of an interest in what they
do?

------
schulz
I did a ton of interviewing a few years ago ~ 100 interviews.

I found about 10% nailed it right away with code that would compile and run.
These were generally people who had been coding a lot recently.

Half of the rest (say 45% of total) got close: Minor syntax errors, logic
errors, stuff that an IDE/non interview situation would have fixed.

45% just spaced. Couldn't right the for loops, conditionals. Couldn't write
basic code.

------
davewasthere
Of people who get to the interview stage, I'd say around %75 at least
successfully do the FizzBuzz question on our test.

But we've cherry picked the CVs a little. And probably only interviewed 50
people over the past couple of years. (And hired 5)

------
sungura
I program a fair bit, have a few Android Apps in the market the problem I see
with these type of tests is that I have a very hard time remembering syntax
and would therefore have a hard time without an IDE.

------
acangiano
I mostly interview students for hands-on internships. I'd say around 10-20%.

------
mikereedell
While we don't ask FizzBuzz in particular we have a question my manager asks:
If you could fold a piece of paper 50 times, how tall would it be (very rough
ballpark figure)?

Our success rate is surprisingly low.

~~~
cantastoria
This is a pretty abrupt way to start off an interview. And I don't understand
the point of this question w.r.t assessing someone's coding ability. Though
I'd be curious to see if the success rate on that question and on a fizz buzz
correlate.

~~~
mikereedell
It's generally asked at the end of the interview.

------
gabyar
About 80%. Perhaps we're more vigorous with resume screening. Over the last 4
years of interviewing, I've become very good at resume screening.

------
mattdeboard
I wish we would run at least some kind of sample problem before hiring people.
_sigh_

~~~
hga
Indeed; if you interview at a company that _doesn't_ to this sort of thing if
you have any choice you're well advised to avoid them. There's simply going to
be little joy working with people who can't program and can either stick
around or who have to get fired. And things can get ugly for those who do have
a clue, from envy to having to do others work to avoid project or company
failure and losing your job.

~~~
mattdeboard
> There's simply going to be little joy working with people who can't program

Well, it does make for exciting bitch sessions in IRC.

------
EponymousCoward
10-20% (pathetic really)

------
hga
The last time I was a part of this activity in a company (1997-8), using
similarly rigorous pre-screening, we had about the same results, only 50% of
the people we interviewed could program their way out of an (EDIT) wet paper
bag.

~~~
kaybe
How would I go about programming my way out of a paper bag? It creates the
weirdest ideas and images in my mind.

~~~
hga
Oops, although this won't help your mind ^_^, the correct expression is "...
out of a wet paper bag". From more apropos usage about fighting or escaping
from such a flimsy thing.

And now that you mention it, I have no idea either ^_^ ... except that it
ought to be easy. In that last company where I was part of the hiring process
I myself got the job only because I was the _only_ one they interviewed who
could pass their dry erase board tests (at the time they didn't have good pre-
screening or access to a good candidate pool; I myself answered a want-ad).
Tests that were to me so simple I barely remembered them (two decades of
experience by then).

~~~
kaybe
Maybe if there was a robot to program, I guess that would count.. now that
would be a great opening move on part of the company! :D Let's see.. you'd
have to get the robot into the right position (maybe find it with sensors -
what kind of wet paper bag are we talking about, the dark kind?) and control
another part that's capable of ripping the bag. Depending on the size of the
robot you'll have to move it around a lot to make a hole big enough for
yourself. Of course, you can also have the applicant design a robot for that
task.. uhm, I'll stop now.

~~~
hga
Well, you know, MIT's introductory EECS course is now Python programming of
robots, so as soon as those students start graduating ^_^....

Thanks for calling me on this rather mixed metaphor I've been using for a long
time in a most amusing way. You're brightened my end of the week.

