
Fizzbuzz isn't a coding test - kapilkale
http://www.kapilkale.com/blog/fizzbuzz/
======
AngryParsley
_Ask some smart business analysts with no coding experience to solve FizzBuzz
in Excel. All should be able to do it._

Instead of asserting this fact, put the hypothesis to the test. Ask some smart
business analysts and see how many of them can solve fizz-buzz. You're much
more likely to win over the audience if you have some data.

Really though, I think this post is needlessly controversial. The whole text
could have been, "It makes sense to hire people with no programming experience
who can solve fizz-buzz." I don't know how many people fall into that
category. I'm not sure how good they'd be or how long they'd take to get up to
speed, but that's fine. It's something I can think about and engage with.

The rest of the post is just arguing over definitions. If by "coding" you only
mean "turning pseudocode into code", then yes a lot of things people call
coding aren't coding. Likewise if by "think analytically" you mean "is able to
solve fizz-buzz," then of _course_ companies want programmers who can "think
analytically."

No matter what definitions you use, the hiring practices are the same. Fizz-
buzz still acts as a negative filter. People who can't solve it aren't hired.
People who can solve it may or may not be hired depending on how other things
go.

------
SomeCallMeTim
I think that FizzBuzz is just filtering out incompetent applicants who
shouldn't be trying for programming jobs to begin with. Joel on Software
described this phenomenon back in 2005. [1] His key takeaway is that 99%+ of
people applying for developer jobs may be unqualified for pretty much any
coding job, but they keep applying anyway.

He describes how he and other companies get 200 applicants for most job
postings, of which only one may actually be "good enough." I've seen numbers
like that myself when on the hiring end. So everyone thinks they're hiring the
top 0.5%, even though the same 199 losers are spamming their resumes to every
job posting.

Well, a simple way to filter out that 199 is to apply FizzBuzz or equivalent.

[1] <http://www.joelonsoftware.com/items/2005/01/27.html>

~~~
halbermensch
_So everyone thinks they're hiring the top 0.5%, even though the same 199
losers are spamming their resumes to every job posting._

Yeah, when you think about it, this whole "we only hire the top 0.5%"
condescension really amounts to a basic statistical reasoning fallacy.

To construct an alternate hypothesis: the "real" threshold for ranked
competence[1] at "elite" companies might, just _might_ , be more like "the top
5%". But we can safely assume that 95% of the available talent pool in this
category will be employed (and not actively seeking), compared with say 70% in
the bottom, or "sub-elite" category, at any given time.[2]

So for example in a "town" of 1000 putative developers, with skill levels
ranging from god-like to "good enough" to mediocre to utterly bogus and self-
deluding, let's say 50 (or 5%) of these are truly "elite", and 950 are "non-
elite".

Further, of the elite category, 47.5 will be employed (and not actively
looking), and only 2.5 will be in the job market -- versus, out of the 950
non-elite developers, we might have 665 (or 70%) employed, versus 285 (or 30%)
on the market and actively looking.

Given this over-simplified (but arguably "less wrong") hypothesis, and
assuming that a given opening draws applicants whose fitness ranking is evenly
distributed across the pool of _in-the-market_ candidates, it's easy to see
that for any given opening, we'll have a ratio of 2.5:285 for elite:non-elite
developers -- or a paltry _0.869%_ (!) of the candidate pool (2.5/287.5) will
appear to be from the "elite" category.

That is, for every 1 "elite" resume in your inbox, you'll be seeing 114 "non-
elite" resumes. And since tech geeks are, by and large, an insecure, envious
lot, they like to brag and fudge a bit -- so that 114 easily becomes 199 in
their minds (or the 1 a 0.6, whichever).

Which gives you the infamous "we only hire the top 0.5%" urban legend right
there. Even though they're all really just angling over the... top 5% (ranked
over the general pool of putative developers).

And since hot-shots like Joel Spolsky, not to mention nearly all of the
investment bank / hedge fund types, and well, probably half or more of
everyone in the monthly "Who's hiring?" thread just keeps repeating it, and
repeating it, and repeating it... well, shucks, it _must_ be true.

[1] As such a thing as generic competence exists can be objectively measured,
etc, etc, but that's a separate issue.

[2] Which corresponds -- conservatively, for the purposes of this argument --
to another widely espoused, but never quantitatively substantiated belief out
in startupland: that "all" of the good developers are "always" employed and
"almost never" on the job market longer than 2 days.

~~~
vampirechicken
...and then you'll offer a salary that has been painstakingly researched and
calculated to be as close to median as you can get it, without going over.

~~~
halbermensch
...or a notch under, for good measure.

------
lopatin
Who cares the about what FizzBuzz REALLY is? Can't we all just agree that it's
a programming test because anybody who calls them self a programmer should
possess these painfully simple analytic skills. If you can translate
pseudocode to code, but can't write pseudocode for FizzBuzz, you're a
translator not a programmer.

~~~
jlgreco
Exactly. Who cares _what_ it is, it remains useful for weeding out the people
who think they can get away with lying about being a programmer.

------
techdmn
I've been doing FizzBuzz and a few other code-related tests for the last few
years. It was an odd shock when I realized that was the /only/ part of the
interview process that had any value to me whatsoever. I mean it. I have seen
people display radically different personality traits between the interview
and work on Monday. The guy who was almost too nervous to speak in the
interview won't stop talking peoples' ears off. The guy who really sold
himself well is very shy and quiet now that he has to show up every day. I
wouldn't believe if it I hadn't seen it. Not to mention language lawyers, etc.
But coding tests have predictive value. YMMV of course.

I'm also surprised how many people get it wrong. I see a lot of this
(pseudocode):

    
    
      for (i = 0; i < 21; i++) {
        if (i % 3 == 0) {
          print "fizz";
        } else if (i % 5 == 0) {
          print "buzz";
        } else if (i % 3 == 0 && i % 5 == 0) {
          print "fizzbuzz";
        }
      }
    

Yeah, see, one of those branches is never going to be reached.

~~~
potatolicious
Consider yourself fortunate, I've screened candidates who didn't get past the
"for" keyword because they couldn't construct a loop in a language of _their
own_ choosing, even after I reminded them that perfect syntax is not
necessary.

I've had a _surprising_ number of candidates who did not know about the
existence of the modulo operator. Not just "I completely forgot about it", but
more like "what's the % sign mean?".

~~~
swivelmaster
I once had a candidate claiming to be a senior PHP programmer solve fizzbuzz
by doing something like this: if (strpos($i/3, ".") != null) or something like
that. I was sad.

~~~
Someone
I googled to see whether this was creativeness or just copying stuff from the
net, and came across
[http://www.php.net/manual/en/language.operators.arithmetic.p...](http://www.php.net/manual/en/language.operators.arithmetic.php).
It states:

 _"Operands of modulus are converted to integers (by stripping the decimal
part) before processing."_

=> 4.5 % 1.5 is zero, as is 4.6 % 1.9. So, in the world of PHP, there may be
logic in that strpos approach. As far as I can tell, it certainly isn't much
worse than
[http://www.webdeveloper.com/forum/showthread.php?76114-Work-...](http://www.webdeveloper.com/forum/showthread.php?76114-Work-
out-if-int-is-a-multiple-of-3) advocates.

And no that is not worth much. I know almost nothin about PHP, and prefer to
keep things that way.

------
danielamitay
> "Write a program that prints the numbers from 1 to 100. But for multiples of
> three print “Fizz” instead of the number and for the multiples of five print
> “Buzz”. For numbers which are multiples of both three and five print
> “FizzBuzz”

Aside from explicitly stating "Hey, encapsulate IF/ELIF/ELSE statements in a
FOR loop", FizzBuzz doesn't seem too analytical--am I wrong?

~~~
rogerbinns
I've been interviewing a bit recently and not a single candidate has actually
managed to complete it successfully (I use a slightly simpler variation with
different numbers and strings). They have all been juniour but all claimed at
least a few years of programming experience (counting university/school).
Thinking I was going nuts, I tried it out on my colleagues including one whose
total programming experience was trying to learn Python a few months ago, and
they finished 100% correctly in under a minute.

I don't ask for correct syntax, an actual programming language, fret over the
100 being included or not etc. Heck I even ask them what the output would be
for the "code" they just wrote and they usually get that wrong too! For a for
loop and 3 arms of conditionals.

I suspect it is a standin for experience. You can get pretty far copying and
pasting sample code you find on the net, maybe altering a number or two. (Or
the far more telling commenting blocks of code out.) These days you can have
something that appears to work by using project creators and tinkering, but
not actually doing a reasonable amount of what could be considered older
school programming.

~~~
kabdib
I ask people to write strlen() for me. (If they don't know C, it's a
showstopper, btw).

Supposedly senior candidates with ten years of experience have failed this.
It's truly amazing. "Oh, I haven't written code in a while" is a common
response. That's a show-stopper, too -- what, you want to /manage/ me or
something? And you can't write code? God forbid.

------
smosher
I see a frightening number of people buying into the disturbing assertions
made in TFA:

\- that analytical thinking isn't necessary for programming.

\- that translating pseudocode is a sufficient test of programming skill.

\- that Excel isn't equivalent to a programming environment, or somehow
doesn't count as programming.

\- that FizzBuzz requires _higher-order_ thinking. It's not like you're asking
someone to implement lists as lambdas. Or the this website's namesake (the Y
Combinator.)

Each one of these items is a big fat WTF.

------
spaghetti
I don't think engineers should ever spend their time asking Fizzbuzz.
Engineers' time is valuable. And blocking off an hour for a phone screen can
interrupt the day. If a preliminary code sample filtering step is used then
engineers can jump straight into the meaty interview questions.

The vast majority of people who don't know programming basics like loops and
conditionals won't make it to the phone screen stage. Someone might look up
the solution to the programming quiz but they'll immediately fail the first
interview question. And this can be partially mitigated by rotating puzzles or
even changing numbers in the puzzles.

The code sample filtering step should have both a sufficiently tricky puzzle
and a bank of at least 20 common corner cases. Just run the test cases
automatically. The vast majority of candidates will be eliminated without
wasting any engineers' time.

~~~
swivelmaster
IMO phone screen (if it really is a "screen" and not a full "interview")
should be fifteen minutes or less. I usually ask a few very specifically CS101
type questions to see if they remember any of it, and if they don't, they
don't come in for an interview.

~~~
spaghetti
I agree about the phone screen length. Whether you call it an interview or a
screen it can probably be much shorter than hour with almost all of the
benefits.

------
alexlande
I've got to disagree here. The whole point of FizzBuzz is that it's a
straight-forward analytical problem that you should be able to do easily if
you understand the fundamentals.

------
antiterra
I think some part of this issue has to be that computer science does not fully
equate to coding, and the balance of coding to theory curriculum varies
significantly. If you're hiring recent grads then it might make sense to give
a little leeway, to help them along and see if you see some spark of analytic
ability. Maybe interview nerves are an issue.

If your candidate can't do fizzbuzz but is claiming to be an experienced
programmer in the required language, the situation should raise a red flag.

------
michael_fine
It's not coding, in the strictest sense that it has no computer science or is
solely syntactic, but it works at weeding out people who can't __build
anything __. At least this level of analytical thinking is necessary for any
programming task, and therefore Fizzbuzz serves its purpose.

------
danielharan
>"Ask some smart business analysts with no coding experience to solve FizzBuzz
in Excel. All should be able to do it."

Bahaha. I want to see you try. And weep.

------
pjungwir
How many people have actually been asked Fizzbuzz in an interview? I recently
interviewed at several places, before deciding I'd rather try my hand at
independent consulting. At least two were companies with (apparently) superb
engineering teams. But no one asked me this question, or any preliminary
coding question at all. I was asked to code, but only after the companies had
paid money to fly me out, put me up in a hotel, and have their team spend a
day talking with me. I would be _delighted_ to be asked this question, because
I'd take it as a sign that the company values its time and is discriminating
in its hiring.

------
tjr226
FizzBuzz isn't a coding test, I agree. It's an interview prep test. If a
candidate can't take 15 minutes to read up on common interview questions and
prepare a simple answer, then they shouldn't be hired.

~~~
btilly
If you have to do interview prep to solve FizzBuzz, then you have no business
calling yourself a programmer.

------
gregjor
It is a coding test, and here are the answers:

<http://www.geekschool.org/programming/fizzbuzz/>

FizzBuzz in lots of languages, including LOLCode.

------
fleitz
Higher order thinking is pretty much required to make any working program of
minimal complexity. aka. be a programmer.

If you have to hand a programmer pseudo code in order to write a program,
you'd be better off hiring competent programmers to write you a compiler and
then you can get rid of the programmers who can't think and code it yourself
in your fancy new higher order language.

~~~
mpdaugherty
This is definitely the relevant point when it comes to hiring programmers.
FizzBuzz is already so close to pseudocode that if you had to write it more
clearly, it wouldn't take any more time to just write the code yourself. A
programmer who can only work at that level is not a benefit to your team.

You really want people who will improve your team's output by an amount
greater than the value of the extra time it takes to communicate with them.

FizzBuzz can be an effective filter to very quickly get rid of the lowest
level of applicants and start having a real discussion with the ones who are
left.

------
recursive
If fizz buzz is too complicated, what would be an example of a problem a
programmer should be able to solve? Why do programmers exist?

------
ChuckMcM
There is analysis and there is coding. I've met and worked with people who had
both skills (awesome) and only one of the two (less awesome). If you're
limited in the number of people you can hire (startup) then you want both
skills in the same person. I think any business analyst would agree with that.

------
kuahyeow
How useful is translating "pseudocode into a working program"? I doubt the
utility of that skill as a programmer

~~~
gregjor
Starting from pseudocode is a big leg up. In most situations the programmer
won't have pseudocode unless he/she writes it. Users don't describe what they
want clearly and unambiguously most of the time -- that's the other big skill
programmers need. The first skill is being able to write FizzBuzz or something
equally trivial in a couple of minutes.

If a so-called programmer or coder can't write FizzBuzz from pseudocode then
presumably they won't be able to write it from a prose description, which
means they are going to be close to useless solving any real problem.

------
wheelerwj
seeing this makes me cry. I can code this is at least three languages (java,
php, python) and I can't even get an interview for a junior dev position. I
got bored the other day and tried to write it recursively because why not,
right?

------
icefox
Spelling mistake: bale => able

~~~
kapilkale
Thanks, pushing fix.

------
tokenadult
It is, of course, up to each manager to decide what a person is being hired to
do. If you want to hire someone to do something useful for you at your
company, it's a very good idea to give the job candidate a work-sample test.
For some kinds of programming jobs, a fizzbuzz coding test is a (very easy)
work-sample test. I've heard a story similar to those in other comments here
about 100 applicants to a company that uses Web back-end programming to serve
up online, adaptive lessons. The test the hiring supervisor used was "Write a
program using a for loop to print the integers from 1 to 100." No language was
specified. He has not yet had ANY applicant, even though all his applicants
have computer science degrees, many having master's degrees, submit a working
program to do that. Not one.

This is discussed so often on Hacker News I have a FAQ on the issue of company
hiring procedures. There are many discussions here on HN about company hiring
procedures. From participants in earlier discussions I have learned about many
useful references on the subject, which I have gathered here in a FAQ file.
The review article by Frank L. Schmidt and John E. Hunter, "The Validity and
Utility of Selection Models in Personnel Psychology: Practical and Theoretical
Implications of 85 Years of Research Findings," Psychological Bulletin, Vol.
124, No. 2, 262-274

[http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...](http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%20Validity%20and%20Utility%20Psychological%20Bulletin.pdf)

sums up, current to 1998, a meta-analysis of much of the HUGE peer-reviewed
professional literature on the industrial and organizational psychology
devoted to business hiring procedures. There are many kinds of hiring
criteria, such as in-person interviews, telephone interviews, resume reviews
for job experience, checks for academic credentials, personality tests, and so
on. There is much published study research on how job applicants perform after
they are hired in a wide variety of occupations.

[http://www.siop.org/workplace/employment%20testing/testtypes...](http://www.siop.org/workplace/employment%20testing/testtypes.aspx)

EXECUTIVE SUMMARY: If you are hiring for any kind of job in the United States,
prefer a work-sample test as your hiring procedure. If you are hiring in most
other parts of the world, use a work-sample test in combination with a general
mental ability test.

The overall summary of the industrial psychology research in reliable
secondary sources is that two kinds of job screening procedures work
reasonably well. One is a general mental ability (GMA) test (an IQ-like test,
such as the Wonderlic personnel screening test). Another is a work-sample
test, where the applicant does an actual task or group of tasks like what the
applicant will do on the job if hired. (But the calculated validity of each of
the two best kinds of procedures, standing alone, is only 0.54 for work sample
tests and 0.51 for general mental ability tests.) Each of these kinds of tests
has about the same validity in screening applicants for jobs, with the general
mental ability test better predicting success for applicants who will be
trained into a new job. Neither is perfect (both miss some good performers on
the job, and select some bad performers on the job), but both are better than
any other single-factor hiring procedure that has been tested in rigorous
research, across a wide variety of occupations. So if you are hiring for your
company, it's a good idea to think about how to build a work-sample test into
all of your hiring processes.

Because of a Supreme Court decision in the United States (the decision does
not apply in other countries, which have different statutes about employment),
it is legally risky to give job applicants general mental ability tests such
as a straight-up IQ test (as was commonplace in my parents' generation) as a
routine part of hiring procedures. The Griggs v. Duke Power, 401 U.S. 424
(1971) case

[http://scholar.google.com/scholar_case?case=8655598674229196...](http://scholar.google.com/scholar_case?case=8655598674229196978&q=Griggs+Duke+Power&hl=en&as_sdt=2,24)

interpreted a federal statute about employment discrimination and held that a
general intelligence test used in hiring that could have a "disparate impact"
on applicants of some protected classes must "bear a demonstrable relationship
to successful performance of the jobs for which it was used." In other words,
a company that wants to use a test like the Wonderlic, or like the SAT, or
like the current WAIS or Stanford-Binet IQ tests, in a hiring procedure had
best conduct a specific validation study of the test related to performance on
the job in question. Some companies do the validation study, and use IQ-like
tests in hiring. Other companies use IQ-like tests in hiring and hope that no
one sues (which is not what I would advise any company). Note that a brain-
teaser-type test used in a hiring procedure could be challenged as illegal if
it can be shown to have disparate impact on some job applicants. A company
defending a brain-teaser test for hiring would have to defend it by showing it
is supported by a validation study demonstrating that the test is related to
successful performance on the job. Such validation studies can be quite
expensive. (Companies outside the United States are regulated by different
laws. One other big difference between the United States and other countries
is the relative ease with which workers may be fired in the United States,
allowing companies to correct hiring mistakes by terminating the employment of
the workers they hired mistakenly. The more legal protections a worker has
from being fired, the more reluctant companies will be about hiring in the
first place.)

The social background to the legal environment in the United States is
explained in many books about hiring procedures

[http://books.google.com/books?hl=en&lr=&id=SRv-
GZkw6...](http://books.google.com/books?hl=en&lr=&id=SRv-
GZkw6TEC&oi=fnd&pg=PA271&dq=Validity+and+Utility+of+Selection+Models+in+Personnel+Psychology&ots=iCXkgXrlOV&sig=ctblj9SW2Dth7TceaFSNIdVMoEw#v=onepage&q=Validity%20and%20Utility%20of%20Selection%20Models%20in%20Personnel%20Psychology&f=false)

[http://books.google.com/books?hl=en&lr=&id=SRv-
GZkw6...](http://books.google.com/books?hl=en&lr=&id=SRv-
GZkw6TEC&oi=fnd&pg=PA95&dq=Validity+and+Utility+of+Selection+Models+in+Personnel+Psychology&ots=iCXkgXrnMW&sig=LKLi-
deKtnP20VYZo9x0jfvqzLI#v=onepage&q=Validity%20and%20Utility%20of%20Selection%20Models%20in%20Personnel%20Psychology&f=false)

Some of the social background appears to be changing in the most recent few
decades, with the prospect for further changes.

<http://intl-pss.sagepub.com/content/17/10/913.full>

[http://www.economics.harvard.edu/faculty/fryer/files/Fryer_R...](http://www.economics.harvard.edu/faculty/fryer/files/Fryer_Racial_Inequality.pdf)

[http://books.google.com/books?hl=en&lr=&id=frfUB3GWl...](http://books.google.com/books?hl=en&lr=&id=frfUB3GWlMYC&oi=fnd&pg=PA9&dq=Validity+and+Utility+of+Selection+Models+in+Personnel+Psychology+%22predictive+validity%22+Duke+Power&ots=5O9Hx_E1vY&sig=g-zERWztBWq3h4guEuv9VVkTh8I#v=onepage&q=Validity%20and%20Utility%20of%20Selection%20Models%20in%20Personnel%20Psychology%20%22predictive%20validity%22%20Duke%20Power&f=false)

Previous discussion on HN pointed out that the Schmidt & Hunter (1998) article
showed that multi-factor procedures work better than single-factor procedures,
a summary of that article we can find in the current professional literature,
for example "Reasons for being selective when choosing personnel selection
procedures" (2010) by Cornelius J. König, Ute-Christine Klehe, Matthias
Berchtold, and Martin Kleinmann:

"Choosing personnel selection procedures could be so simple: Grab your copy of
Schmidt and Hunter (1998) and read their Table 1 (again). This should remind
you to use a general mental ability (GMA) test in combination with an
integrity test, a structured interview, a work sample test, and/or a
conscientiousness measure."

[http://geb.uni-
giessen.de/geb/volltexte/2012/8532/pdf/prepri...](http://geb.uni-
giessen.de/geb/volltexte/2012/8532/pdf/preprint_j.1468_2389.2010.00485.x.pdf)

But the 2010 article notes, looking at actual practice of companies around the
world, "However, this idea does not seem to capture what is actually happening
in organizations, as practitioners worldwide often use procedures with low
predictive validity and regularly ignore procedures that are more valid (e.g.,
Di Milia, 2004; Lievens & De Paepe, 2004; Ryan, McFarland, Baron, & Page,
1999; Scholarios & Lockyer, 1999; Schuler, Hell, Trapmann, Schaar, & Boramir,
2007; Taylor, Keelty, & McDonnell, 2002). For example, the highly valid work
sample tests are hardly used in the US, and the potentially rather useless
procedure of graphology (Dean, 1992; Neter & Ben-Shakhar, 1989) is applied
somewhere between occasionally and often in France (Ryan et al., 1999). In
Germany, the use of GMA tests is reported to be low and to be decreasing
(i.e., only 30% of the companies surveyed by Schuler et al., 2007, now use
them)."

Integrity tests have limited validity standing alone, but appear to have
significant incremental validity when added to a general mental ability test
or work-sample test.

<http://en.wikipedia.org/wiki/Employment_integrity_testing>

[http://apps.opm.gov/ADT/Content.aspx?page=3-06&JScript=1](http://apps.opm.gov/ADT/Content.aspx?page=3-06&JScript=1)

<http://www.princeton.edu/~ota/disk2/1990/9042/9042.PDF>

[http://www.hotelschool.cornell.edu/research/chr/pubs/reports...](http://www.hotelschool.cornell.edu/research/chr/pubs/reports/abstract-14602.html)

To sum up, if you want someone who has some minimal level of programming
ability to work for you, you have to TEST that ability among all applicants
for your position.

~~~
JoeAltmaier
Great literature summary. The apocryphal story however - I call BS. Everybody
can write a loop from 1 to 100.

~~~
greyfade
You'd be surprised. There are quite a few "programmers" who can't. I've met a
few. Hell, I've even heard someone claim years experience and they couldn't
write a simple loop.

~~~
JoeAltmaier
The apocryphal story was, this interviewer had never met any programmer who
could write a loop. Obvious bs.

------
tisme
If I were presented with the 'fizzbuzz' test on a job interview I'd walk out.

I would really hate to work in a place that had such a ridiculously low bar to
entry that they would consider that a valid test.

~~~
potatolicious
You misunderstand the demographic of people who apply to programming jobs.

The vast majority (in my experience, 80%+) have _no business_ writing code _at
any level_. We're not talking about bad programmers here, we're talking about
_people who cannot program, full stop_.

I can deal with "I'm not nearly as good as I say I am", I cannot deal with
"What's an if statement?", but yet the majority of applicants at most large
software firms _are_ like this.

If you are a competent dev, FizzBuzz isn't designed to trap you, it's designed
to trap someone far, far, far, far, far, far, far, far less competent than
you. Just jump through this hoop and we can start the _real_ interview.

You also make the mistake that FizzBuzz is the _whole_ interview. It isn't.
I've run many highly rigorous interviews, FizzBuzz is just the bar for me to
talk to you past the 5 minute mark.

What I still don't get is how these people even exist. It's like showing up to
an accountant's interview with a box of crayons and a bib. It's disturbing.

