
The camel doesn't have two humps: Programming “aptitude test” canned - Smaug123
http://retractionwatch.com/2014/07/18/the-camel-doesnt-have-two-humps-programming-aptitude-test-canned-for-overzealous-conclusion/
======
optforfon
The title seems to intentionally misrepresent the retraction to push an agenda
(or the author has some issues with basic logic)

The retraction is that the strong/definitive proof that "the camel has two
humps" was fabricated/exaggerated.

The converse - that it has one hump hasn't been proven. From the quote,
Richard Bornat doesn't say that they re-crunched the numbers and now there is
proof of one hump.

The camel has gone back to having an unknown number of humps

~~~
braythwayt
In the absence of that paper, we had plenty of reasonable hypotheses about the
ability to program. Like “Put 10,000 hours into it, and you’ll master it.”

Or “It follows a distribution similar to other skills involving mathematics.”

The paper presented an unusual hypotheses, that there was this sharp
distinction between those who can program and those who never will, and that
this distinction was somewhat orthogonal to other measures of scholarship. So
for example, one might be a good architect but never get good at programming
no matter how much one tried.

You’re right we haven’t disproven this hypothesis, but given its novelty, the
burden of proof is on "the camel has two humps." With the paper retracted, we
do not presume it has an unknown number of humps, we presume that skill
writing programs is going to be similar to other skills we observe.

As this article observes, unusual outcomes in attempting to teach programming
may just as easily be explained by the fact that in a young field, we may not
know how to teach programming.

If we’re going to chase ‘humps,’ perhaps we should look for unusual
distributions in the skill of teaching programming. We may be terrible at it,
and perhaps the skill we are really observing is the skill of learning to
program in spite of didactic and structural obstacles.

~~~
throwaway856
Meanwhile, it's taken for granted that not everyone can be a doctor, dentist,
or lawyer. You don't see doctors telling people that anyone can "learn to
med," but programmers see nothing wrong with devaluing their profession in the
eyes of the masses by telling them that anyone can learn to do what they do,
or worse, that it's easy, and that we shouldn't assume anything else until its
proven otherwise.

Programmers are far too egalitarian for our own good and our increasing
marginalization and stagnant wages are the inevitable consequences.

~~~
braythwayt
You are misinterpreting what the paper said and what I am saying in order to
oil the axe you are grinding.

First, one can dispute “programming ability is distributed in two humps”
without implying that “everyone can program.”

Second, your proposition isn’t even a consistent proposition. If programming
talent really is rare, then being egalitarian won’t have any effect on wages:
Some people can, some cannot, and the free market will quickly sort out that
you cannot build pyramids with thousands of unskilled programmers.

If wages are stagnant, one possible explanation is that more people can
program than you would care to admit. Which explains why you are ranting about
spreading the myth that programming ability makes you a special, delicate
flower. If you can socially engineer people into not becoming programmers, you
believe there will be more money for you.

What you fail to realize is that as an industry, “a rising tide lifts all
boats.” The more talented people there are, the larger the pie we get to
share. More programmers equals more software, equals more tools, equals more
companies, equals more hardware, equals more demand for software, and so it
goes until software has finished eating the world and it becomes a mature,
stagnant industry.

If you feel your wages are not rising as quickly as you like, perhaps you
should look into other possible causes, such as a lack of skill in
negotiation, or choosing to be an employee instead of an entrepreneur, or
wage-fixing by companies with no-hire policies, or the practice of handing out
paper stock options in lieu of cash for startup employees.

~~~
MustardTiger
>First, one can dispute “programming ability is distributed in two humps”
without implying that “everyone can program.”

Yes, but people are disputing “programming ability is distributed in two
humps” despite there being clear evidence of that being the case, and no
evidence to contradict it. The distribution has not changed, or been
retracted. Only the unsupported notion that the distribution is innate and
unchangable.

~~~
braythwayt
I am not responsible for explaining to you what "people" are saying, whether
they are right or wrong.

Please cite this "clear evidence" you speak of.

~~~
MustardTiger
I did not ask you to explain anything. I said your explanation is not
representative of what is happening. And the clear evidence is the paper in
question. Read the "retraction". They do not suggest the bimodal distribution
is incorrect, but that they should not have suggested it was inherent and
unchangable, when the evidence does not support that.

~~~
hyperpape
_His test is not a very good predictor: most of those who appear to use a
model in the pre-course test pass the end-of-course exam but so do many of
those who do not. And it doesn’t predict the level of performance, as we
discovered when we tried some deeper statistical analysis (Bornat et al.,
2008). But the phenomenon is real and the prediction it makes is reproducible,
as Dehnadi showed in a meta-analysis in his thesis (Dehnadi, 2009). That meta-
analysis is summarised in (Dehnadi et al., 2009).

But that’s not all. It’s not enough to summarise the scientific result,
because I wrote and web-circulated “The camel has two humps” in 2006. That
document was very misleading and, because web documents persist, it continues
to mislead to this day. I need to make an explicit retraction of much of what
it claimed. Dehnadi didn’t discover a programming aptitude test. He didn’t
find a way of dividing programming sheep from non-programming goats. We hadn’t
shown that nature trumps nurture. He had, however, found a predictive
phenomenon, though he had no explanation of it._

How do you read these paragraphs, or what part of the retraction supports the
"two humps" claim and how do you understand that claim?

~~~
MustardTiger
I don't know how I could make it any clearer than what you quoted. What do you
think "But the phenomenon is real and the prediction it makes is reproducible,
as Dehnadi showed in a meta-analysis in his thesis" means? Or "We hadn’t shown
that nature trumps nurture. He had, however, found a predictive phenomenon,
though he had no explanation of it."?

------
panic
This retraction shows we have to be extra critical of things that reinforce
our beliefs. Working as a programmer, it can be easy to believe that some
people just aren't cut out for it. That doesn't mean a publication written
using official-sounding words and methods that happens to match this belief is
true at all.

~~~
splintercell
But the retraction is super terrible. Imagine someone writes an article which
makes a lot of sense, now that person writes a retraction which simply says
"oh I was in great mental and psychological trouble when I wrote that".

No, if you want to retract it, then make a case why this is not true. We don't
believe that article because we trust the author.

~~~
hyperpape
Did you look at the actual retraction? It explains what the original data
demonstrated and references further work by him, his student, and other
researchers that helps show what the test can and can't show.

Here's the retraction:
[http://www.eis.mdx.ac.uk/staffpages/r_bornat/papers/camel_hu...](http://www.eis.mdx.ac.uk/staffpages/r_bornat/papers/camel_hump_retraction.pdf)

------
YeGoblynQueenne
So, this discussion reminds me a lot of discussions in logic programmig
circles about why so many programmers can't learn to program in Prolog, no
matter what. There's papers dedicated to the question, people who had done
research on the question back in the '90s when Prolog was still popular (yeah,
it was, google it). The verdict: no verdict. Some few special and magical
people seem to be able to learn to program in Prolog, but the majority can't.

Well then, why not set the bar for who can program and who can't on who can
code in Prolog, rather than who can write FizzBuzz? If the point is to sort
out the men from the goats, or whatever uncle Joel said we're trying to sort
out, surely using Prolog as the sorting tool would leave many, many fewer
"real programmers" than FizzBuzz. Hell, we could even combine the two:
FizzBuzz in Prolog! See how well you can do at that, internets!

And why stop at Prolog? There's languages that are way, way harder to program
in! Hey, if you're a Real Programmer you should be able to write FizzBuzz in,
dunno, Ook. Brainfuck. Leboge? Mondrian? Perl, even?

Or maybe, I don't know, we can start asking people to show us what they can do
instead of trying to find things they can't? Does that make sense to anyone
else? I'd personally love it if people who want to hire me took the time to
check out my github and the git repository I'm serving off my public server,
in order to get an idea of the things I have already done (and so demonstrably
can do) rather than trying to catch me with my pants down in front of a
whiteboard.

Look. I write a lot of code. I go over a lot of code I've written. I'm an
idiot 80% of the time, but the other 20% of the time I figure it out, fix my
bugs and go to town. It works, OK? If interviews can't appreciate this balance
then they 're useless.

------
peterjmag
Down for me, but Google's cache caught it:

[http://webcache.googleusercontent.com/search?q=cache%3Aretra...](http://webcache.googleusercontent.com/search?q=cache%3Aretractionwatch.com%2F2014%2F07%2F18%2Fthe-
camel-doesnt-have-two-humps-programming-aptitude-test-canned-for-overzealous-
conclusion%2F&oq=cache%3Aretractionwatch.com%2F2014%2F07%2F18%2Fthe-camel-
doesnt-have-two-humps-programming-aptitude-test-canned-for-overzealous-
conclusion%2F&aqs=chrome..69i57j69i58.1622j0j4&sourceid=chrome&ie=UTF-8)

------
_Codemonkeyism
1\. Interesting.

2\. I still wonder why around 30% of people with programming experience who
have applied to a job when I was recruiting couldn't solve FizzBuzz[1] or
string reverse. All of them either University MSc or years of programming
experience.

The article hints at using the wrong teaching methods.

[1] Given the written instructions: 'Write a program that prints the numbers
from 1 to 100. But for multiples of three print “Fizz” instead of the number
and for the multiples of five print “Buzz”. For numbers which are multiples of
both three and five print “FizzBuzz”.'

~~~
p0nce
The problem is that Fizzbuzz and string reversing are a kind of trick
questions. Either you have already done them or not. Good people will fail at
trick questions very easily in interviews. Since they don't represent real
work and aren't statistically significant, why are we using them? They are
merely adding random noise in the interview process.

~~~
kagamine
Never had cause to reverse a string but have needed to reverse an array. Never
had cause to do something like fizzbuzz but have needed modular operator
(actually both in the same function) but if I had never had reason to validate
barcodes of differing lengths and validate the check-bit, I would probably
fail on fizzbuzz too. In fact I probably would fail today if asked, who has
time to remember something that useless, and that easy to look up?

~~~
_Codemonkeyism
I would like to learn and understand how you think you would fail on FizzBuzz
if you get these instructions [1]:

'Write a program that prints the numbers from 1 to 100. But for multiples of
three print “Fizz” instead of the number and for the multiples of five print
“Buzz”. For numbers which are multiples of both three and five print
“FizzBuzz”.'

[1] if stuck on modulo I usually suggest to assume there is a function
divisible

~~~
kagamine
I have a terrible memory and struggle to be confident, I would probably just
not manage to write anything under interview conditions. When I first learned
about fizzbuzz I did it and wondered what all the fuss was about, it's not a
complicated logic puzzle, just that I'm used to writing code while connected
to the Internet, my brain simply doesn't try to remember things I _know_ I can
look up in 1 second.

------
smoyer
I have a really good way of separating candidates into those who will be
successful and those who won't - During the interview I ask how they became
interested in computer science, programming, etc as well as asking them to
provide information on "pet projects" or other significant work they've
accomplished.

If someone describes staying up all night as a 13 year-old because they had to
finish the code they were working on, that's someone who was passionate about
programming before they found out it's one of the better paying professions.

If someone points to finished projects (especially side projects or OSS), then
they also have the ability to focus. If someone points to a bunch of barely
started projects, they might have great ideas but in a business you also need
to execute.

So ... passion and focus, plus a reasonable amount of domain knowledge will
make a great employee.

~~~
pjc50
_If someone describes staying up all night as a 13 year-old_

So, you're explicitly selecting by class background and upbringing? This
particular metric also tends to select against women, if you have any
applying.

~~~
alistairSH
Not just class, but age. Those of us who grew up in the 70s or 80s were much
less likely to have a PC in the house as a child. High-speed internet wasn't
commonly available until I entered college (and even then, it wasn't something
found in homes - just offices and colleges).

~~~
collyw
Plus parents occupation.

I can remember on two occasions bosses of mine who helped kids with some form
of homework assignment. If your dad (or mum) isn't technical then you are less
likely to get an early start.

------
URSpider94
I think that people are completely misinterpreting the article and the
retraction in this thread, whether willingly or not.

I don't think anyone is seriously questioning the fact that some people have
more aptitude or intelligence than others. That's pretty commonly accepted
fact. However, the general belief is that intelligence follows a roughly
Gaussian (bell curve) distribution, and that it's roughly continuous.
Mechanistically, one can imagine that intelligence is a convolution of genetic
and other factors that are so thoroughly mixed that there are no discrete
steps.

The theory behind a two-humped camel model of aptitude or intelligence would
then be that some magic x-factor is so significant that it splits the
underlying distribution in two, into the "cans" and the "can nots".

While the existence of such an x-factor would be a really interesting finding,
it wouldn't change the underlying fact that some people have more innate
ability to program than others, just like some people have more innate ability
to be doctors or lawyers or concert pianists than others. It would just tend
to make that difference much starker.

What it also wouldn't change is the fact that innate ability doesn't always
correlate to success in a given career, or in life. Many other factors other
than innate ability, such as drive and ability to collaborate with others,
affect whether someone is going to be successful at their job. Also, people
can learn. Even someone who doesn't have innate talent can become very
skilled, if they work really hard at it.

So, even if the study were valid, which it's not, the finding itself is more
of an academic interest than an excuse to start treating coding job interviews
as a "sword in the stone" test of whether you've got "it" or not.

------
PaulHoule
This is one of the few times I've seen SSRIs blamed for jackass behavior.

~~~
drunken-serval
I'm going to second what possibility said. Bipolar depression is a very
different animal from standard clinical depression. Mania is ugly, it turns
someone you know into someone you don't.

~~~
PaulHoule
Until recently we were living with a maniac who didn't need SSRIs to go manic.
It is astonishing how ugly it gets, I've still got a broken mirror on the
floor in the other room that we haven't cleaned up yet.

------
PaulHoule
Psychologists would reject any kind of test that doesn't come out as more or
less normally distributed out of hand. A common complaint about oddball tests
such as the Meyers-Briggs test or Hubbard's OCA is that they have not been
validated to "behave well".

This may well be to due the fact that psychologists have been using parametric
statistics instead of nonparametric statistics. In a lot of cases us
nonparametric types have given up a little statistical power for "cheap and
cheerful" methods that are harder to get wrong.

Psychs and medicals find it very expensive to treat thousands and thousands of
patients correctly and carefully observe the results. This is why those
Cochrane reviews for the case for almost any drug or other treatment are so
depressing -- maybe they need all the statistical power they can get.

~~~
aidenn0
I think you have that wrong. One major criticism of Meyers-Briggs is that it
measures things that _are_ normally distributed, and assigns them categories.

For an easy example of why (if true) this would be a valid criticism, consider
IQ, which is normally distributed. I will give you an IQ test and assign you
into one of two categories: smart or dumb. If you are below the median, you're
dumb, if you're above the median, your smart.

It should be obvious why this makes little sense; the large number of people
very close to the median are divided in two and put in the same category of
those with either very low or very high IQs.

To assign people to one of two categories, one would want to see a distinct
bimodal distribution, then there would be a small number of people for whom
which distribution they belong to is ambiguous, and the majority could be
confidently assigned to one or the other.

------
ebrenes
So, this article talks about the retraction of the 2006 paper, which the
authors admitted had some failings.

However, I fail to see much discussion about the subsequent paper [1] that
addressed these failings and featured improved experiments that seem to still
uphold the hypothesis.

[1]
[http://www.eis.mdx.ac.uk/research/PhDArea/saeed/SD_PPIG_2009...](http://www.eis.mdx.ac.uk/research/PhDArea/saeed/SD_PPIG_2009.pdf)

------
aidenn0
I always thought the data from "the camel has two humps" merely demonstrated
the lack of good pedagogy in the field of programming.

~~~
cb18
> _All teachers of programming find that their results display a 'double
> hump'. It is as if there are two populations: those who can [program], and
> those who cannot [program], each with its own independent bell curve. Almost
> all research into programming teaching and learning have concentrated on
> teaching: change the language, change the application area, use an IDE and
> work on motivation. None of it works, and the double hump persists. We have
> a test which picks out the population that can program, before the course
> begins. We can pick apart the double hump. You probably don't believe this,
> but you will after you hear the talk. We don't know exactly how/why it
> works, but we have some good theories._

> _Despite the enormous changes which have taken place since electronic
> computing was invented in the 1950s, some things remain stubbornly the same.
> In particular, most people can 't learn to program: between 30% and 60% of
> every university computer science department's intake fail the first
> programming course. Experienced teachers are weary but never oblivious of
> this fact; brighteyed beginners who believe that the old ones must have been
> doing it wrong learn the truth from bitter experience; and so it has been
> for almost two generations, ever since the subject began in the 1960s._

[http://blog.codinghorror.com/separating-programming-sheep-
fr...](http://blog.codinghorror.com/separating-programming-sheep-from-non-
programming-goats/)

emphasis: _All teachers of programming find that their results display a
'double hump'. It is as if there are two population_

emphasis: _Experienced teachers are weary but never oblivious of this fact;
brighteyed beginners who believe that the old ones must have been doing it
wrong learn the truth from bitter experience_

So the disparity has presented itself in all known historical forms of
pedagogy. Some have learned while others haven't, some have even learned with
entirely self-directed pedagogy.

When a array of teachers has been giving it their all for decades but the
disparity remains, should we really blame the teachers?

~~~
aidenn0
Harvey Mudd has had some success in increasing reenrollment in CS classes
(there is a required first-year CS class there) by changing pedagogy.

The search space of possible ways of teaching programming is so large, and the
field so young, that I'm not convinced we're anywhere near optimal.

~~~
cb18
I agree, in that I think there is always room for improvement.

What I think needs to be criticized is a tendency to go on these wild goose
chases searching for a magic factor that will explain everything, when there
are other obvious factors nearly staring us in the face that we don't want to
address.

It's the opposite of Occam's razor.

~~~
aidenn0
Indeed; spending your time searching for the silver bullet when you've got a
full magazine of lead bullets.

------
regularfry
(2014)

~~~
JoBrad
And yet, people in these comments are still clinging to it. Internet memes are
the new afterlife.

~~~
vorg
Perhaps the people in these comments have had many programming experiences
working with people who have great trouble coding, _no matter how patiently we
explain something to them_. As a result, programming workplaces are often
structured so one 3x programmer looks after a group of -0.5x programmers,
cleaning up after them, while another group of 2x programmers are siloed off
doing some real work, unencumbered by any -0.5x programmers and not threatened
by the 3x programmer. This is the double hump in action.

Despite working in these types of place, I still got a shock, though, when I
started doing freebie work on open source software and worked out that such an
aptitude-less programmer was managing a large open source project developing a
programming language. Perhaps some people doubting the double hump are from
places like Google where all engineers are vetted via Computer Sciences
degrees, and they don't have repeated everyday experience of it.

