
Coding Horror: Separating Programming Sheep from Non-Programming Goats - joeyespo
http://www.codinghorror.com/blog/2006/07/separating-programming-sheep-from-non-programming-goats.html
======
raganwald
Jeff closes with the following words IN BOLD:

“The act of programming seems literally unteachable to a sizable subset of
incoming computer science students”

There is, of course, an alternate explanation:

“We in Universities seem literally incapable of teaching programming to a
sizable subset of incoming computer science students”

Experiments going back to the 1980s with teaching children how to program
using Lego blocks and robots with Logo seem to indicate that nearly everyone
_can_ learn how to program, but possibly not when they’re 18 or 19 and
possibly not with the kind of academic environment represented by the test.

~~~
paganel
> Experiments going back to the 1980s with teaching children how to program
> using Lego blocks and robots with Logo seem to indicate that nearly everyone
> can learn how to program, but possibly not when they’re 18 or 19 and
> possibly not with the kind of academic environment represented by the test.

I only learned how to program at around age 20 or 21, but at that point I had
already been a CS student for 2 years, and of course had failed most of the
actual programming-related classes (even though I had earned a A+ in calculus,
with congratulations from the professor). I am now a professional programmer,
I'm in my early 30s, and as such I'll quickly write down my thoughts on this,
maybe it helps someone make some sense of it.

Now, about the particular example in the article, I can still remember the
first programming class during my first semester at Uni. I can still remember
about how baffled I was when seeing almost the exact same code as in the above
article, I was wondering to myself: "assignment works left to right, is the
only natural way, what do you mean now "a has the value of b"? You first write
down a, the 'equal' sign, and then b, it's only normal that b, coming at the
end, takes the value of a". And so on. Someone also mentioned the
"competitive" nature of a CS-class, which is 100% spot-on. I had never owned a
computer during high-school, while some of the my then-colleagues had been
programming for 10 years already, and because of that I was afraid to ask
questions or even to ask for help from my colleagues.

And in case you're wondering how did I finally became a programmer, well, it
was thanks to Python. At first I started writing some ugly scripts in PHP, but
after 6 months I stumbled upon Python. I don't know exactly why, but it's
something about this language that makes it very easy for non-programmers to
grasp programming language concepts. After one year of doing Python I finally
had the balls to try and take my most difficult programming class again. I got
an A (the class was taught using the C programming language), with the
professor surprised about how had I managed to do that.

~~~
Jach
I think Python's a great language to start with, maybe. What turned me on to
PHP and web programming is that I could do things easily and get visual
results back. Flask is a better way to build server side apps than raw PHP,
but it hides a lot and a significant part of the Python language needs to be
understood to understand a lot of it.

I spent a year learning about website design shortly after turning 14 and
during that time eventually came across PHP. For about another year until 10th
grade started, I kept doing simple PHP sites. I had looked at Python once and
felt kind of offended by it, and I didn't understand OOP. "Why do I have to do
mystr.split(' ') instead of explode(' ', $mystr)?" "Why don't variables have
sigils?" "What's this import crap?" "Why do I have to have indentation?" (I
commonly didn't indent my PHP code during this time.) I also used Windows. I
still hate cmd.exe.

I took the AP CS course in 10th grade, and Java forced me to at least
partially understand OOP and why it may be useful or cool. (We also did
everything on a Mandrake Linux computer.) I looked at Python again about
midway through the year and started loving it (I was in the habit of indenting
with one space at that point since the teacher required at least some
indentation so Python's forced indentation didn't bug me anymore). After
several months in Python I began to understand that real OOP went far beyond
what Java supported. Now that I'm 21 Python is still my favorite language, but
I do really like Clojure.

So I still think Python is a perfect beginner's language, but I'm not entirely
convinced that Python as Python is enough to spark an interest to go do your
own thing. The first thing I reached for when I learned Python well enough to
do anything was video game programming. Making interactive applications that
aren't limited to the terminal held my interest a lot more back then.

(My CS teacher also resolved many difficulties with the = sign by always
reading code like "a = b" as "a gets b". Some people have trouble enough with
homonyms in language, seeing the '=' mean something completely different to
what they've been used to for 10 years can be startling.)

~~~
dodedo
'My CS teacher also resolved many difficulties with the = sign by always
reading code like "a = b" as "a gets b"'

This is one of the great things about Pascal. The := sign is used for
assignment, and the equals sign alone is used for equality. It's very natural
to call := 'gets' and keep a clear distinction between assertions and
assignments.

~~~
Jach
I totally agree, := should be used more. (More than once I've had (define :=
define) in my personal Scheme code.) Unfortunately my enthusiasm in the past
has been met with the quip "Any language that uses := as the assignment
operator inevitably dooms itself to never becoming more popular than Pascal."
Not many people care anymore. (They did at one point; a lot of 80s and some
90s C code is so macro'd up it looks a lot like a dialect of BASIC.)

------
jerf
As I recall, people have attempted to replicate the results this blog post is
based on and the result was debatable. Which is to say, not that they were
completely unreplicable, but that under further investigation debate has
intensified. It isn't clear that this methodology actually works, but it also
was not conclusively destroyed. It's probably not a good scientific base to
draw a lot of conclusions on right now.

~~~
gwern
That is correct - the authors themselves failed to replicate the results. I
kept some notes over the years (saw this OP in 2006); from
<http://www.gwern.net/Notes#the-camel-has-two-humps>

...replications of the test have had issues; from
<http://crpit.com/confpapers/CRPITV78Bornat.pdf>:

> "We now report that after six experiments, involving more than 500 students
> at six institutions in three countries, the predictive effect of our test
> has failed to live up to that early promise.”

And
[http://www.eis.mdx.ac.uk/research/PhDArea/saeed/SD_PPIG_2009...](http://www.eis.mdx.ac.uk/research/PhDArea/saeed/SD_PPIG_2009.pdf)

> “A test was designed that apparently examined a student’s knowledge of
> assignment and sequence before a first course in programming but in fact was
> designed to capture their reasoning strategies. An experiment found two
> distinct populations of students: one could build and consistently apply a
> mental model of program execution; the other appeared either unable to build
> a model or to apply one consistently. The first group performed very much
> better in their end-of-course examination than the second in terms of
> success or failure. The test does not very accurately predict levels of
> performance, but by combining the result of six replications of the
> experiment, five in UK and one in Australia. We show that consistency does
> have a strong effect on success in early learning to program but background
> programming experience, on the other hand, has little or no effect.”

------
cletus
I see this largely as a failure of education. You see this issue in many areas
and much earlier on like teaching mathematics. In schools there tends to be
one method used and that method doesn't suit everybody.

I see this at work. I've known programmers who, when they get a new hire, will
basically let them flounder. I guess the basic idea is that if you're "good"
you'll "figure it out" (with minimal time investment to boot). And that will
work with some people. My own philosophy is that there are a significant
number of people who need direction. Don't overwhelm them with "everything".
Give them small, structured tasks to learn processes, small parts of the
system, build confidence and give some measure of progress.

That all being said, I honestly don't know how I learnt to program at all. I
can teach people who know how to program (the basics) more. But I can't teach
someone who doesn't know how to program how to program.

I've come around to thinking that Zed Shaw (as just one example) is right
about this. With his "Learn X the Hard Way" books he starts out by basically
saying "just type this in" (and don't copy and paste). Don't worry what it
does. You'll figure that out later.

I see people (in many different areas) who are under the mistaken impression
they need to know "everything" before they can do anything (or even in games,
make a decision of any kind). They feel the need to weigh up all the options
and consider all the consequences. That's not actually how people learn
(IMHO).

Children don't know any better so can just learn this way. Adults often let
self-doubt, fear of embarrassment or fear of "what might happen" get in the
way such that they won't allow themselves to learn.

So perhaps what universities should do is separate students who already know
how to program from those that don't. Those that don't don't take ANY theory
at all until they do an introductory course that starts with "just type this
in". If you lack the ability (and _willingness_ ) to turn a thought into a
program, no matter how simple, you're doomed to failure. This, I believe, is
something that can be taught.

The ability to just try stuff and see what happens in a key element in
continual learning (IMHO) and something that needs to be ingrained in
programmers (to be) from day one.

I'd really love to see the outcome of this approach versus more traditional
approaches.

~~~
parfe
> _I see this largely as a failure of education. You see this issue in many
> areas and much earlier on like teaching mathematics. In schools there tends
> to be one method used and that method doesn't suit everybody._

Good thing professional educators are almost as observant as you in regard to
teaching.

<http://lmgtfy.com/?q=differentiated+instruction>

I really love how HN conceit lets people post about entire industries as if
_those people_ just have no clue what they are doing. And not only do people
here hold those pompous and generally ignorant positions, but other people
support them!

~~~
dpritchett
I read cletus's post to suggest that primary math education (i.e. young
children) isn't personalized very much. This is demonstrably true, at least in
the U.S. A lot of it is budgets and politics. No need to infer that cletus
believes he is a more thoughtful educator than people who spend their careers
on it.

~~~
gioele
> primary math education (i.e. young children) isn't personalized very much.
> This is demonstrably true, at least in the U.S.

I would say that it is demonstrably not true in many EU countries. Maybe "non
personalisation" is the tendency of lazy teachers, but I remember pretty well
how my math teachers repeated the same subject thrice: the first time
describing its core in an abstract way, the second time with concrete examples
(apples and pears) and the third time in very little incremental steps. The
first explanation was for naturally able math learner, the second for those
how needed practical examples and the third for less able pupils. Some of us
complained about the fact that it was a boring way to teach (and you know how
horrible bored kids can be in the class) but I'd say it was worth it to have
all my classmates learning more or less at the same pace with nobody left
behind.

Actually, I complain more about the fact that we had to change math teachers
so many times because of educational reforms and budget cuts/reorganizations.

~~~
hobin
Which country do you live in? I don't know about the quality of schools in
general here (the Netherlands), but I've met too many people who say they
"can't do math" to think very highly of our elementary educational system.
Granted, though, this may have more to do with the fact that it's generally
acceptable to 'just not able to do math'.

(This is not to say that I think everyone can do math equally well, but I've
seen some otherwise intelligent people making such comments that made me want
to rip my eyes out.)

~~~
flaviusb
I think of it like Sturgeon's Law applied to humans. Most humans just are not
very good at most stuff.

Look at the percentage of programmers that are basically not any good at
programming all and yet are able to continue being called programmers; if we
apply standards of expectation uniformly, then you would expect to see even
people in technical fields be able to be useless at math and have everyone
basically just shrug and accept it.

------
lutorm
That article is positively drenched in _fixed mindset_
([http://www.scientificamerican.com/article.cfm?id=the-
secret-...](http://www.scientificamerican.com/article.cfm?id=the-secret-to-
raising-smart-kids)). Those prophecies tend to be self-fulfilling. But as
raganwald says, the only _evidence_ is that universities fail to teach a
sizable fraction of the population. Maybe the trick is not to play with IDEs
or whatever else they claimed to have tried, but to get rid of the computer
and focus on what programs are: instructions for how to do something.

The example of "mental model of assignment" is ridiculous. a=b can mean
whatever. It means something totally different to a mathematician than a
programmer. Say:

"The piece of paper on the left says _dog_ and the piece of paper on the right
says _cat_. Overwrite the contents of the piece of paper on the left with the
contents of the piece of paper on the right. What is written on the pieces of
paper?"

I bet pretty much everyone would have a correct mental model of assignment if
the problem was stated like that.

~~~
patio11
_I bet pretty much everyone would have a correct mental model of assignment if
the problem was stated like that._

Try this experiment with five people at a public place, say exactly what you
said, and give them exactly one try at the correct answer. I'll give $10 to
charity for each one that getes it right. If all five of them get it wrong,
you owe me an upvote.

(Fair warning, the terms of this bet are ridiculously unfair to you.)

~~~
vibrunazo
My bet is on lutorm, the point is that the original questions are ambiguous
and if you remove some of the ambiguity you increase the chances of someone
getting it right.

My theory is that those who got it right (at least most of them) didn't get it
because they "formed a consistent model". But simply because they made the
same assumptions people before them did when creating the language. But of
course, would need more data to be more sure of anything.

------
gatlin
I studied to become a computer science teacher. I have not finished my
certification for various reasons but I have some limited field work under my
belt along with numerous discussions and readings. And, of course, my
anecdotal but legitimate experience in public schooling.

Let's get one thing straight: the problem is bad teachers. My mentor once said
it kind of sucks that UTeach was only available to UT students and by
extension those who statistically succeeded in traditional schooling because
they are usually the least acquainted with the system's failures.

That "test" they show inside is bullshit, first of all. If they showed that to
me as a completely green 11 year old without explaining the syntax or
semantics of what was going on, I'd probably have to ask some clarifying
questions. And, I think it goes without saying, if the professor is the type
of teacher to go into a class with the expectation that half the class will
fail, I'd probably have shaky confidence and would hesitate to ask questions
since everyone is quickly cast into a mold of either "have" or "have not."
I've taken math courses before by these people.

Many so-called "gifted" math students (and we're all students) are great with
applied math but are just as stumped by theory as everyone else (anecdotal).
Testosterone and typical youthful arrogance turns classrooms into settings
hostile to people aren't extremely competitive and who are somewhat unsure of
themselves. Combine this with a teacher who was once one of the cocky ones and
a few snide remarks about how some students "just don't get it" and what you
have is a self-fulfilling prophecy.

If your students fail to learn, it means you did not teach them adequately.
Learning is the product of the work of learners: you have to give them the
proper resources to work with. Humans are exceptionally capable. Children are
extremely inquisitive and interested in so many mundane things; they can be
taught math. This is beaten out of them as they get older and they're told
there is a cap to what they can accomplish.

Bah. Humbug.

A good set of resources for anyone looking to teach programming and/or
computer science is CS Unplugged (<http://csunplugged.org/>). No computers.
Much of it relies on human interaction. It's fun, it's versatile, and most
importantly it distills the important parts of computer science while shedding
the stuff that gets in the way: confusing, advanced user interfaces; arcane
terminology and confusing history; and the stigma that a computer brings to
those who've never worked with them before.

~~~
bryanlarsen
You're a direct target for the article. Quoting: "brighteyed beginners who
believe that the old ones must have been doing it wrong"

People have been trying and failing for 40 years to improve CS teaching.
Granted, most professors don't try and just teach the way it's always been
taught, but some, like you, do.

The big question then becomes: is this an impossible problem, or simply a very
hard one? When you start teaching, you will find that a certain percentage of
your students fail to "get it". Will this be your fault? Probably not. At the
very least, to "get it" requires effort, and some of your students won't put
in that effort, and you can't force them to. There are many ways you can
encourage the effort, but you can't force it.

What we can do, though, is shift the goalposts. Turn a 70% failure rate into a
30% one into a 10% one into a 2% one, et cetera. It looks like csunplugged is
one of those tools that will shift those goalposts.

~~~
gatlin
To start, I agree with you 100% on all accounts. Some students may truly never
succeed but I think that by declaring the case closed and assuming it's purely
a function of student ability, we lose out on valuable information about
humans learn because: what if that's wrong?

The fact that the goalposts can be moved is awesome and a testament to what
can happen if we assume that it's a communication or motivation problem, not
student ability.

There are lots of fringe students who just think oddly. Living in cooperative
housing in Austin has really driven this home. They require odd teachers.

------
mhartl
It's interesting that this gets highlighted as an example that baffles the
'non-programmers':

    
    
        int a = 10;
        int b = 20;
        a = b;
    

This looks obvious to an experienced developer, but it is a clear-cut case of
horrible notation. The first two uses of '=' are mere variable bindings,
whereas the third is a _mutation_. Using '=' for bindings is dubious but
defensible, since

    
    
        int a = 10
    

does indeed guarantee that _a_ = 10 in a mathematical sense. Using '=' for
mutation is crazy, though, since assignment isn't commutative:

    
    
        a = b
    

and

    
    
        b = a
    

are _not_ equivalent. The pinnacle of confusion can be seen in the following
abomination:

    
    
        x = x + 1
    

This looks like an algebraic equation with no solution. So, you take people
who have been taught to use '=' for mathematical equality their whole lives,
and now they're supposed to swallow these absurdities? No wonder people are
confused.

What we're seeing here is essentially a _pun_ : the use of '=' for at least
three different things (binding, mutation, and mathematical equality). To my
knowledge, only Lisp gets this right (although there must be others). For
example, in Scheme we would have

    
    
        (let ((a 10)
              (b 20)))
    

for bindings,

    
    
        (set! a b)
    

for mutation, and

    
    
        (= a b)
    

for equality comparison—three ideas, three different notations.

I'm certainly prepared to believe the hypothesis that some people will just
never 'get' programming, but we should consider the possibility that bad
notation is part of the problem. Given how entrenched it is, changing notation
might be unwise, but at the least we can _acknowledge_ and _explain_ ambiguous
notation rather than treating it as self-explanatory.

~~~
gruseom
_The pinnacle of confusion can be seen in the following abomination: x = x +
1_

Bill Wadge invented the dataflow language Lucid in the early 1970s by
pondering this very paradox. He wrote a nice series of blog posts about it
that begins at <http://billwadge.wordpress.com/2011/03/23/lucid-the-origins>.

His insight was that x = x + 1 isn't contradictory once you recognize that the
two x's have implicit temporal subscripts. That is, "the value of x at time
t+1 = the value of x at time t, plus 1". Make that temporal dimension explicit
in the language and you can go back to having a nice equational semantics.

It's interesting, though, and perhaps germane to your point, that this insight
turned out to lead far away from mainstream languages.

I've come to the conclusion that math and programming are very different
things because math isn't executable. Programs have to run on a physical
machine, math doesn't. The "physical" part changes everything. As someone
pointed out in a blog post recently, even programs written in the purest most
side-effect-free languages have side-effects when they run. I suspect that
functional programming ultimately runs aground on this discrepancy - i.e. on
the intrinsic imperativeness of the machine. The "x = x+1" paradox can be seen
as a distillation of this imperativeness.

Perhaps we should teach programming not as math but as a kind of mechanics.

------
jseims
I used to teach the first CS course at both Harvard and University of
Washington. I didn't have this issue at Harvard, but at UW there was
definitely a bimodal distribution in abilities. Roughly half the students had
what I feel is a more visual / associative way of thinking vs. the focused
structure you need to program.

The concept that the "non programmers" could never understand was the
abstraction of functions. Like:

foo(a) { // stuff }

foo(b);

would confuse them because they'd get hung up on foo's parameter declared as
'a', but called with 'b'.

~~~
Raesan
I took a few CS classes at the University of Washington in the 08–09 academic
year. The intro CS classes had recently been redesigned [1] to be more
effective. Anecdotally: I talked four of my friends into taking one or both of
the intro classes (split evenly between science/non-science majors), and they
all did well and really enjoyed them. I think how programming is taught at
colleges is definitely a fixable problem.

[1] <http://www.cs.washington.edu/homes/reges/sigcse/>

------
sassafras
Teaching is hard.

I understand the impetus to filter out students that are less naturally
inclined to it, and if your goal is to establish something of a programming
conservatory, then administering a test like this would be an effective
barrier. However, I think perhaps more advantageous reading of these results
would be to recognize that the ability for students to construct these mental
models is the single most important foundation in their education.

Anecdotally, I have had some experience teaching non-programmers basic
programming skills at the university level, and it's absolutely the case that
some kids, even bright ones, struggle greatly with these constructs at first.
They look at the symbols and freeze, intimidated by the arcane expressions,
and control flow is often non-intuitive to them. To me at least, that's a
pedagogical challenge, not a lost cause. You should have seen the exclamations
of wonder and excitement when some of them finally got it, once a particular
way of illustrating it stuck for them.

------
tokenadult
The date of this interesting submission (haven't I seen this on HN before?) is
Jul 14, 2006. What has happened since then? What follow-up research has there
been on the methods of first-time undergraduate introduction to programming
courses? (If I remember correctly, someone on HN has pointed to follow-up
papers by the same author in some earlier HN thread, but what I would
especially like to see is attempts at replication by other researchers

<http://norvig.com/experiment-design.html>

and attempts by other researchers to look at the problem from other points of
view.)

There are quite a few undergraduate disciplines in which it is assumed that
most first-year undergraduates will not continue studying the discipline. In
some disciplines that is considered a bug, but in some that is considered a
feature. It's too bad that many undergraduates registering for a first-year
course don't know that their professors don't expect most learners in the
class to succeed. What would college enrollments in different subjects look
like if there were more disclosure of that pessimistic attitude?

(Basis of knowledge: I have one son who is an adult now, the child of two
nonprogrammer parents who is now a computer science major, who first learned
programming through a CTD course in programming taught by T. J. Leone

<http://tjleone.com/>

(who now offers a different course, based on different materials) and the EPGY
distance learning course in C programming

<http://epgy.stanford.edu/courses/cs/>

and some more or less haphazard local classes in programming arranged by our
homeschooling support group, which includes parents who are programmers by
occupation. He was advanced in mathematics before he began his programming
studies.)

------
tylerritchie
I recall reading about a doctoral student or new professor who was teaching an
intro computer science course and the whole while all of his peers were
telling him that his course was too difficult for the dolts he had as
students. (I can't find the article with what little I remember) By having his
students complete much harder, smaller homework than normal (I think?) he was
able to push his students farther in the one term than most would be able to
get in several.

As the final exam approached the instructors asked his peers to go over the
exam he had written up to which they replied something like, "you can't give
this to your class, it's too hard and they will all fail and that will make
you look bad." The instructor was worried, decided his students could hack it
and gave them the exam despite of peer unease. Upon grading the exam, a higher
than "normal" proportion (as measured from years previous) of the class left
with higher grades and greater grasp of material.

Now hopefully someone knows what I'm talking about so I can re-read that
account.

------
lifeisstillgood
I am willing to bet a largeish sum of money that had such a test been around
for literacy sometime in 1870 when mass education in the uk started in
earnest, we would have found a double hump of people who had a consistent
mental model of sentence construction, story arcs, past and future tenses.

The fact almost everyone has same mental model now is down to very early age
education. When your parents are literate you will be too.

It's possible this test could be done today in countries with terribly low
rates of literacy

------
neilk
I agree that there probably is a sharp distinction between people who can keep
very rigid rules in their head, and project forwards to anticipate what will
happen, and lastly also reason backwards from a desired state to figure out
the necessary instructions.

The question is whether this can be taught. When I read the study I see
another possible conclusion; that we haven't sufficiently prepared everyone
for the basic reality of computing -- that this is about arbitrary machine
rules, and how to think about them.

Some of us may come into the world having that 'mechanistic' worldview, or
perhaps some early experiences teach it to some earlier than others.

But we're evolved to deal with organic environments. Where precise definition
and full knowledge aren't possible, so instead we have fuzzier categories and
flexible responses. Where everything intelligent in the environment can be
negotiated with, using empathy. In programming 101 you have to abandon that
mindset.

------
leoedin
I had a Matlab programming class in my first year of a mechanical engineering
degree. What was painfully obvious to me then was that the people with
programming experience (some of us had done some sort of computing or computer
science course at school) excelled - I got 100% in the course, many of my
friends got 95%+ (which in the UK, where 80% is considered excellent, is a
very good mark).

Many others really, really floundered. They had no clue what they were doing.
I finished the task in about a 1/3 of the time, others were working right up
to the deadline and ended up getting 50% or so.

I'm not an excellent programmer. My skills might be above the average in
mechanical engineering, but that really isn't saying much. I'd bet that double
hump is less to do with some innate ability and more to do with exposure to
programming at a younger age.

------
sunils34
It's really all about how you teach programming. My first exposure to
programming was in high school where a math teacher who had never coded a day
in her life was thrown into the position for some extra cash. I witnessed the
dichotomy first hand here--and I was clearly in the wrong bell-curve. But that
summer, I attended a summer program where it was taught properly. First
explaining the context and history of programming, and then exposing us to
real world applications. This had a huge rate of success for everyone in that
program.

I don't think that the goats will ever go away, but I do think that if taught
properly, that the ratio of people who understand to those who don't can be on
the order of someone who understands an art history class to someone who
doesn't-- or any other area of specialization for that matter.

~~~
jpastika
Similarly, my first exposure to programming was in a high school AP (advanced
placement/college credit) class taught by a math teacher. The teacher would
give us an assignment and then leave the room for the rest of the period. As a
group we would come up with an answer in about 5 minutes and then spend the
rest of the time playing games on the class computers. His teaching method
wasn't exactly effective for me, and needless to say I didn't learn a whole
lot. I ended up getting a 1 (on a scale of 1-5, 1 being terrible) on the exam.
The way I ended up learning and becoming infatuated with programming was with
my TI-82 calculator. By trial and error I would write programs for games,
class notes (hi-tech cheat sheets) and animations. In college I began taking
"real" programming classes, but found the teaching to be both dated and
lacking real-world application. I kept learning through side projects though
and now, 12 years later, I realize that in this vocation, there is never an
end to learning and that people can learn in so many different ways.

I think the misconception some teachers have is that if a student is
struggling, he or she is either incapable of learning the material or just not
trying. The reality is that it could just be a reflection of how effective the
teacher's methods are for that particular student.

------
apu
From a comment I made 2 years ago when this paper was brought up:

 _I've been fairly skeptical of this paper ever since reading Alan Kay's
rebuttal:<http://www.secretgeek.net/camel_kay.asp>

I trust Kay much more than these authors on matters of computer science,
teaching, or ideas in general._

To expand a bit, I've since mentored over a dozen students and I've found that
in most cases of students who couldn't program well, the issue was "further
upstream." That is, there was some misunderstanding at a more fundamental
level that was preventing them from continuing forward. Once that issue was
fixed, they rapidly catch up to where they should be.

------
drumdance
FWIW I remember having a hard time understanding equations when I first
learned them in algebra. I remember that most of the class argued with the
teacher about it. Eventually I _was_ able to grok it, of course, but the
average person doesn't get much beyond algebra.

Also, assignment is rather strange concept relative to math education. I don't
remember any test where we had to create an algorithm. Variables where things
you solved for, not tools.

------
Simucal
Eh, that test seemed like it could be unintuitive to those that have never had
programming exposure before.

How are they to know that "=" is assignment and not testing equality? I guess
you could glean that from the context where they immediately ask you the
values of the variables after the operation but I could see someone
misinterpreting the semantics of that operator.

~~~
msbarnett
The point of the test wasn't to determine whether or not people who had never
been exposed to programming could correctly divine what operation '=' performs
in C.

Rather, the idea was to see whether or not, over the course of 20 or so
questions, their answers indicated that, whatever they had decided '=' did,
right or wrong, they applied their conclusion in a consistent manner.

Thus it was irrelevant whether or not they got the semantics of the operator
right; what was important was determining whether or not they had operated as
though operators in general _have_ semantics, or whether they operated as
though things behave essentially randomly.

It's more or less an attempt to determine whether the test-taker behaves as
though the universe is a rational and understandable place or not.

~~~
_delirium
This is sometimes called "procedural literacy" or "operational literacy"
([http://grandtextauto.org/2004/06/02/procedural-literacy-
an-i...](http://grandtextauto.org/2004/06/02/procedural-literacy-an-idea-
whose-time-has-come-43-years-ago/)), a sort of meta-programming-literacy that
isn't literacy in any specific programming language (or even necessarily
programming at all), but rather the ability to reason about how systems behave
in rational, programmed manners.

------
pnathan
I've worked with freshman CS students.

The aggregate breaks down like this, loosely:

20% succeed, always. Hackers/nerds, basically.

80% might succeed. They don't have the talent. They succeed or fail on
teacher's competence and their own hard work and background.

20% fail, always. Cheaters, people who are constitutionally incapable of
getting it, misplaced students, people with health issues, etc.

The 80% is what a teacher can affect. If they don't have the thinking
(math/logic) background or the curiosity, they probably won't do well.

Teaching is hard. It's probably the hardest task I've ever done.

~~~
hornbaker
Harder than percentages?

Sorry, couldn't resist. :)

~~~
pnathan
Yep! Type in haste, repent at leisure.

10/80/10 would be how I would break it down.

------
lnanek
From the article: >To write a computer program you have to come to terms with
this, to accept that whatever you might want the program to mean, the machine
will blindly follow its meaningless rules and come to some meaningless
conclusion.

I guess it is reasonable that people have a hard time with this. They've grown
up their whole lives dealing with people and animals and themselves. These act
according to wants and gloss over minor errors, a fuzzy world full of fuzzy
actors. Heck, even our species has lived and evolved mostly dealing with such.
It doesn't make much sense to them that one typo could cause a super powerful
computer or expensive device to keel over and be useless and do nothing.

------
drblast

      int a = 10;
      int b = 20;
      a = b;
    
      The new values of a and b are:  False!
    

All that example would really measure is who has had previous programming
experience.

This is obviously a teaching failure.

I don't accept that there are certain things people "just don't get." Watch
the documentary "Waiting for Superman," about some heroic teachers who didn't
accept that either and did something about it.

~~~
cbr
> All that example would really measure is who has had previous programming
> experience.

This test is intended only for people with no previous programming experience.
From the 2006 paper: "none had any previous contact with programming, and that
all had enough school mathematics to make the equality sign familiar".

------
vectorpush
In my experience, the biggest barrier to students grasping CS concepts has
been an aversion to parsing errors. I've seen students slowly melt down as
errors and warnings thrown up by the compiler/interpreter slowly chip away at
their sanity. It's almost as if these students take it as a personal insult
that the computer isn't behaving as expected and their snowballing
frustrations preclude any opportunity for them to learn. The problem only gets
worse as the class progresses because the topics grow in complexity but the
student's strategy for dealing with problems
(frustration/agitation/resentment) remains ineffective.

------
hluska
In 1995, I used to tutor my classmates in a first year computer science
course. I noticed that the people I tutored always had a bad attitude towards
programming. I used to hear (and have to deal with) objections like, "this is
stupid" or "this is boring"...

In retrospect, I don't know why I put up with all of that. I think it was 18
year old Greg's way of meeting girls...:)

I wonder if attitude towards programming is responsible for the distribution?
Seems to me that people either love programming or loathe it - there isn't a
whole lot of middle ground.

------
Groxx
From the draft article:

> _There was an attempt to administer the test to another 30 or so students on
> a foundation (pre- degree) programming course at Middlesex University. That
> administration failed, because the students – rightly, in our opinion – were
> incensed at the conduct of their teaching and the ar- rangements for their
> study, and simply refused to do anything that wasn’t directly beneficial to
> themselves._

Uh... what? What is the "conduct of their teaching and arrangements for their
study"?

That random bit of information is followed later by this:

> _3.3 Third administration

Because of what we found on the first and second administrations, there was no
third administration._

That doesn't follow _at all_. The first quote states the students _refused_ to
take the test, while the second states a final test would have been
_valueless_. I very much doubt that it wouldn't have had value, as it could
strengthen or weaken their claims, so I'm forced to conclude the students
refused to take an exam? Something doesn't seem right in this classroom.

Ultimately though, without any information about _how_ things were taught, I'm
not sure this study tells us much. With a _lot_ of studies across a variety of
teaching styles we might be able to extrapolate something, but a single one
just throws doubt on the methods of teaching which is a _massive_ un-
controlled and un-known variable to the whole system.

------
tsunamifury
The single concept that kept me from understanding basic coding was the
mutability of variables.

After years and years of algebra teaching me that variables were inherently
immutable, I automatically assumed that was a universal law that was never
broken.

Then again that proves that I at least thought there was a system.

~~~
polymatter
too bad they didn't introduce functional programming to you then. because in a
language like Haskell, pure variables are inherently immutable.

------
zerohp
My guess is that some students never really let go of mathematics. Without
this step, you cannot even begin to form a mental model of how the machine
evaluates an imperative program.

Students with no experience programming may be better off starting with a
language like Haskell.

------
chl
"Two years ago we appeared to have discovered an exciting and enigmatic new
predictor of success in a first programming course. We now report that after
six experiments, involving more than 500 students at six institutions in three
countries, the predictive effect of our test has failed to live up to that
early promise. We discuss the strength of the effects that have been observed
and the reasons for some apparent failures of prediction."

<http://www.eis.mdx.ac.uk/research/PhDArea/saeed/paper3.pdf>

More:

<http://www.eis.mdx.ac.uk/research/PhDArea/saeed/>

------
MikeMacMan
This makes sense. I majored in Japanese in college. There were 3 kinds of
students in our language classes:

1\. Students for whom it came naturally 2\. Students who could be competent if
they worked really hard 3\. Students who, no matter how hard they tried,
weren't going to learn to speak Japanese. Wasn't gonna happen.

Bad teaching methods might result in some #2 types being mischaracterized as
hopeless #3's, but I think no amount of pedagogical innovation can overcome
the fact that some people just can't do certain things. (Of course, it could
be a function of age and brain wiring, since all citizens of Japan, even the
stupid ones, have no trouble learning Japanese.)

------
sicxu
Everybody learns natrual language, yet large number of people cannot learn
programming language. It sounds like learning programming language is harder
than learning natrual language. This is not intuitive to me. A lot of concept
in natrual language are much more abstract and difficult to understand than
the assignment concept. Even the leap from one apple, two apple to one, two is
a much bigger conceptual leap. I see the story as a failure in education.

------
Kilimanjaro
If you don't know anything about programming then it is ok to assume a = b as
a comparison, so the answer is False?, or that a can be assigned to b? left to
right? right to left? who knows? how about assigning just adds the new value
to the existing one and removes the assigned value? so a=30 and b=0? What's
this int about, interrupt? interfere, interrogate?

This is just retarded. Tell me the rules first so I can infer on right
premises.

~~~
ertdfgcb
Well, part of what he said in the article was that it was more important that
the student can create a consistent mental model. So even if you have no idea
what the '=' means, you can infer a possible meaning from the answers and use
that every time you see it (on that test).

------
Shebanator
Weird to see a blog post from 2006 show up here. Especially when I read it and
discovered I'd already commented on it...

------
cabalamat
Suirely the correct answer is "it's impossible to tell with the information
given me".

------
NegativeOne
What are the possible answers ? Both values equaling 20 sounds right to me,
and I can't come up with anything else that it could mean.

------
jessepollak
Does anyone know where I might find the actual test that they administered?
I'd love to see if I could produce similar results..

------
user24
can a moderator remove the "coding horror" prefix from the headline? It adds
nothing but brand.

------
kghose
May I know where these folks teach, so I may never send my children there. I
expect teachers to teach new concepts. I expect them to gradually train an
untrained mind.

This test merely indicates that teachers, by tossing out untrained minds and
using a subset of already trained minds, can skip the challenging part of
their job and take credit where none is really due.

Grrr.

------
dpkendal
Aside: I think it would be better were the animals the other way around: sheep
aren't very smart (they have a tendency to start following one another for no
real reason and this can quickly build up into a stampede) but goats are
intelligent, curious creatures, much like programmers.

------
aklemm
I passed the test. w00t!

------
michaelochurch
I alluded to this (believing it was true) and was rebuffed:
<http://news.ycombinator.com/item?id=3434582> . Doesn't seem to be the case.

I think that many more people can learn how to program, and become _good_
programmers at that. It's true now that the majority (at least 80%) of
programmers are incompetent, but I think that has more to do with environment,
tooling, economics, and the short half-life (5-6 years) of a software
developer, than intrinsic limitations on ability. I wrote about this here:
[http://michaelochurch.wordpress.com/2012/01/26/the-
trajector...](http://michaelochurch.wordpress.com/2012/01/26/the-trajectory-
of-a-software-engineer-and-where-it-all-goes-wrong/) . The tools and freedom
that a developer needs to become good (1.5+) are simply not available in
average software jobs.

I think the big problem is that we teach programming using languages like
Java, which are actually very complicated. There's a lot of information in
"public static void main(String[] args)" that just seems like line noise to a
novice, and that people shouldn't even be exposed to until they develop the
taste to know when to use OOP and when not (and that takes years, IMO). In
teaching, we should start with simple languages like Scheme so we aren't
requiring people to lex and parse and generate .class files in their heads
before they even know what those terms mean.

Scheme and Python are better starting languages, in my opinion, and we should
be starting programming education around age 6. We can (and should) expose
growing programmers to C, Scala, Clojure, and Haskell later.

Also, starting with complex languages and tools (Java-style) means we have to
teach in an inverted way. In your first Java exercise, you don't write a
_program_ , you write a _class_. Why? No good reason. There's no intrinsic
reason a program has to be a _class_. This sort of botched, improperly-coupled
teaching produces a generation of programmers who think every program must be
a class, and who produce shitty AbstractVisitorHandlerFactory programs as a
result.

This is one thing that annoys me especially in the teaching of Scala-- one of
the most exciting languages to come out of the past 10 years, but one that is
poorly understood and generally taught quite badly. What angers me especially
(I'm writing a Scala tutorial to fix some of these mistakes, probably making
others in the process) is when case classes are taught _after_ classes. For
example, the O'Reilly book (which is generally quite good) covers _traits_ (a
very powerful and highly nuanced advanced OOP feature) in Chapter 4 and _case
classes_ in Chapter 6. Case classes, for the uninitiated, are immutable
records (data objects) and provide a lot of niceties (automatic .equals and
.hashCode, class name as constructor) for free. Since case classes are simpler
than full-featured classes, they should be taught _first_.

The right way to teach programming is to teach the simple stuff (immutable
records, referentially-transparent functions, mutable state in _simple_ data
structures) first, and _then_ provide some insight into when and why you might
want to use the more complicated stuff (OOP, type classes in Haskell, functors
in OCaml).

This is the reason why people who start in functional programming usually
become top-5% programmers within ~3 years and people who drink the OOP Kool-
Aid almost never do. Mutable state isn't evil, and good "functional"
programmers use it all the time, but if you don't start with the right
(simple) _default_ abstractions-- immutable records and referentially-
transparent functions-- you are not learning how to write clean, simple,
maintainable code. Not only is this an important engineering skill on its own,
but it's also important for the purpose of growth, because reading good code
(one's own and others') is a great way to improve as a programmer.

------
its_so_on
if I had never had any exposure to programming, and you told me that a equals
ten, b equals twenty, and a equals b, I would say that one of these three
equations is obviously false (the system of three equations has no solution),
you're full of shit, and probably made a mistake and expect me to put 0,0 for
a and b respectively. Or I would guess randomly or ignore the question.

IF I HAD NEVER HAD ANY EXPOSURE TO PROGRAMMING.

------
paulhauggis
it's funny. I met a guy that had a PHD in computer science. He not only didn't
really know how to develop software, but he didn't know anything about the
internal makings of a computer (CPU, HD, etc).

------
snitzr
// The solution test in JavaScript

var a = 10, b = 20; // Declare variable values

a = b; // Assign b to a

console.log(a); // Output a to the console

console.log(b); // Output b to the console

// Log: 20

// Log: 20

