
Can a simple algebra test predict programming aptitude? - dennyferra
http://www.codeup.com/can-a-simple-algebra-test-predict-programming-aptitude/
======
bitplanets
Why do you care about it in the first place? Also what is your definition of
good programmer? Just make a function? Make a class? Make a big app?

IMO you don't need to learn too much math before programming to be a good
programmer. Basic operations are good enough, if you need more you learn as
you go, you'll never know what you need. Lazy learning, like lazy variable
loading.

The test you are making are more like problem solving which is closer to what
programming is and not math. Of course basic math (+, -, /, *), but what it
really matters is knowing what operations to do. The operation itself is the
easier part.

A better test would be to make a chose between which approach to take to solve
a problem and explain the rational behind. That is what will define a good
programmer.

Also what better indicator you want than programming itself? You'll never make
a choice based on this.

Also one thing is to figure out how to solve "here are 3 consecutive integers
with a sum of 69. What are they?" than solving "here are X consecutive
integers with a sum of Y. What are they?". Same in "Adriana’s age is 1/3rd of
her dad’s age. If her dad is 36 years old, how old is Adriana?" vs "W’s age is
X of Y’s age. If Y is Z years old, how old is W?". The complexity increases a
lot.

~~~
the_af
I disagree with many of your points.

First, the algorithmic part of programming is a lot like doing algebra. So
much so that algebra is an important part of any serious CS syllabus. Clearly
"writing a function or a class" is not the skill they are testing here, but
problem-solving.

Knowing how to solve simple algebraic problems engages your "problem solving"
skills in a similar way as solving something by writing a program. Note that
algebra is not "basic math (+,-,/,*)" like you said. Algebra, like
programming, requires the ability to understand and write abstractions, and to
figure out how to approach a problem.

> Also what better indicator you want than programming itself?

They clearly want to establish a correlation between something else and
programming. This is interesting in itself.

~~~
seanp2k2
IMO, what algebra and programming have in common is simply abstraction, so
people in the mindset of recognizing patterns to which abstraction can be
applied tend to do better as programmers / automation engineers in general.

~~~
sklogic
Algebra and programming are very, very similar. Algebra [1] deals with formal
languages and their rewrite rules in general. Programming is dealing with a
certain (but yet very wide) class of formal languages and an extremely wide
variety of rewrite rules (i.e., algebras).

It does not matter that nobody is using, say, a term "refactoring algebra",
but it is still, essentially, an algebra.

[1] Here I'm referring to a generic meaning of "algebra", as in
[https://en.wikipedia.org/wiki/Universal_algebra](https://en.wikipedia.org/wiki/Universal_algebra)

------
mathattack
For an article about math, they seem surprisingly weak at differentiating
between causation and correlation.

 _It’s fairly common for incoming Computer Science majors to ask the question,
“Why do I have to learn all this math if I just want to learn to program?” The
correlation above suggests a possible answer: The ability to understand basic
mathematics is likely correlated with the ability to “think algorithmically,”
which is well-known to be a foundational skill for expert programmers._

If it's just a weak (as the chart suggests) correlation, it doesn't mean
learning one will make you better at the other. And if you do assume causality
- it can go either way. Perhaps learning math makes you better at programming.
Perhaps learning programming makes you better at math.

My 2 cents... It's more complicated because there are other things involved.
Perhaps it's the nerd gene that makes people who like computers also more
likely to play D&D and be in band. (I was a card carrying member) Does one of
these 5 variables cause the other, or are they all part of the same thing?

~~~
walshemj
Should if you are going into a CE or CS course not have learnt to code at
school before going to university English lit or classics students are
expected to be able to read.

I learnt to program when I was 13 and that was in the middle stream at my
upper school

~~~
kasey_junk
This is highly dependent on your opportunity. As hard as it is to find
qualified math and english teachers prior to the college level, think how hard
it would be to find programming instruction.

That is to say, when I was in school before university there was no choice for
studying programming _.

_ There was but it was offered at a private religious school/camp, not via the
public schools.

~~~
walshemj
This was in a comprehensive i.e. a bog standard school and was in the middle
i.e. vocational stream - possibly the approach of having specialisation at the
middle and upper school works better for feeding children into UNI/Vocational
track.

~~~
kasey_junk
Right, so the bog standard schools you had the opportunity to attend offered
something mine (and many others) do not. As someone who is always looking for
good developers I certainly don't want to limit the number of people who can
learn CS to only those who happened to live somewhere that offered programming
at the public school level.

~~~
walshemj
possibly not fetishizing CS over a more useful general introduction to
programming might help.

I haven't had to do any of the big O stuff in anger but I have had to make
sure a major telcos billing system reconciled.

------
Morgawr
If there is one thing that has always bothered me, is people evaluating
student's prowess in "Computer Science" (which is not programming, shame on
you) by having them learn your normal run-of-the-mill programming
language/paradigm like Javascript, Python, PHP and whatnot and then claim they
are not apt to program because they cannot solve problems or handle variables,
mutability, etc etc.

There are A LOT of different programming/automation paradigms in the world,
lots of different languages, declarative, functional, OOP, imperative, logic,
etc etc, maybe some of these students just can't get the convoluted mutable
and unsafe structure of a Python or Java program but they might grok and
appreciate the safety and simplicity of ML/Haskell or the flexibility of Lisp,
why would you turn them down in such a way or force them through a path they
don't really need to go through?

During my first year of Bachelor Computer Science at University we had
introduction to programming in Scheme. It was a mind-expanding experience to
me, I already knew how to program in C, Java, C++ and C# with a bit of
PHP/Javascript back then and I was much ahead most of my peers but still I sat
at the computer, trying to earn this alien language to me, and luckily I was
humble enough to force myself through it and adapt my mind to a different
paradigm. I know for certain that a lot of my friends and colleagues who
already knew how to program struggled hard on that exam and some had to re-
take it the following year (Which was unfortunately changed to C++ and they
managed to pass without troubles) simply because they didn't think Scheme was
a "real" programming language or useful at all. It was just too different from
their previous experience.

What's even more interesting, I know some people from that class who had never
touched a programming language before and weren't particularly strong at math.
Those people are the ones I recall enjoying the course the most, they found it
the easiest among all the other courses we had and passed it with excellent
grades. Simply because their mind was apparently better wired for such
paradigm and they had no preconceptions or prejudices that prevented them from
learning it properly.

So my bottom line is, what makes you think that some people might not just
have a "differently wired" brain that makes them think more easily with a
different non-imperative paradigm for programming?

~~~
guard-of-terra
Recursion is as hard as mutable state to grasp if not harder. We might find a
few people who can do one thing sligtly better than the other one, but not
complete blanks.

~~~
rebootthesystem
In 35 years of programming across platforms and languages covering embedded to
web applications it is my opinion that recursion is dangerous and useless.
There is no way I would use recursion to evaluate how good a programmer might
be. To quote the movie: "Frankly dear, I don't give a damn".

~~~
mytochar
Why?

~~~
rebootthesystem
Just in case this is what you were trying to understand:

[https://www.google.com/search?q=recursion+is+dangerous&rlz=1...](https://www.google.com/search?q=recursion+is+dangerous&rlz=1C9BKJA_enUS590US591&oq=recursion+is+danger&aqs=chrome.1.69i57j0&sourceid=chrome-
mobile&espv=1&ie=UTF-8&hl=en-US)

[http://m.slashdot.org/story/198499](http://m.slashdot.org/story/198499)

You can certainly dig up more information. For me, and this is just my opinion
based on 35 years of software and hardware engineering, recursion is a neat
academic idea but in the real world it is useless and dangerous.

~~~
mytochar
I'm seeing stack overflows (not a problem with tail-call optimization) as the
biggest problem here. There are equally dangerous problems, though. Buffer
overflows are a very real concern, especially in embedded software; but that
doesn't mean that people don't ever use pointers. Everything about software
has risks. Yes, they probably shouldn't have used recursion there in that car;
but honestly, it's just a stack. Enough calls and some state could have been
frozen. They should have put mitigations in place to stop the potential for a
stack overflow to have actual, detrimental results. Do you refuse to ever make
a function call for fear that you might be at the top of the stack somehow?
After all, how can you know that you haven't accidentally created a state
machine with function calls and are at the peak of it now, and end up causing
the same problem. Recursion may have been the final manifestation, but it's
certainly not the only place that failed.

I argue against your thought that recursion is useless, as well. It's a
natural and intuitive solution for a lot of problems. The very concept of a
dynamic programming solution starts with a recurrence relationship. It starts
with recursion. Sorting algorithms are intutively recursive in many cases,
Quicksort and Mergesort are just two examples.

And, 'dangerous' is an interesting term. I understand you've worked in
embedded systems and hardware, and so those things are often critical systems
that have to be super secure and completely bug free. Not every system is like
that, though.

I want to understand why you believe that recursion is both useless and
dangerous. Could you give some broad examples of the dangers of recursion from
your years of experience?

~~~
Retric
It's trivial to blow the stack even on languages with tail call optimization.

~~~
sklogic
Ok, try to blow the stack in a stackless language implementation.

~~~
Retric
F(x) = F(x) + F(x)

Eventually you hit OOM or some death limit even on a stackless language.

~~~
sklogic
This is a memory leak now, not a stack overflow.

~~~
Retric
Thus demonstrating a finite stack. "Stackless" just a more efficent way to use
memory. You could get the same things from dynamicly allocating and
dealocating stack space as needed.

Put another way, allocating 1GB of stack space on a machine with 2GB of RAM is
at worst the equivelent of using a stackless lanugage on a machine limited 1
GB of RAM.

PS: It's actually worse as stackless languages are stricly slower than
actually having the equivelent stack space pre alocated and you need to put an
pointer for each stack call pluss overhead for alocating memory.

------
Animats
The scatter plots don't show that strong a correlation.

From the test questions, this isn't an "algebra test". It's a word problem
decoding test. That's appropriate to programming, where you have to go from an
informal specification of the problem to a formal one.

~~~
xivzgrev
Yes it is algebra. Take second problem for instance.

X + (x + 1) + (x+ 2) = 69. Solve for x.

~~~
ludoo
Or just recognize that three _consecutive_ numbers mean you divide by three
and get the median of the three numbers. So 69/3-1, 69/3, 69/3+1, giving 22,
23, 24. No algebra involved, you just need to interpret the problem and build
a tiny model in your head. Exactly like (some kinds of) programming.

~~~
rudolf0
That's exactly how I solved it. I'd consider myself a pretty decent
programmer, but I did poorly in algebra and calculus at middle, high school,
and university level.

Give me "x + (x + 1) + (x + 2) = 69" and I wouldn't know where to start. I
could probably turn that word problem into that equation, but I would have no
idea how to actually solve the equation algebraically.

But given the word problem alone I knew to divide by 3 and add the neighboring
numbers. Now, actually dividing by 3 I'd need to use a calculator or spend a
lot of time with a pencil and paper, but at least I knew the steps to solve
it.

I'm awful at rote math and mental math, which is why I'm thankful I can make
the computer do it for me.

~~~
rverghes
Are you saying that if the word problem was "2 consecutive numbers and a third
number that was five higher than the second number" you'd be unable to solve
it?

"Tricks" work for simple cases. Algebra works for all cases.

~~~
lostcolony
Yeah, I would be unable to solve it, given that I'm pretty sure there is no
solution if we assume that "consecutive" implies integers (and I'd be curious
what 'consecutive' means if we don't).

If instead we made it "2 consecutive numbers and a third number that was five
higher than the second number add up to 70" (i.e., 21 + 22 + 27), that's
easily solved using 'tricks'. Really, the trick mention (which is what I did
in my head, too) is just a rephrasing of the algebra. That is, I would
subtract 4 off of the thing that is 5 higher (so that it's now 1 higher; the
problem is now the same, get 3 consecutive integers), subtract 4 off of the
number I'm trying to get (so 66), so the problem is now 3 consecutive integers
that equal 66, and solve the same way (66/3 = 22, so -> 21, 22, 23), and then
just add the four back in to the highest (21, 22, 27).

~~~
rverghes
You're right, my mistake. That "five higher" should have been "four higher".

The thing is that your solution _is_ the algebraic solution. You're
simplifying an equation by balancing both sides, when you subtract four. Just
from our perspective, you simplify to a still difficult state, instead of the
easiest possible state.

Algebra is just a way of formally stating what you did, and then offering some
simplifications that speed up the process. Or offering more powerful methods
that making solving more difficult problems easier.

~~~
lostcolony
'The thing is that your solution is the algebraic solution'

I agree. That's why I said

'Really, the trick mention(ed) ... is just a rephrasing of the algebra'

Just because you do number juggling in your head rather than write it
formulaically doesn't make it an inferior technique, or make it not algebra,
which was my point.

------
Paul_S
Does a "simple algebra test" correlate with IQ? Does "programming aptitude"
(whatever the hell that is meant to be) correlate with IQ? The headline would
then follow.

I know this is a socially unacceptable opinion but I think those correlations
exist.

~~~
mgirdley
Codeup CEO here. We actually did some brief testing with classical IQ tests
and saw two problems:

1) IQ tests are really long and expensive, so difficult to implement

2) They didn't correlate with performance as well as algebra.

~~~
Paul_S
I've read your answer about your methodology of gauging "programming aptitude"
and I'm worried it might be susceptible to your judges' biases - it being
purely subjective. I'll put more faith in your point number 2 if you come up
with a more objective measure.

~~~
mgirdley
Agreed that an objective measurement would be ideal and this is second best.

However, we've had a hard time creating or locating an objective measurement
of general programming skill. As demonstrated by the industry's non-use of
standardized tests for programming, I don't think anyone else has either.

If you have a lead on something, would love to see it.

~~~
strict9
Trying to create an accurate measurement of programming skill is far more
difficult than looking for correlation on tests compared with some other
subjective measure.

This misguided attempt at instrumenting "skill" could prevent highly capable
individuals from entering the field who don't have a solid math foundation,
for societal or other reasons.

~~~
sdenton4
OTOH, identifying a decent mathematical thinking background as a predictor of
CS potential is a pretty big win at a societal level. Teaching math is
relatively cheap; you don't even have to buy computers...

------
j_baker
This article is built on a fundamentally invalid premise:

> 30-60% of CS college majors have failed their Introduction to Computer
> Science course because they simply could not learn to program.

This "You either have it or you don't" mindset is strikingly elitist. I simply
don't buy the idea that there are people out there who absolutely cannot learn
to program. I mean, put someone on a desert island and tell them they can't
leave until they can write a program that uses a linked list, and I'm pretty
sure most people would be able to get off the island eventually.

Of course, that doesn't mean that some people can learn to code more easily
than others. But I'd be willing to posit that the vast majority of people
could learn to code given enough time and the proper instruction.

~~~
mytochar
This StackExchange seems to have a pretty good overview on this very question:
[http://programmers.stackexchange.com/questions/163631/has-
no...](http://programmers.stackexchange.com/questions/163631/has-not-everyone-
can-be-a-programmer-been-studied)

There is a No True Scotsman argument that arives here "Well, if someone later
became a programmer, despite failing those tests earlier, then clearly they
were skilled at becoming a programmer" blah blah; but it is still an
interesting phenomenon. And yes, there will be outliers, of course; and there
will be people that gradually mull their brain to change the way it thinks;
but, it is still an interesting phenomenon.

I bet we could see this in other fields, too, it's just that programming is
presently the hotness.

(edit: added more)

As for elitism. What is elitist about it if that /is/ true? Oh no, Billy can't
program, but he's great with cars. Sarah sucks at linked lists, but she has a
better understanding of the human body than any of her peers in the ICU. And
so on.

------
mentos
I am not great at higher level Math (calculus) and I'm good with algebra but I
do not think I'd consider those good predictors. I feel like what allows me to
program is strong spatial reasoning. Doing a quick google search I found this:
[http://www.quora.com/Whats-the-nature-of-the-relationship-
if...](http://www.quora.com/Whats-the-nature-of-the-relationship-if-any-
between-high-spatial-reasoning-skills-and-programming-aptitude)

Probably just anecdotal but curious to know what others feel about spatial
reasoning? I think specially with object oriented programming it is helpful to
be able to visualize all of the 'actors' in your head and how they
relate/intertwine.

~~~
hackinthebochs
I would also agree that spatial reasoning is key to strong programming
ability. Most data structures are best understood through spatial reasoning,
for example. But I would also say that spatial reasoning is key to
mathematical ability. I'm actually surprised that you would consider yourself
strong with spatial reasoning but had trouble with calculus. Perhaps you just
had a bad teacher or lacked proper motivation?

~~~
kasey_junk
I'm terrible at spatial reasoning but have no problem with formal logic. I'd
also like to think I have strong programming ability (of course we don't know
how to measure that).

Spatial reasoning is a key mathematical ability for certain classes of
problems (a big chunk of calculus as you mention) but helps not at all with
another big class of mathematics (set theory for instance).

Finally, to you data structures are best understood via spatial reasoning, but
there is nothing actually spatial about data structures so that is probably
most likely just your own preference for modeling them.

~~~
hackinthebochs
I'm sure my bias plays a role, but I would disagree that there's nothing
spatial about data structures. Linked lists, trees, heaps, etc, all have the
characteristic property of a geometric interpretation: a natural notion of
"distance" inherent in their definitions.

~~~
walshemj
Set Theory?

------
engrsrce
What? Where's the correlation? The data looks like a random scatter plot to
me. It reminds me of this Physics report from ages ago...

[http://pages.cs.wisc.edu/~kovar/hall.html](http://pages.cs.wisc.edu/~kovar/hall.html)

~~~
mark-r
I think you can deduce _some_ correlation just by noting that the upper left
and lower right parts of the graphs are relatively empty. Certainly there are
statistical tests that would be more concrete. You can see some figures in the
bottom right corners, but they're never discussed in the text.

------
mattlogan1
An R^2 value of 0.33 does not even come close implying linear correlation.

~~~
lvs
This is the most important comment in this dramatically overwrought thread.
The post's own data analysis refutes its hypothesis unambiguously.

------
iliketosleep
Basic algerbra is used all the time in programming. It's part of a
programmer's fundamental skill set. Of course, somebody who already has that
particular skill will definitely have the advantage in a short programming
course. It's like starting a 100m sprint with a 3 second head start. It says
nothing, however, about who will be the better programmer over the long term.
So I don't think their test is a good predictor of programming aptitude.

------
tokenadult
I'll reply to a specific comment already made as a subcomment here, to comment
about the larger issues that have come up in several top-level replies in this
thread.

 _I don 't think such a hypothesis is awkward but I doubt high IQ as an
excellent predictor for an individuals performance in tasks that are not IQ
tests. As a layman when it comes to psychology I think IQ measures some things
but the way the brain works, no real world task a person does is exactly like
those IQ tests._

The interesting blog post submitted here is talking about the bread and butter
of "industrial and organizational psychology," namely about how to select
individuals for a training program. There are three generations of published
research on this topic already, and there is a huge amount of ongoing research
on this topic, because organizations all over the world want to figure out how
to select successful applicants when there are more applicants than places in
school or work programs.

The short answer is that there is a HUGE body of research to show that the
single best hiring process you can use for hiring a worker, if you want to get
a worker who will perform well on the job, is to use an IQ test for hiring.[1]
The long answer is that some other applicant characteristics matter too, of
course, but the single best thing to look at in a job applicant is "general
mental ability." Work-sample tests are also very good for hiring for specific
jobs, and are grossly underused in hiring in the United States.

To the point of the interesting submitted blog post, one always has to be
empirical about these issues. The people running the bootcamp so far have
found data that suggests that the algebra test they have tried is a bit more
revelatory than the IQ test they tried, and less expensive besides. One
response to that might be to suggest a test like the Wonderlic test (an
inexpensive IQ test designed for company hiring procedures) but in the end,
results matter. If empirically at this bootcamp, the algebra test works better
than some other selection procedure, it doesn't even really matter why it
works, just that it serves the bootcamp's purpose of identifying successful
students from among multiple applicants. The data set is still small. I am
very glad that the blog post includes a scatterplot of the data. More
bivariate data should be shown that way, in blog posts on dozens of topics.

[1] My FAQ post on company hiring procedures, which I am still revising to put
on my personal website after composing it for Hacker News, provides references
for this and some commentary on legal issues in hiring.

[https://news.ycombinator.com/item?id=4613413#up_4613543](https://news.ycombinator.com/item?id=4613413#up_4613543)

~~~
michaelt

      there is a HUGE body of research to show that the 
      single best hiring process you can use for hiring 
      a worker
    

The best process, assuming you have to use a single process for every job in
the world? Or the best process even when hiring for a specific role?

I can understand IQ being the best choice if you had to judge poets, plumbers,
town planners, golf instructors, programmers, salespeople and warehouse
workers using the same process. But surely if you're exclusively hiring for
one of those roles, you'd want to test their domain-specific knowledge?

~~~
lmz
As was written above:

> Work-sample tests are also very good for hiring for specific jobs, and are
> grossly underused in hiring in the United States.

~~~
michaelt
But isn't essentially all hiring for a specific job?

~~~
tokenadult
Industrial and organizational psychologists have researched this issue, with
hundreds of studies spanning most different kinds of jobs in most developed
countries now published. A work-sample test is especially strong for spotting
people who need to do something right away after they are hired. For any
worker who may need to learn new things on the job--and that is a lot of
different kinds of workers, in a lot of different industries, and almost all
managers--a "general mental ability" test adds to the predictive validity of
selection applicants who will do well on the job over time as industry
conditions and market pressures and technologies change.

------
analog31
I can't dispute the author's statistics, after all, they're just a function of
the data. But looking at the first graph, it seems like a generous
interpretation of "predict." The anticipated signal-to-noise for any
individual student looks like it would be pretty poor.

One other thought occurs to me. Would the same algebra test produce similar
correlations with courses such as English and History, when adjusted for the
differences in overall pass rates for those subjects?

It seems odd that something as interesting as programming isn't introduced
until college. That seems like accepting students as music majors, who have
never played music before. If I had my druthers, computation and simplistic
programming would be part of the mainstream K-12 curriculum.

------
pflats
>It’s fairly common for incoming Computer Science majors to ask the question,
“Why do I have to learn all this math if I just want to learn to program?”

Well, there's the reason you have a high drop-out rate in CS. People don't
know what it is! Computer Science is not a vocational program. Computer
Science is not computer programming.

If someone had talked to these kids asking “Why do I have to learn all this
math if I just want to learn to program?” and told them "You don't have to
learn all this math if you just want to learn to program" before they became
CS majors, then they might not have become a CS major in the first place.

~~~
Someone
And, likely, some good computer scientists would be lost to the world. You
should also ask them why they want to learn programming. Many of them will
answer they want to become programmers, while they do not have a clear picture
of what that entails (it is not code writing all the time, most programmers do
not write games in languages they chose, a lot, despite the fact that it is a
job where all workers can automate repetitive jobs, a lot of the work is
repetitive, etc)

Maybe, once they realize what is the difference between programming and
computer science, they prefer computer science. Problem is you cannot learn
them the difference in a few weeks; it takes years to sink in.

So, what do you let them do in the mean time? Waiting is a waste of the years
in which learning is easiest for them. So, do we let some grown-up decide who
likely will make a good computer scientist, or do we let many more start on
that trajectory and see how far they get?

I think the latter is the better choice, if we also provide smooth ways to
move from one to he other.

[slightly related: I once read a teacher in a nursing school state: _" when
they come in, all the boys want to ride an ambulance, and all the girls want
to work with kids. We have to work a bit on that in the first year"_]

------
wilsynet
Most programming jobs involve a lot of algebra. You have variables, you do
arithmetic to those variables, you apply functions. It's not a great surprise
to me that if you are good at high school algebra, you can probably learn how
to program.

You might not be a great programmer, but you'll be able to do it.

~~~
innguest
An algebra is just a system of rules.

Being good at number rules (school algebra) does not have much to do with
being good at state rules (code algebra).

I'd say there's more correlation between _functional_ programming and school
algebra, if anything.

~~~
the_af
Being good at a set of abstract formal rules is probably at least _some kind_
of indicator that you'll be good at another set of abstract formal rules.

~~~
innguest
Having recently taken a battery of psychological tests that measure several
different cognitive functions, I am one living example that that is not the
case.

~~~
the_af
How can you be an example that what I said "is not the case"? Are you saying
that because _you_ are not similarly good in two different sets of abstract
formal systems, that this isn't a good indicator _in general_?

------
michaelhoney
Lot of people in here offering cheap opinions, while mgirdley's team has done
the actual work. They're not saying that the kinda of algebra test they're
proposing is foolpoof, perfect, subjectivity-free, or even the best way of
testing for programming aptitude.

What they _are_ saying is that the test is an efficient, effective way to do
it, with better ROI than IQ tests.

It doesn't actually matter _why_ that's the case, though Animats' suggestion
seems plausible: "From the test questions, this isn't an "algebra test". It's
a word problem decoding test. That's appropriate to programming, where you
have to go from an informal specification of the problem to a formal one."

------
neaanopri
I think that many people have difficulty learning to code in college because
many of them aren't that enthusiastic about it. At least anecdotally, many
people at my school are in computer science because that's where the zeitgeist
is. If it was the eighties they'd be getting ready for law school.

I think that people would do much better in intro CS classes if we could get
rid of some of the hype around programming and tech in general, and treat it
like other courses. That way, far more of the students would be genuinely
interested in the subject.

------
valarauca1
I'm very good at Algebraic math. But I don't think an algebraic test will show
programming aptitude. While algebraic and calculus analysis are a form of
problem solving that works incredibly well. It is by no means the fastest, or
most eloquent way of solving a problem. Its just a single tool that a problem
solver can use.

The main thing I fall back on is Heron's Problem. The classic
algebraic/analytical solution is to find the local minimum of a function.
Which works, this method isn't flashy yet completely functional. It requires
no insight, just mentally vomiting something you learned in high
school/college.

The geometric solution is far simpler, and offers insights into the foundation
of trigonometry. Its really so simple and eloquent you feel like an brutish
idiot doing a long form analysis.

I offer this because tools are merely tools, and math supplies many tools. You
wouldn't hire a finish carpenter just based on his/her ability to swing a
sledge hammer would you? I mean s/he will have to swing a hammer, but will it
have to be a sledge hammer? Will they have to use a nail gun? Screw driver? I
hope they are proficient in all of these tools, and since I'm outsourcing my
time to them, I hope they know which tool to use at which time.

------
CGamesPlay
The correlation doesn't seem particularly strong, but the dependent variable
here isn't right. It's a stack ranking that's forced to a normal distribution.
A normal distribution might be appropriate if they randomly selected US
citizens to join the program, but they don't. Students self-select to apply
and they reject the bottom 66% of applicants. The resulting talent profile is
far from being normally distributed.

------
acscott
You can correlate most anything with most anything. I'm not saying don't try,
just to don't give correlations much credence in and of themselves. Thinking
globally, what are the odds a coincidence doesn't happen? And when you exclude
outliers, you may be excluding the most profound subjects, just as when you
include non-outliers, the subjects may not be able to generalize outside the
system of measurement.

------
vorg
> Despite hours of studying and tutoring, most of these underperforming
> students struggle with, and many ultimately give up on, programming as a
> career

They should follow the lead of those who figure out earlier they're not cut
out for programming: embellish the CV with fake study, and go straight into
the workforce as a programmer. Or they could get transferred into programming
from some user department pitching to bring a "valuable user perspective" to
IT. Or grease up some IT manager after-hours who'll bring them in as
contractor with "special skills not readily available in the labor market" to
bypass the usual HR checks and aptitude filters. Or if they're not of the same
ethnicity as the HR personnel, send in a double to sit the aptitude test for
them, knowing HR staff don't check applicant ID's too closely because they
know no-one's going to make it through to an IT interview if they do. Most HR
staff and IT managers just want the staff count up, they don't care whether
they're productive or detrimental to the projects. If they have staff they
have people to blame.

------
edtechdev
So I guess the real underlying message they are trying to say is, if a student
can't pass an algebra test, they probably can't learn to program, so don't
even bother trying to teach it to them. A more generalized form of the
hypothesis is that: most people can't learn how to program.

Both forms of the hypothesis are pretty easy to refute and have already been
refuted. See [http://code.org](http://code.org) and
[http://madewithcode.com](http://madewithcode.com) for example. I just taught
2nd graders programming yesterday who don't even know how to divide, let alone
any algebra.

And of course, there's always the radical idea that maybe you could actually
teach them some algebra skills in the context of programming tasks. Just like
some calculus students may need some remedial instruction and support.

Luckily, there's a whole field of research with several articles on this very
topic. The field is called computer science education. See SIGCSE, for
example: [http://www.sigcse.org/](http://www.sigcse.org/)

In fact, the field has already researched and debated this issue before, as
well. There was a controversial article called "The camel has two humps" in
2006 that made the same claim as this post that a simple aptitude test could
predict whether someone could learn how to program or not. The article (which
was never even officially published) was later retracted:
[http://retractionwatch.com/2014/07/18/the-camel-doesnt-
have-...](http://retractionwatch.com/2014/07/18/the-camel-doesnt-have-two-
humps-programming-aptitude-test-canned-for-overzealous-conclusion/)

~~~
mgirdley
I don't think that's the message.

We are focused on making people professional, employable programmers. While I
agree that everyone can play with scratch or build a basic webpage, that's
different than the programming skills required to be employed in most dev jobs
today.

(Codeup CEO here.)

------
ekm2
An older study suggests :

 _In both genders,performance on Mathematics was found to be the best
predictor in programming ability,followed by performance on a spatial test._

[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.110...](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.110.9842&rep=rep1&type=pdf)

------
tomrod
Surely the confidence bands in the plot aren't 95% confidence intervals...

[http://uhlbcmcbx4a4vt1d.zippykid.netdna-cdn.com/wp-
content/u...](http://uhlbcmcbx4a4vt1d.zippykid.netdna-cdn.com/wp-
content/uploads/2014/12/898x640xPredictProgrammingAptitude1.png.pagespeed.ic.KzRMOyK9kJ.png)

------
washedup
R-Squared of 0.33?? Not a great fit, considering the sample size.

~~~
rhiever
An R^2 of 0.33 is a fairly decent correlation for a study involving human
subjects. It's obviously not perfect, but the authors are not claiming that
it's perfect.

------
drcomputer
How is programming aptitude measured and defined?

~~~
mgirdley
Codeup CEO here: it's the instructors evaluating the students based on their
interactions with then and answering the question: how capable is this person
of building quality web applications?

~~~
palakchokshi
That is so subjective that it should be ignored. I've interviewed people who
interviewed very well but in practice they failed at basic computer
programming concepts and implementations.

~~~
kasey_junk
We simply do not have an objective measure for program quality and thus do not
have an objective measure for programming aptitude.

In the face of this problem we can either give up and not try to measure
anything, or we can use an obviously subjective measure and understand where
the weaknesses are.

------
ivan_ah
I'd be interested to see more questions from the test, but from the examples
given I'd say s/algebra test/problem solving test/ is in order.

In that light, the findings become less surprising---still interesting though,
since we can test math-word-problem-solving ability quite easily with pen and
paper. Could this mean my KhanAcademy math achievements are a better hiring
signal than my github?

In terms of hiring, it certainly makes more sense for candidates to pass a 1h
test rather than expect them to "prove themselves" by solving a week-long
coding challenge...

------
abandonliberty
[http://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines](http://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines)

~~~
nether
you're conflating that with graham's submarines, showing a mere correlation,
not causation.

------
anon4
_we’ve possibly found a simple method for sorting out the ninja programmers
from the less capable programmers_

Can we please stop calling them ninja/rockstar/etc programmers and simply use
"professional programmers" or "master developers"? This language is cute when
you're 16, I guess, but personally I don't want to be compared to a middle-
ages asian hitman for hire.

------
codezero
Aptitude tests really annoy me. At least for me, when I was told that I was
bad at math, it discouraged me and I didn't pursue it. Later when I was all
grown up and decided to go back to school, I really applied myself and it
turns out I was pretty good at physics and math, despite my aptitude when I
was a teen.

------
skywhopper
I'll only comment on the opening line and the research it points to about the
failure rate in intro to Computer Science. The author of the codeup piece
immediately jumps to the conclusion that it's an inability on the part of the
failing students to be able to learn to program (a detail that the study
linked doesn't comment on). However, my first reaction is that if the failure
rate is higher than we want, it means we either (1) aren't teaching the class
well; or (2) the students are coming in without proper preparation.

Many students coming into an intro to CS class are already programming
whizzes, and then there are others who have no experience with programming at
all. So, already the instructor is in an impossible situation. Do you bore the
promising kids with the major head start, risking losing future good students?
Or do you ramp the class up to their speed and risk losing the kids who didn't
come in already knowing the material? Obviously, most departments and
instructors will, wittingly or not, choose the latter.

------
drivingmenuts
All of this is predicated on the idea that programming is solving sexy math
problems.

Reality is that most programming is taking bits out of one bucket, combining
them, and dropping them into some other bucket. It's the digital equivalent of
shovel work.

------
slashnull
"Programming"?

What kind of "programming"?

What "ability"?

Ability to write space-O(my_god * n) sorting algorithms? Ability to duct-tape
queries to an ill-documented SOAP webservice and upsert the return into a
Your-Boss-Normal-Form database schema? Ability to brutally optimize the
runtime of a really hairy numerical analysis algorithm? Ability to design a
centralized data pipeline architecture and lead 12 hackers during the
implementation and migration? Ability to find the off-the-shelf OSS project
that solves the first 50% of the problem instead?

I was watching the news the other day on TV, which is something I almost never
do, and there was a segment about the educative value of tablets, smartphones
and laptops for teenagers and children. The value of the freely available
information on the internet was discussed, along with the quality of free
and... not free educative and scientific applications, and the ill effects of
such devices on people's attention span, and... That's it.

Example devices included many, many Apple products, a few off-the-shelf
Android devices. A Windows machine running MS Word.

Same ideology regarding technology in education: locked-down workstations with
user-friendly applications, internet access which blocks every port except for
80 and 443.

A relentless and sustained effort to erase everything but the topmost layer of
the IT stack. Locked-down devices. People come to me asking me to remove
viruses, to speed up their old XP machines fraught with annoyware. No Windows
license, don't want to pay for one, don't want Linux. Computers run on magic,
right? How can you know that programming stuff and not be able to remove all
that _bad magic_ from my computer. Computer Science students who can't find
the slash on their keyboards. People who see computers as a monolithic entity
rather than a brittle but transparent stack of conceptually distinct layers.
The new generation of systems administrators, can't scp a tar at the other end
of the lab.

A teacher spending three hours explaining red-black trees to a class scrolling
down 9gag; students, diploma in hand, who aren't entirely sure what's the
difference between an array and a hash, who are not sure of what happens when
one puts an array in another array, in PHP, after 120 hours spent
theoretically writing PHP.

Computer Science students convinced that Linux is 100% not worth learning, in
any way, because nobody uses it.

This is how the knowledge economy dies.

~~~
WoodenChair
This is one of the most epic rants I've read on HN in quite a while. Thank you
for the comedy, but it's unclear how anything beyond the first couple of
paragraphs relates to the discussion here. You're saying that our education
system produces ill-prepared CS graduates right? What does that have to do
with the correlation between an algebra test and programming aptitude?

~~~
slashnull
1) it was a rant that I needed to make

2) what the exact fuck is "programming aptitude" anyways? The article defines
it as "having good grades in a CS program". Which is part of why I needed to
rant against CS education.

------
poppow
Appreciate the idea behind the article which may have value, but the data
types seem ordinal? 1.0-4.0? is that trying to mirror US GPA system.
Normalized based on what? p-values are an archaic only useful for MBAs and Med
students.

~~~
mgirdley
We presumed and used a normal distribution for each class.

The numbers were arbitrary. Just what felt easier to understand for the staff.

~~~
poppow
Normality on ordinal scale data. You've done the equivalent calling functions
expecting numeric input with ordinal data.
[https://en.wikipedia.org/wiki/Ordinal_data](https://en.wikipedia.org/wiki/Ordinal_data)

------
m-photonic
Don't have the study handy, but measures of working memory such as the OSPAN
were found to predict proficiency in a programming course even better than an
algebra test that participants also took.

------
faitswulff
It would be interesting to administer these algebra tests to junior/mid/senior
level programmers currently in the field.

------
GFK_of_xmaspast
One thing I'm a little uncertain of is the ideology behind these kinds of
things, it seems like the primary purpose here is to keep people out of the
field.

Also I think those are some pretty weak quantitative results, and good
exhibits for argument against p-values.

~~~
jcfrei
Why do you think the results are quantitatively weak? My first critique is
just the small sample size with ~60 students and the tests, which appear to be
different between the students.

~~~
lomnakkus
The OP alluded to p-values, and I think this may be relevant:

    
    
       https://www.youtube.com/watch?v=5OL1RqHrZQ8
    

(In short: p-values are pretty much useless as an indicator of "this
experiment can be reproduced".)

Not that this necessarily refutes the original article, I just thought it
might be relevant to discussion.

~~~
rhiever
If you look in the methods, the experiment has been successfully reproduced
twice after the initial trial with distinct cohorts of students.

------
joemaller1
Is the complete algebra test available somewhere?

------
jackhammons
no

------
pduszak
No.

------
Rooster61
I really don't want to sound rude, but this sounds like a classic case of
correlation not causality, and I could see the consequences really hurting
some people.

I am REALLY terrible at algebra (to the extent that I am borderline LD in
math), and I have completed a degree in CS and hold a steady job as a software
engineer. And I am definitely not the only one. Ask a room full of developers
how many are bad at math, and I guarantee the results will surprise you. On
the other hand, I know a plenty of folks I went through school with that were
math majors so good at algebra they could do full page derivations piss drunk
without at hitch. And yet, they would take an intro java course and be totally
lost. If they didn't get a java intro class, how do you think they would have
done with something as algorithmic as an assembly language, a functional
language, or understanding the nuts and bolts like Turing machines and
automata?

Algorithmic thinking is definitely an important part of programming, but it is
just that, only a part. And quite a few types of development don't emphasize
that sort of thinking. I could perhaps see this being a PARTIAL solution for
areas where one would be doing a lot of functional programming, but the vast
majority of the job market these days is still OOP.

This could really quash the job prospects of programmers who are perfectly
capable of writing quality code but are poor with algebra.

~~~
spacehome
I don't think the author claimed causality. He's interested in prediction, for
which correlation is perfectly well-suited.

Also, nobody is claiming that the correlation is perfect. You can perfectly
well see outliers in his graphs, but outliers don't disprove the correlation.

~~~
Rooster61
Fair point, but showing these graphs complete with the outliers gives a very
poor platform to base handing out algebra tests in interviews without
establishing causality.

~~~
ObviousScience
> very poor platform to base handing out algebra tests in interviews without
> establishing causality

Why?

It probably shouldn't be your only criterion, but if it has a high
correlation, why wouldn't that be sufficient for handing out algebra tests?

A high correlation corresponds to predictive power, even if it's not a causal
relationship.

~~~
sysk
I think the point being made is that there are probably predictors which have
many orders of magnitude more predictive power (e.g. work experience,
education, interview questions, programming exercise, available source code,
etc.).

The male gender is probably correlated to programming ability as well due to
the fact that there are more male programmers but it's not a very useful
predictor.

~~~
marcosdumay
Predictors for a school to apply? I wouldn't bet on it. None of your
suggestions seem fit.

Also, for a job it's probably a better predictor than a resume.

~~~
sysk
> Predictors for a school to apply?

First, it was suggested that this test should be used for programming job
interviews. Second, why would a school who teaches programming care if
students had programming ability? If you answer it is because the school wants
to maintain a reputation of producing good programmers (hence adding value to
the students' resumes), it contradicts your statement that this test is a
better predictor than a resume. Otherwise, people would just study algebra
instead of going to that school.

~~~
marcosdumay
The author uses it as a predictor for acceptance at a school.

For the way, he was around the thread explaining that he can train a limited
number of people, and that some of them not learning is wasteful and
uninspiring.

