
Programmer Competency Matrix - olalonde
http://www.starling-software.com/employment/programmer-competency-matrix.html
======
ColinWright
Those who are interested in this subject and want to discuss it, you may be
interested in seeing the points - good and bad - that have been made on
previous occasions that this has been submitted. Here are a few previous
discussion:

<http://news.ycombinator.com/item?id=232192> (indiangeek.net) 75 points 4
years ago 33 comments

<http://news.ycombinator.com/item?id=554338> (indiangeek.net) 83 points 3
years ago 33 comments

<http://news.ycombinator.com/item?id=1022394> (indiangeek.net) 67 points 2
years ago 40 comments

<http://news.ycombinator.com/item?id=1949915> (starling-software.com) 155
points 1 year ago 106 comments

<http://news.ycombinator.com/item?id=2823912> (indiangeek.net) 12 points 1
year ago 1 comments

<http://news.ycombinator.com/item?id=3434350> (starling-software.com) 2 points
9 months ago 0 comments

Discussions like this can be of value, and no doubt some of the newer people
on HN may have points to add. OTOH, many points will already have been made,
so seeing the past discussions may be of benefit.

~~~
praptak
111 comments 1 year ago had 155 upvotes, not 1. The upvote count patterns are
interesting nevertheless :)

~~~
ColinWright
Sorry, I simply cut-n-paste from the search results - I'll go fix that now.

Thanks for the heads-up - much appreciated.

 _Added in edit: Now fixed - points and comments. Thanks again._

~~~
indiecore
> _I simply cut-n-paste from the search results_

Ah, we have a level 0 on the HN matrix here apparently.

~~~
ColinWright
Well, to be honest, I have deep-scripted accessing HN, but I tend only to use
it when doing big data crunching on trends and similar. When providing cross-
references I've usually just put the IDs - actually counting points, comments
and age is something I've not done for a while.

Don't think I'll bother again, I get enough grief and hate-mail just from
trying to point people at earlier conversations.

~~~
javert
I really appreciate it when earlier conversations are pointed to. (The
additional points/comments/age numbers are wonderful, too, if you feel up to
providing them.)

I wonder if a bot could be written to automate this? There are lots of
horrible reasons one could have for writing a HN comment bot, so I'm hesitant
to even suggest it, but this seems like a really useful/good bot.

~~~
ColinWright
I have one, I used it, I got significant hate-mail, I decided I didn't need
the bile I got, I discontinued it.

~~~
AWalrus
Actually, I can't tell you how many times a HN comment like yours has turned
me on to an ultra-valuable related discussion that I would never have seen or
thought to look for.

So don't let negative comments (perceived or actual) speak for us silent
appreciaters!

~~~
ColinWright
Thank you - much appreciated.

------
andrewvc
This is someone's (very arbitrary) opinion in a grid. I hope you don't
actually interview people. Figuring out someone's skill level is a soft
science; it takes months after a hire to really know where someone is.

Some people I know are great at fixing bugs but have narrow language
experience. They're damn valuable. Others integrate well into teams and
provide great product feedback. Where's that on here?

Experience isn't measured in years but in knowledge gained. You really start
seeing it 4-5 years in, some people move forward, some don't.

There are so many holes in this. I feel like whoever wrote this has a strong
desire to reduce people to a number. It's not only terrible, it's a great way
to fool yourself.

~~~
Swizec
Some people have 10 years of experience. Others have 1 year of experience 10
times.

~~~
tptacek
What a great quote. I wonder where it came from.

~~~
Jun8
This is a quote I use very often! As with many other great quotes, I think it
originates in English from Trevanian's _Shibumi_ (when Otake-san, the Go
master is giving his final teaching to Nicholai):

Do not fall into the error of the artisan who boasts of twenty years
experience in his craft while in fact he has had only one year of experience —
twenty times.

Most probably Trevanian has drawn upon some other source, tough I've never
been able to track that.

If you haven't read this fantastic book I strongly recommend it.

~~~
chernevik
It's an ok potboiler that relies ever-more improbable plot devices.

It's also impossibly incorrect politically -- so transparently so that I think
it's essentially harmless. But if you don't understand why some people would
feel that why you really should watch your step.

------
Xcelerate
I'm not sure this has anything to do with "competency". When I was in high
school, I implemented a kd-tree for a ray tracer using photon mapping. I knew
that my n^2 algorithm was slow and that there was probably a better way, so I
Googled and figured out how to write code for such a structure.

Since I'm not a programmer (I'm a chemical engineer), I don't instantly know
all of the orders of various algorithms (which according to this article makes
me "incompetent"). But you can bet that when it comes to implementing
something, I'll research and figure out the most efficient algorithm available
and use it.

I'm not sure being good at programming has much to do with having a whole
bunch of CS knowledge; rather, it's being able to figure out what knowledge
you need to know -- when you need to know it.

~~~
krosaen
I agree with this - e.g not being able to bust out a dynamic programming
implementation perfectly on a whiteboard doesn't make you a bad programmer -
more important is to understand, "hey, looks like when I divide this problem
up it has overlapping subproblems - let me research using memoization or
dynamic programming". But that still assumes you have the right terms in your
head to be able to do the research - and to me that's where some fundamental
knowledge of algorithms and data structures comes in handy.

~~~
cwp
True. But note that the matrix isn't a binary "good/bad". If you know the
concepts that underly dynamic programming and can look it up when needed,
great. If you have used it before and can bang out an implementation without
looking it up, well, that's even better.

------
lathamcity
Here's mine:

Level 0: Saw the post title and didn't think he needed to look at it because
he was so confident in his abilities already.

Level 1: Read the post, noticed a lot that he didn't know but didn't intend to
do anything about it.

Level 2: Read the post and went out and learned everything in it that he
didn't know.

Level 3: Read the post, figured out which things from it he didn't know but
might need to or want to, and learned those.

Oh, and a special level -1: Read the post, noticed a lot of things that he
didn't know, and decided the post was stupid because clearly he's an amazing
programmer and the post didn't agree with that self-assessment.

~~~
gridspy
Nice.

Is there a level for "Skimmed the first part of the post and then went back to
working on his/her startup" ?

~~~
BasDirks
No but there's one about commenting on the post in a way to make oneself seem
clever. I have hereby attained it.

------
npsimons
Interesting. One thing that caught my eye was "someone who knows most of the
tools from Scott Hanselman's power tools list." I had never heard of Scott
Hanselman, nor his power tools list and had to look it up. Frankly, it looks
useless to me as a programmer and looks more like a list of desktop gadgets,
and watered down versions of tools you get for free with most Linux
distributions. He also doesn't mention Linux (or OSX), nor Emacs nor vim. The
great majority of the tools are Windows specific or Windows only (I caught one
or two that weren't).

~~~
gosub
Knowing most of the power tools mentioned in this book
<http://shop.oreilly.com/product/9780596003302.do> would do a better job as a
discriminating factor, in my opinion.

~~~
npsimons
That's one of my favorite books. I know it isn't, but I've often imagined that
the tool on the front is a hole hawg ;)

------
lucian1900
Many are biased (objects > functions), several are not useful and a few are
plain wrong.

Meh.

~~~
Zimahl
Agreed. I'm not sure how 'Source Code Version Control' and 'Build Automation'
is anywhere near the words 'Software Engineering'. And 'Automated Testing'
isn't a given at a lot of shops.

Professionally, I've never set up version control. It's there when I start the
job, it'll be there when I leave. I've branched and merged but a lot of times
this is handled by a build manager. Build automation I have never expressly
done - a shop that needs continuous integration is typically one where it is
already set up.

Seriously, unless the job is 'Build Manager' then the question should be just
used to gauge what you might have to teach them. 'Have you used source/version
control software? Which ones?' 'Do you have experience with build automation
tools?'.

~~~
yen223
Let me get this straight, you do not know how to set up version control,
therefore knowing how to set up version control is useless?

It's not hard to start a Git repo you know, it's something you can pick up in
under a day.

~~~
genwin
It wouldn't be useless but it could be unnecessary. I don't anticipate ever
needing to set up version control, unless it's my own business. Any place I
work will almost certain already have it set up, and if not, someone else will
be tasked to set it up.

~~~
xtreme
I actually like to put many of my documents under version control. Also, if
you do any project as a hobby, using a vcs would probably help a lot.

------
HyprMusic
I personally believe I fall into "n (Level 2)". I recently completed a degree
in Software Engineering, and despite programming for a long time before that I
would have ranked myself at "n2 (Level 1)". The degree pretty much brought me
and everyone up to scratch with "n (Level 2)" in terms of knowledge. Having
actually tried to work with some of my classmates - who also achieved first
class degrees (some with higher grades than me despite little prior
programming experience) - they're mostly terrible programmers.

As far as I'm concerned, this table is purely for working out a programmer's
education, not competency. Experience and passion are the only ways I chose to
judge my colleagues.

------
pykello
"able to identify NP problems" ? Most of the problems we face are NP. Maybe he
was meaning NP-complete problems. See <http://mathworld.wolfram.com/NP-
Problem.html> for the definition.

~~~
praptak
_NP-hard_ is the actual class of problems you want to identify, although
people who can identify NP-complete problems have no problems spotting NP-hard
ones.

------
sequoia
Inexperienced programmers consult documentation a lot and the more experienced
you become, the _less_ you consult documentation? If this were true, the
initialism "RTFM" would never have been coined.

~~~
kisielk
That one seems totally backwards to me. Experienced programmers should be able
to read and understand more technical documentation such as language specs or
RFCs and use it to figure out the answers to their problems. Inexperienced
programmers are the ones that sak for help at the first sign of trouble.

------
eranation
Here is a simpler matrix

\- can the candidate do the job? (github projects, portfolio, hands-on test)

\- when they talk, do you understand?

\- when you talk do they understand?

\- do they sit all day and read HN / Stackoverflow?

\- on the other hand, they never heard of the above

\- do they get the job done or try to make it perfect?

\- do they fit the culture of the company

\- can they learn a new skill quickly

give 1 point for each of the above and you got a much better matrix (if you
want to hire people you won't need to fire)

------
DodgyEggplant
As usual, already written in the bible:
<http://www.joelonsoftware.com/articles/fog0000000038.html>

------
coob
Where's the interpersonal skills section?

Attitude section?

~~~
mattdeboard
Those are part of the "existing-in-modern-society-as-a-human-being competency
matrix", along with bathing and wearing pants as often as possible.

~~~
javert
I think there are programming-specific ones.

For example, on attitude: Learning to be patient. Patient with planning the
right solution, selecting the right libraries, fixing bugs, dealing with other
developers who may not be as experienced as you, etc. I think patience is an
overlooked "programmer's virtue."

~~~
wtetzner
Actually, impatience is the virtue. <http://threevirtues.com/>

~~~
kstenerud
Ugh. A pox on people who write programs that "anticipate your needs". They get
it wrong more often than right, slowing you down, blocking you, or
interrupting you and breaking the flow.

And to make matters worse, those features tend to be baked into the system
such that you can't disable them.

Impatience is no virtue.

------
buro9
The original is here: [http://www.indiangeek.net/wp-
content/uploads/Programmer%20co...](http://www.indiangeek.net/wp-
content/uploads/Programmer%20competency%20matrix.htm)

------
gavanwoolery
This is technically not a "bad" measure of performance, but the real danger is
the word "competence." Imagine I am hiring two candidates, one has log(n)
(level 3) competence across these charts, while the other has very
mediocre/poor competence. What if I am working on an application is not
performance-critical? What if the "incompetent" guy/girl works twice as fast,
because he does not worry about crafting coding-works-of-art? What if the
"incompetent" guy is three times cheaper to hire, and eager to prove
him/herself, thus putting in extra hours voluntarily? What if I am hiring for
a problem with a very specific domain, and expertise is required in only one
area?

~~~
bmohlenhoff
You don't need a wizard for every task. There's nothing wrong with taking
someone from the left side of the matrix. You just have to consider what
they'll be working on and what resources are available to them (ie. mentors,
etc.) Hiring decisions are comprised of tradeoffs, just like everything else
in engineering.

------
arbuge
This is a rather narrow viewpoint of what makes a great programmer in my
opinion. One completely missed element here is the ability to track
organizational complexity. For example, you have a site with 100,000 lines of
code. Making one change somewhere affects 10 other things elsewhere, each of
which might affect another 10 things. Keeping track of all that is what I'm
talking about. Same skill you find in great chip (integrated circuit)
designers. Personally I think it's far more important than some of the
concepts here, which focus mainly on algorithm-level stuff.

------
npsimons
Interesting. One thing that caught my eye was "someone who knows most of the
tools from Scott Hanselman's power tools list." I had never heard of Scott
Hanselman, nor his power tools list and had to look it up. Frankly, it looks
useless to me as a programmer and looks more like a list of desktop gadgets,
and watered down versions of tools bundled with most Linux distributions. He
also doesn't mention anything for Linux (or OSX), nor Emacs. The list is very
Windows specific.

------
danso
Thank goodness B-trees are considered the most-advanced of the data structure
knowledge. My com sci classes never did a great job of making those
applicable.

~~~
quotemstr
B-trees are great! They have better locality than binary trees --- that's why
filesystems are universally based on B-trees: seeks are slow. As cache effects
become ever more important, B-trees are starting to become important even for
problems that fit in main memory.

~~~
leif
And as data and disks grow, we'll see write-optimized structures like LSM
trees and Fractal Trees in this matrix as well.

------
seattlecoder
I land solidly in the right-hand column, particularly on the systems
programming front: I've made major contributions to the NT kernel. I'm not
sure this matrix has all that much significance: Google rejected me after the
in-person interview: do they want people who can sort N items in O(1) time,
work 30 hours per day, and create new scripting languages over lunch?

~~~
moconnor
Apply again. Google rejects a lot of good programmers on their first
application. It's not a perfect process.

------
pjungwir
I would like to see a section that includes things like identifying/clarifying
customer requirements, finding business solutions, controlling scope,
organizing a project around some methodology, leading a team, etc. I see the
lines for Problem Decomposition and Communication, but I think there is
something much broader that is being left out.

------
bane
Either where I went to school had an exceptional CS program, or most CS
programs don't cover what my run-of-the-mill state school covered during my
undergrad. About 10 years ago I came out of school at around a level 2 or 3 in
almost all of the areas on this grid. I would wager pretty much any 3.5+GPA
student in a decent CS program should be the same.

I didn't pursue being a professional developer so I've lost most of these
things, but I remember having worked on almost all of these things (such as
they were at the time). Some of these things I haven't heard mentioned _since_
my undergrad!

Interestingly, we never studied functional languages, but did cover Prolog.
This has definitely created a deficit in my thinking and consideration about
certain problems/languages. If that's any indication this matrix can give
insight into the abilities of people who simply haven't covered certain
topics.

------
ianstallings
This is the stupidest thing I've seen on this topic. Are we trying to
eliminate good candidates? Because that's what this will help with.

If programmers spent half the time they spent evaluating others on being
better programmers we wouldn't have any problems.

------
wwwtyro
I really like the idea of a way to measure my knowledge and skills. My
preference would be for a directed knowledge graph, where nodes list the
knowledge and skills fundamental to the nodes they connect to. This way, I
could start from the bottom of the graph and place a checkmark on each node
that I felt comfortable with. The end result would be an outline of what I
needed to learn next to progress.

Alternatively, if I were in a rush to learn a specific item, the tree would
give me a roadmap for getting there.

I'd be interested in a wiki-like interface for building such knowledge graphs,
so that they could be built quickly and completely.

------
sbierwagen
I'm not sure if "has used git" should be log(n)-- I'm only barely a
programmer, and I've never used CVS/SVN, but I use git to version control
markdown files.

~~~
yen223
Using Git requires a bit more brainpower than using CVS or SVN. If you know
how to use Git, you definitely know how to use SVN.

~~~
sbierwagen
I've never branched, or performed a merge. I use git as a pure CVS clone.

------
tomrod
Cool -- I actually hit n or log(n) on a couple of these. Neat! Most of the
rest I'm right at the n^2. Guess I still have a long way to go.

------
ep103
"Checks all arguments and asserts critical assumptions in code "

Wait, I just finished code complete, and it told me that you specifically
_don't_ check all arguments throughout your program. Instead, you choose a
list of classes at that will check arguments for input as a firewall, and
assume all classes behind those logically will receive the correct, sanitized
input.

Did I misunderstand this?

~~~
npsimons
If you're paranoid (eg, programming embedded where it absolutely, positively,
should not fail), you _always_ check _all_ arguments and return values.
Sometimes libraries shipped from the vender are _not_ perfect, sometimes the
vendor's compiler will do something, _ahem_ , unexpected with your return
value in some edge case where you've exceeded the stack. I'll agree that
wrapping with inlining is a good idea to make code readable, and these days
you usually don't have to worry about a lot of hardware limitations
(especially in the general case: desktop/Android/iOS/web/etc).

~~~
quotemstr
> programming embedded where it absolutely, positively, should not fail
> [means] you always check all arguments and return values

Let's distinguish "checking" invariants, preconditions, and postconditions
from blithely propagating logic errors as error codes all the way up the
stack. If you're programming in "paranoid mode", as it were, and you detect
inconsistent state, you should _abort the program_. If you don't, you're in
uncharted territory and you have no idea how your program will act. If you
abort the program and restart the system, you'll probably end up back in a
useful state.

Many, many times, I've seen code like the following:

    
    
        HRESULT hr;
        hr = DoComplexThing();
        if (FAILED(hr)) { RaiseFailFastException (...); }
    

Say someone passes a NULL pointer to a function somewhere deep inside
DoComplexThing's implementation that doesn't expect NULL. If this function
helpfully "checks" all parameters, detects the NULL, and returns E_POINTER,
E_POINTER will probably propagate all the way up the stack to the top level,
where we'll abort the program. Now you have a crash report, but you have no
idea where the problem actually _is_.

If this function had instead not checked the NULL pointer and crashed or
explicitly asserted its preconditions instead of treating contract violation
as a runtime error, the problem would be a lot easier to diagnose.

In general, you can separate problems into two classes: logic errors and
runtime errors. Logic errors are indications of the program being written
incorrectly. Your program should blow up when it detects one. Runtime errors
are errors that could conceivable occur at runtime due to reasons outside the
program's control (e.g., removing an SD card). Only runtime errors should be
propagated using your error reporting mechanism of choice.

------
jonke
I'm amazed that Oz surface up in this matrix. The number of people actually
program in Oz for production purposes must be less than 100. It's a lovely
language and I can miss somethings one could do in (Mozart)-Oz that I can't do
in Haskell.

------
ww520
It covers a good section of the competency area and kudos for putting it
together. It's certainly much better than the puzzle type questions.

It's kind of generous in its rating for advance though. May be there should be
a level for expert. But good work.

------
zobzu
So in the end I figured out you can't be no good programmer if you don't know
prolog.

~~~
sjf
yes.

------
zaptheimpaler
This probably isn't very useful for evaluating an interview candidate as a
whole, but I like how comprehensive it is in terms of technical knowledge.
It's a useful roadmap to evaluate your own technical competency too.

------
Weaponx2007a
This reminds me that once upon a time scholars thought that art was
measurable.

------
lists
I'm sure it's a perfectly good topic to discuss, but I wonder what's
motivating this sudden trend of "You must be this tall to call yourself a
programmer" posts?

------
KirinDave
Why do I feel like "basic competency for someone with a few years experience"
is the top end of this grid? Seems like some low bars.

------
juddlyon
According to this, I'm a worthless newb.

Sounds pretty accurate. :)

~~~
krapp
I work with PHP and javascript so I already knew I was scum...

~~~
npsimons
Hey now, don't be so hard on yourself - at least you work with Javascript ;)

~~~
krapp
I did teach myself enough Perl to write what was possibly the worst flat-file
bbs script ever, ages ago on Tripod. >_>

------
riazrizvi
Now if you could write tests that correctly determine which box a candidate
fits in, you would have something very powerful indeed.

------
adjwilli
Ha, wow, I think am a terrible computer scientist and software engineer, but
probably a decent to maybe even good programmer.

------
gearoidoc
Unfortunately none of the skills mention the ability to create something that
people ACTUALLY USE.

------
zem
maybe it's just because i'm mostly an algorithms guy, but the bar for "O(log
n)" systems programming seemed to be somewhat higher than that for
datastructures and algorithms. and having "author of framework" be on the same
level is another huge leap.

------
hunterhusar
Someone has way too much time on their hands.

------
criveros
What is a new grad expected to be n, n^2?

------
kahawe
> _automated testing_

I love how they non-chalantly squeezed "automated testing" in there... because
to really do well in that area you need so much more than the average handful
of junit-or-whathaveyou tests and the whole "comes up with good test cases" is
just extremely shallow. While developers do well to implement some simple unit
tests, automated testing includes SO MUCH more they just left out. It is a
whole software project and includes a lot more people than just "programmers"
and "testers".

Another personal pet-peeve of mine are those endless suggestions of how to
pick and hire developers, how can you get the very best of the best for the
likely average salary you are offering to begin with - yet we have a
continuous stream of coding-related horror stories.

It doesn't take any magic, it doesn't even take a catalog of weird questions
that might work for google. Ideally go with recommendations, people who know
people that are good. Or pick the interesting CVs and cover letters, avoid the
buzzword rally. Then interview them and talk about what they have done so far
and go into some details there, scratch on the surface, get a feeling for what
they got going on. By now you should have quite a good idea. Then hire them on
a temporary contract and see how they do. Be ready to give the new-comers and
graduates a chance and be ready to invest in them, both in trainings and
compensation.

IF you are in a position where you just cannot afford to put good faith in a
common sense choice for a candidate then you should not be hiring in the first
place because that means you either don't have the money for one more hungry
mouth anyway or your project is so far into the death-march that another pair
of hands is not going to rescue the sinking ship even if you get a true
kernel-hacker to work on it.

