

Programmers and “productivity” - mwsherman
http://clipperhouse.com/blog/post/Programmers-and-productivity.aspx

======
btilly
The article makes a number of major mistakes that discredit its whole thesis.

1\. My source for the productivity figure is _Peopleware_. They cite a set of
"coding war games" that found a repeatedly measured 10x variation between the
best and the worst programmers in their work environment. The productivity
difference showed up under a variety of different measurements. There is no
vagueness about the measurement techniques or doubt about the figure.

2\. The 10x figure was from best to worst, from best to median was (IIRC) 2.6
times.

3\. A significant portion of the figure they measured was correlated with
environment, and the strongest predictor of performance was the performance of
your coworkers. That said their study technique couldn't tell the difference
between whether environment attracted the best people, or whether environment
caused people to become more productive. (From the point of view of a smart
employer there isn't too much difference between those hypotheses.)

4\. People accept compensation in many ways, and not all of them involve
money. I believe that the better a programmer is, the more apt they are to be
willing to trade salary for better work environment, more interesting co-
workers, more interesting problems, etc. As a concrete example I have heard
anecdotes from many people claiming that people are willing to accept
sometimes substantial pay cuts for the privilege of working at Google.

5\. Whatever the cause, in the _Peopleware_ study the difference in pay
between the top and bottom half of programmers was less than 10%. Furthermore
productivity was basically uncorrelated with various factors people look at in
the hiring process such as years of experience. That is direct evidence that
it is a bad idea to measure productivity by salary.

When you put all of these factors together, you simply can't point to market
forces then salaries to discount real performance differences between working
programmers.

~~~
joe_the_user
Ah,

2.6 "Best to median" sounds like a much more plausible figure. The x10 figure
is usually quoted in such a way that it seems like it is "best to median".

~~~
patio11
Numbers I cannot disclose to you from a major Japanese multinational which
actually tracks programmer productivity (grumble grumble why is this rare
grumble grumble) cause me to not have major plausibility problems with the x10
best to median.

~~~
fp
Can you talk about the metric(s) by which they measure productivity? I would
appreciate it.

~~~
patio11
We're essentially a large consultancy who bills by the kousuu -- "unit of
engineering labor", somewhat arbitrarily defined as "what we think one
engineer should be able to do in a day". Projects are successful if we bill
more than N * K * the average labor cost of producing one kousuu, where K is
the number of kousuu and N is a constant representing overhead, profit, and
the cost of sale.

Projects are sized based on a negotiation between us and the customer. It is
then broken into parts, which are given their own kousuu value, via
guesstimation by a senior engineer on the team. (You'll notice this process
involves an awful lot of guesstimation. This tends to converge on reasonable
numbers because guesstimation is happening among multiple actors who don't
have an incentive to collude -- kinda-sorta like a market of players with
imperfect knowledge can converge staggeringly quickly on a good guess of the
intrinsic value of something.)

We then measure how many kousuu of work you complete for a particular project
per day you report you are working on the project. That is currently our
primary productivity measure, although we also track customer satisfaction,
"claims" (a backchannel by which customers here get free work done to fix
issues in the deliverable which are determined to be due to our mistakes),
subjective quality assessments from the engineers, follow-on business from the
same customer, and any value we can extract from the project for the company.
(For example, in a 10 kousuu project if we manage to extract 2 kousuu worth of
functionality that the customer paid for into our internal framework, then
deploy that to seven customers and charge them each for it, we make out like
bandits.)

Anyhow, we keep obsessive records of who does what and how long it took, so
figuring out productivity is "just" a matter of dividing. (We only figured out
the dividing trick recently. Hard to move a large organization from the way it
typically does things, you know?)

------
camccann
The expectation of market clearing for programmer wages is completely
implausible here.

Going independent requires additional skills beyond the programming
productivity that we're supposedly trying to measure, is perceived as riskier
(and it's generally understood that most people are irrationally risk-averse),
and may involve additional practical complications, such as if our
hypothetical programmer relies on salaried employment to provide health
insurance for a family. So "would this programmer make more independently" is
pretty much an irrelevant question.

As for moving between "regular" jobs, there's little evidence that most hiring
processes can distinguish programmers by productivity, nor that most
organizations measure the productivity of employed programmers in a useful
way. Switching jobs also has practical obstacles, such as the disruption of
relocating, and avoiding the perception of "job hopping". So as far as we can
tell, salaries have little relationship with productivity, which is kind of
the whole point.

Fundamentally, the article makes the mistake of taking principles that are
_simplifying assumptions_ in economics and then arguing as if they're
universally correct. This is roughly akin to an argument like "Things tend to
fall down, so it's a fallacy to ask why helium balloons float."

Of course, you can simply define someone's productivity as the market value of
their time and thus "prove" that order of magnitude differences in
productivity between programmers don't generally exist, but proof-by-
redefining-the-terms doesn't really make for a persuasive argument.

------
keefe
"If our newly-independent programmer does not make 10x the money, we must
admit that she is not the 10x programmer we thought she was." Not a good
metric. The signal is at least polluted by social skills and geography - in
some areas of the midwest, the social construct of the traditional company is
much more entrenched than in some places in CA. Also, I'm 29 and I have no
family - I could work totally different places and locations than a family man
with roots, which opens up my possibilities much more. imho, conflating the
concept of profitability and productivity is not a good idea.

~~~
joe_the_user
Yes,

I'm actually a skeptic on the generic x10 programmer-productive claim but the
claim "Markets tend to clear" when applied anything and everything is more of
a justifying fiction than a useful measure. Alan Greenspan and friends used it
repeated to justify the housing bubble. And yet this kind of reasoning is
still here.

Sigh...

~~~
gruseom
_I'm actually a skeptic on the generic x10 programmer-productive claim_

Why? There have been many studies of this and their findings are consistent.
IIRC the numbers range from 5-to-1 to 100-to-1 with the median in the order-
of-magnitude range.

Pages 14-15 of Robert Glass' _Facts and Fallacies of Software Engineering_
cover this topic.

[http://books.google.com/books?id=3Ntz-
UJzZN0C&printsec=f...](http://books.google.com/books?id=3Ntz-
UJzZN0C&printsec=frontcover&dq=facts+and+fallacies+of+software+engineering&cd=1#v=onepage&q=individual%20differences)

------
gruseom
I agree with the other commenters: the author rather trivially assumes his
conclusion by taking for granted that the market is rational. It isn't. One
can argue about why. I tend to believe it's because we're still early in the
history of the software: the closest analogy we have is the Industrial
Revolution, and Ford and Taylor don't appear until well over a century into
that.

The other side of the coin, of course, is that an irrational market represents
major opportunity for anyone who figures out how to exploit it. So we all
should stop complaining and start exploiting. Hopefully the hacker renaissance
and startup wave of the last few years are the start of such a process, but
it's too early to know for sure.

------
Deestan
Article appears to be deleted. Cannot find anything with similar title in
recent archives either.

~~~
ComputerGuru
It's working fine for me - maybe my ISP is caching it or something??

I uploaded a copy (temporarily) to my site, feel free to grab it (PDF format):
<http://mqudsi.com/junk/Programmers_and_Productivity.pdf>

~~~
TheSOB88
You are a nice guy.

