
Um, I have a question (about programmer productivity) - cawel
http://github.com/raganwald/homoiconic/blob/master/2009-02-12/a_question.md
======
lgriffith
Joe paints a three bedroom house in three days and gets paid $300.

Jim paints a 5 ft by 3 ft watercolor landscape in three days and sells it for
$1,000.

They both get their job done in three days. One covers far more area with
paint than the other. One gets paid more than the other. Both of their
customers are happy with the result because they paid for it. Who is the most
productive?

Is it even meaningful to ask that question?

Doesn't it depend upon WHAT it is that is produced? Doesn't the WHAT have to
be the same in both cases to be able to say which is more or less?

Perhaps before we talk about being more or less productive we need to identify
what it is that we are producing. We should make sure its real, meaningful,
and relevant to our purpose and that we are all talking about the same thing.
Then finally, we must measure something that can't be easily inflated or
faked. Only then can we talk about more or less comparisons and be saying
something other than words without real meaning.

As I see it we have two questions to answer:

What is it that we are actually producing when we write software?

How can we reliably measure it?

We don't have good answers for either. Actually, I don't think we even have
moderately good bad answers.

Isn't it interesting that a large and increasing fraction of the world's
economy is based upon something we don't really know what is and can't
actually measure?

~~~
sfamiliar
when we're measuring productivity, what are we trying to measure? i mean,
really? ultimately its how much value is created, in what span of time.
relevant to this example, the questions to be asked are:

1) how much does joe's paintjob increase the value of the home for the HGTV-
obsessed house-flipper?

2) how much will the value of jim's watercolor appreciate after he kills
himself following a week-long binge of heroin?

put another way: how efficiently is value being created? the important answers
lie more often in the decision process, and less often in the implementation
schedule.

for example, a web developer working on a brochureware site: the right
question may be along the lines of 'how will the work you're doing affect the
conversion rate?'. in reality, those decisions aren't up to him -- the might
lie in the internet strategy department or some such. they push the order, the
developer implements. a real measure of results is closer to 'did the designer
make the right decisions to yield results?' rather than 'did the developer get
the code done more or less quickly than last time?' in my experience, when
someone who is a non-developer is trying to quantify developer 'productivity',
what they're really doing is dodging the question of whether or not their
decision-making was good or bad, usually when they know it's bad and are
looking for something to roll downhill.

more interesting to me, as a developer and sometimes manager, is whether or
not a developer can deliver when they say they can? if clay says it'll be done
in a week, and then it is, he's been sufficiently productive. 'sufficiently
productive' is maybe the best thing we can do. the race analogy is an apt one
-- if developer 1 can produce a set of functionality with the same defect
level faster than developer 2, we can say that d1 is 'more productive', but
it's a difficult thing to quantify absolutely. take the number of hours a
developer spends working on a given project point, multiply it by their hourly
rate, and you get a development cost for that project point. then consider the
value that project will add to the product, over time. if you have 30,000
customers who haven't bought the product yet because it doesn't have Feature
X, and adding feature X costs dollars Y, then your value-add is something line
Z= customers * product price. your efficiency is something like Z/Y.

clearly you want to minimize cost, and therefore keep your efficiency as high
as possible, but the ways to manipulate that, and a lot of points to consider:

a simple example -- if you can hire a developer that will take twice as long,
but work at 1/3 of the rate, would the revenue lost during the extended
development period negatively affect the profit efficiency?

the important questions are somewhat higher up.

~~~
lgriffith
Hmmmm..... Maybe a not so horrible bad answer is as good as we can do.

I like the notion that productivity should be considered as rate of delivery
of value. Unfortunately, this gets into the sticky questions of value to whom
and for what purpose. An even more sticky question would be how to measure it
if you can answer the first two questions.

On the "somewhat higher up" questions. Are we to hold the programmer
responsible for such things as "conversion rates" which may be more impacted
by advertising, quality of web site, and momentary media buzz than by anything
the programmer did? If so, how is this measuring the productivity of the
programmer. Its difficult to make the connection. That is unless you are the
whole team from start to finish.

Perhaps the best we can do is compare present and past "productivity" to see
what has improved and what has not. If there is a net improvement, then
"productivity" has increased else not. Trying to put a number on it beyond
plus or minus may be a hopeless dream.

Still, for something that seems to be driving the world's economy, a hopeless
dream is not very satisfactory.

~~~
sfamiliar
no, we absolutely should not hold the developer responsible for the higher up
questions -- the responsibility should fall on the shoulders of the person who
made the decision. did the developer implement what he was asked? did he
render solid feedback which was promptly ignored? if so, he bears no
responsibility for the quality of the decision, only the quality of the
implementation. if he made the decision, of course, that's a different matter.
and in any case, the measurement of his 'productivity' is meaningless as a
part of this metric.

comparing two programmers is possible. comparing a single programmer against a
theoretical ideal is improbable. finding a remotely accurate formula for
measuring developer productivity and yielding data that's actionable in the
business space is really, really unlikely.

------
gruseom
Programmers know when they've been more productive because they feel it. This
is not the kind of measurement that is accessible to metrics and formal
studies, but that doesn't mean it isn't real. It does mean that the measurer
and the measured have to be the same person, or people working closely
together.

~~~
raganwald
So...

We can't quantify it or inspect it. We can't corroborate it. We can simply ask
a programmer whether they are productive or not and trust them to answer for
themselves.

And if Alice meets Bob on the street and they disagree over what makes them
productive, there is really no way to settle the question. If Alice says she
is more productive using Ruby than Python, but Bob says the opposite, we must
trust that they are both right, and furthermore we can't really draw any
conclusion.

Because Charles may report that he is more productive with Ruby, or Python, or
perhaps Arc, and again he is the only one who knows.

~~~
prospero
It's an unsatisfying conclusion, but is it really all that surprising? To coin
a metaphor, a programming language is a shoe; only you know if it fits right.
It might stretch out and become more comfortable over time, but it's not a
debatable point that it's pinching your toes.

Programming isn't a natural mode of thought, and programming languages aren't
a natural mode of expression. I think it's unrealistic to expect that we'll
ever find a universal best practice.

~~~
raganwald
Your _simile_ is what disturbs me: Often when programmers describe their
productivity they are really describing their comfort level while carrying out
programming tasks.

However, when we use the word "productivity" in every other aspect of
business, we are describing the production of output. We don't measure how the
output is produced, or the subjective feelings of the producers.

So what _output_ are we discussing when we talk about productivity? Or are we
using the word in a special way that has nothing to do with productiviy as it
is used elsewhere?

~~~
prospero
Point taken on the simile. It's like that like never existed.

As to the rest, I dunno. Let's consider the following scenario: every time you
wrote something down on paper, there was a chance the paper would
spontaneously burst into flame. There are strict, reproducible rules governing
when the paper would and would not catch on fire. People go to school to learn
these rules, which are varied and subtle.

In this world I've described, is the best writer a person who can write the
most essays that don't spontaneously combust? I feel like as programmers we
think that because we can easily quantify whether something works, we can also
quantify whether it's _good_. Or worse, that everything which doesn't burn
away is essentially the same: just words on paper.

It's simply not true. The fact that writing a program feels more practical
than writing a novel doesn't change that.

------
awi
Programmer productivity can be measured.

In Peopleware DeMarco and Lister explain the Gilb Measurability Principle by
what they call Gilbs law: "Anything you need to quantify can be measured in
some way that is superior to not measuring it at all."

That does not mean that it will be easy, cheap or perfect, only that it will
be better than not measuring.

Tom Gilb: "Sometimes, it seems difficult to identify satisfactory scales of
measure. Often, the only ones that can be found are indirect, imprecise, and
have other problems associated with them. From my point of view, these
problems can be tolerated. The specific scales of measure, and meters for
measuring, can always be worked on, and improved over time. In all cases, an
attempt at quantified specification is better than vague words."

~~~
raganwald
I agree with Gilb's law, and I measure a number of things on projects I lead.
However, there is a difference between measuring things that are indirect and
imprecise and measuring productivity.

So I disagree that productivity can be measured, but agree that there are
things you can measure and agree that measuring those things is valuable.

In the quoted example, Joel's eigth test measures the environment for
interruptions. This is a good thing to measure. However, this is not the same
thing as measuring productivity, because then we jump to the conclusion that
"Working alone in an office with a door is more productive than pair
programming because pairs interrupt each other constantly."

So, my thesis is that you can and should measure lots of things, but if you
are not directly measuring productivity it is misleading to pretend that you
are.

------
prospero
My personal measure is that I'm most productive when I'm least aware of time
passing. The longer the day felt, the less productive I was. This doesn't give
me any insight into the relative productivity of the guy sitting next to me,
but if we treat this as a personal quest for productivity rather than an
institutional one, it suffices.

The trouble starts once we get the idea that our personal quirks are somehow
universal.

~~~
jcs
I agree that when I am absorbed in a task, time passes quickly. But I'm not
sure it's a measure of productivity. An evening can be over too quickly if you
fill it with dinner and a good movie.

However, waking up always catches me by surprise.

------
mattmaroon
You never will know just how productive you are or were, and you're best just
accepting that. I learned from playing poker that when the quality of your
work and your results (in the short term at least) have no provable
correlation, it's easy to fool yourself into believing one way or another, but
in the end you can only look back and guess in hindsight.

The nice thing about programming (or writing, or tons of other creative
endeavors) is that feeling you are more productive will likely actually make
you more productive. If you're frustrated because your tools are holding you
back, that's going to spill over into your work and make you unproductive. If
you feel like you're getting things done, you're more likely to get things
done later. So you just use the ones you like best, and assume you'll be more
productive because they're the ones you like best.

The only time when you're certainly not productive is when you're arguing on
the internet.

------
nostrademons
Sometimes I think that success is all about luck, and so the most productive
team is the one that manages to expose themselves to as much luck as possible.
Perhaps that takes the form of number of crazy ideas thrown out per unit time.

Then again, other times I think that success is all about luck, and so the
most productive team is the one that manages to avoid blowing it when they do
happen to get lucky. Think of Digital Research, Yahoo, or Netscape. Maybe
productivity here is measured in disasters avoided.

I know for sure that productivity isn't something as mundane as effort
expended or lines of code written. Because really, the vast majority of
features you implement won't matter to anyone.

------
snprbob86
Is it worth measuring the granular "how productive" or is it enough to measure
the binary "productive or not"?

I'd argue that the biggest battle is preventing people from being
unproductive. Measure how often people have a productive state of "true", but
don't measure them against each other.

~~~
Devilboy
The question here is: How do we KNOW that methodology X makes us more
productive developers? To answer this question you need to know more about
productivity than just a highly subjective 'yes' or 'no'.

------
JesseAldridge
Here's how I track my own productivity: I take notes. I'm using Freemind these
days. I have my code at the top of the screen and a mindmap at the bottom. One
of the nodes in that mindmap is a "ToDo" node that contains various tasks. As
I work on a task - whenever I ask myself a question, or hit an error, or
whatever, (usually around every 1-5 minutes) I'll make a note in the mindmap.
I have another node called "Done" where I keep a log. Whenever I complete a
big task, I move the node and all it's children from the ToDo node to the Done
node. The child nodes in the Done node are organized by date.

How you do it isn't the important thing. The important thing is that you can
look back later and review your work.

------
aneesh
Part of the problem is that people talk about productivity in the abstract,
which is like measuring your height with an invisible ruler and saying "I've
gotten taller." Once you define a metric, it's easy to measure yourself
against it. Lines of code? Easy to measure. Weeks taken to ship something?
Easy to measure. Number of features completed on your checklist? Easy, if you
made a checklist.

None of these metrics are perfect, but they're a heck of a lot better than
nothing, or "I don't know". You can iterate, and refine the estimates as you
go along. Just make a conscious effort to measure _something_. Getting actual
data trumps all these endless arguments.

------
jimrandomh
Comparing productivity between two methods or environments is easy. You have a
programmer read and judge the quantity and quality of code produced one way,
then read and judge the code produced the other way, and decide which you'd
rather have.

Want a number, rather than a subjective judgement? Too bad. Want a manager to
make the determination, rather than a programmer? Too bad. Want to compare two
coders without showing both their code to the same judge? Too bad. Experienced
programmers know what works for them, but don't expect any good studies.

------
krschultz
Terrible title but a great question. The obvious metrics are lines of codes or
tasks completed. At our startup we have a Trac setup and make lots of little
tickets and check them off. Maybe 2 or 3 major tickets get completed per week
per developer. More than that and you are making the tasks too granular and
are wasting time doing admin overhead, less than that and they aren't specific
enough a la GTD.

~~~
whughes
He briefly mentions some of this; not all tickets are created equal, and
neither are all lines of code. Writing thousands of lines of boilerplate code
may not be as productive as fixing dozens of critical bugs without adding
anything new. Similarly, hitting on a lot of relatively simple tickets may be
less productive than fixing the toughest tickets.

~~~
jcbozonier
I agree. It's also important to remember that if those tickets didn't generate
revenue and if they would never generate revenue no one would really want you
to spend time on them.

Most businesses seem to use software as a set of automated business rules or
as a service to clients and what the owner of such companies are looking at is
profit. How much money did it cost for you to write software that made $X
million? Honestly I think that's the only metric people really care about.

What about maintainability and the SOLID principles? That's a matter of by
spending $N million on maintenance how many millions have we saved on future
cost to implement profit driving features.

If what you are doing at your company doesn't somehow lead back to profit
(even something as simple as "letting the devs do this keeps the good ones
here") I guarantee no one will want you to do it. If you do it and it does
drive some revenue, the question will be was the cost worth the reward.

The problem most companies have is it is very difficult to relate what each
task a programmer works on to each dollar of revenue earned.

------
dean
Scanning this thread, everyone is looking at this question on a personal,
introspective level. If you really want to know if you are productive, look
outside yourself. Ask your employer. Or your colleagues.

Pointy-haired bosses notwithstanding, your employer has a stake in your
productivity, and in my experience, they typically have a pretty good idea of
who is productive and who is not.

------
fizx
I personally use Pivotal Tracker's velocity feature. I write a one liner to
describe a feature, and assign it a 1, 2, 4, or 8 point value, depending on
estimated difficulty. Tracker will then graph points/time, which is
productivity. This lets me keep track of myself over time, but points vary
person to person.

It's hard to get an exact measurement of productivity, unless you have
multiple people do the same sample problems. There is no standard unit of
programmer production, because in the real world you should be solving an
unsolved problem every time (or using a library).

------
srn
I measure my productivity in a way akin to the ratio of time spent adding
features added + fixing other people's bugs over the time fixing bugs I create
and rewriting my own code.

Granted I am an "engineer" in a small company, not just a programmer, which I
guess means I do things like write requirements, evaluate new platforms etc.
IE lots of things not directly tied to implementing new features or improving
existing ones. Not sure how that factors in.

------
andhapp
I would say when one writes less buggy code in a short time he is in the ZONE
and that would depend on the problem domain, technology and programmer's
passion towards it.

