
The miserable programmer paradox - cool-RR
http://blog.garlicsim.org/post/2840398276/the-miserable-programmer-paradox
======
edw519
_A good programmer will spend most of his time doing work that he hates, using
tools and technologies that he also hates._

Oy, another "good programmer, bad programmer" post.

A smart accountant once told me that the answer to "How much money did you
make?" is always, "Who wants to know?" If it's an investor, the answer is "A
lot." If it's a customer, the answer is "A little." If it's the IRS, the
answer is "None."

Same thing here. The answer to "Who is a good programmer?" is always, "Who
wants to know?"

To a project manager, the programmer who hits every deadline (regardless of
quality) is a good programmer.

To a customer, the programmer who solves their problem quickest is a good
programmer.

To a business owner, the programmer who makes them the most money is a good
programmer.

To a PHB, the programmer who makes them look the best is a good programmer.

To a journalist, the programmer who tells the best stories is a good
programmer.

To a junior programmer, the best mentor is the good programmer.

To another programmer, the programmer they are most likely to want to go into
battle with is a good programmer.

To a blogger, the programmer who best fits the profile of the point he is
trying to make is a good programmer.

~~~
cool-RR
I understand that many people read my essay as if I was trying to judge who is
a good programmer and who isn't. That wasn't my intention-- I just wanted to
reflect on a problem that good programmers have.

~~~
Stormbringer
Your essay is a tautology disguised as a morality tale.

You define a bad technology as one that can't be automated.

And then, those things which can be automated, will be.

Automation speeds things up, hence the majority of things left to do are the
things that can't be automated.

Which you already defined as bad.

Hence the majority of things for a programmer to do are 'bad'.

====

Anyway, the thing that leaps out at me as laughably bad is your editor where
you have to type in a particular line number.

Dude, it's 2011. Even Eclipse (which I despise since it has a truly shitty UI
which keeps trying to reduce my coding area to the size of a postage stamp)
has a feature where you click on the bug and it takes you to the line that the
compiler reported. And it has a nifty feature where it shows which lines are
broken.... which has only been in Visual Basic _SINCE THE FRIGGIN' EARLY 90s_

I'm gonna have to assume that you're using Emacs or Vi or some piss-ant
command line equivalent. Let me tell you guys, the holy war is over... _YOU
BOTH LOST_!

Emacs vs Vi reminds me of the Iran-Iraq war. A war that went on for ages, in a
place that nobody cared about, between two groups that nobody cared about,
that stagnated and stunted their development, while the rest of the world left
them behind.

~~~
wladimir
Why do you get so frustrated? The guy was just giving the 'line number' thing
as an example of something that can be easily automated and saves a lot of
trouble. He wasn't flaming for some holy war, he wasn't saying jumping to line
numbers is the most advanced thing.

And the comment about the iran/iraq war is simply disgusting. I don't
understand why someone voted you up.

~~~
Stormbringer
I think you're reading the tone wrong.

But you're completely wrong about your last objection. The thing that is
disgusting is war, what makes it even more disgusting is when nobody cares
because it is 'just a bunch of brown people killing each other'.

The complete apathy of western nations towards wars in distant places and
genocide is disgusting. What the Kurds are going through and have gone through
is disgusting. What is going on in Africa is disgusting. America trying to ban
the promotion of abstinence in Aids torn countries (because it isn't
'culturally sensitive') is disgusting. What the Arabs did to the Jews back in
the 60s (booted them out, seized their property without compensation) was
disgusting. What the Arabs did to the Palestinians back in the 60s was almost
as bad. That certain factions in western nations want Israel to make
concessions (and get nothing back in return) with people who have written into
their constitutions that the Israeli state should be abolished by force, is
disgusting.

The use of children as soldiers, is disgusting.

What America did to Indonesia is disgusting (crushed a democracy, put a
military dictator into power). That the Dutch were so oppressive that the
conquering Imperial Japanese were seen as liberators, is disgusting. What the
Japanese did to captured chinese women is disgusting. The Japanese treatment
of POWs was disgusting. The American arrest and imprisonment of their own
citizens of (distant) Japanese descent during WWII is disgusting.
Extraordinary rendition is disgusting. Tazing people in wheelchairs or already
behind bars is disgusting. American cops beating the crap out of ordinary
black citizens for no good reason is disgusting. American treatment of
immigration issues is disgusting. What the U.S. did to native Americans is
disgusting. Guantanamo bay is disgusting.

What the British did to the Jews in the early 20th century is disgusting. That
the Arabs thought Hitler was a great guy because he wanted to kill all the
Jews is disgusting. What happened in Bosnia/Serbia is disgusting. What the
British did to the Cypriots is disgusting too (betrayed them to the Turks).

But no, please feel outraged _at me_. I'm sure that is the appropriate
response.

Speaking of Iran, wasn't there a democracy, that the US overthrew, tried to
put a puppet in power and ended up with a rabid militant theocracy instead?
And _I'm_ the disgusting one? Seriously?

Did I mention the Americans regularly overthrowing democracies and trying to
put puppets in their place? Has that ever worked for you guys? Why do you keep
trying the same thing that fails over and over again? Are you insane?

------
nostrademons
This is the human equivalent of Amdahl's Law. Say that you have a boring task
where 90% of it can be automated, and that part can be sped up by a factor of
100. Now, the workload that used to take up 90% of your time takes up only
0.9%. The part that _can't_ be automated remains constant, however, and so you
now spend over 90% of your time doing work that can't be automated. The total
amount of work you can do increases dramatically, but a much higher fraction
of it (by time) will be distasteful.

In practice, it doesn't seem quite as bad as the article makes it out to be.
I've found that by now, I spend the vast majority of my workday learning
things, and the limiting factor in my productivity is how fast my brain can
learn. Typing speed ceased to be a bottleneck in high school when I got up
around 90 WPM. Thinking speed ceased to be a bottleneck a couple years after I
started programming professionally, once I became fluent enough with the
syntax and basic libraries of the languages I typically use that I don't have
to reach for the docs all the time. The bottleneck is now learning libraries,
data sources, requirements, and algorithms that I _don't_ typically use. And
while this can be a little annoying - it's mentally exhausting and not great
for the ego that my productivity is limited by intrinsic problems with my
brain - getting paid to learn things all day is a pretty nice state of
affairs.

~~~
Stormbringer
Your math is wrong. 10% (cannot be automated) + 0.9% (can and has been
automated) = 10.9%

What you do with that extra 89.1% you just liberated is unspecified. You're
making an assumption about what happens to that extra time, I think you should
be more explicit about it. You say:

 _"The part that can't be automated remains constant, however"_

You need to back that up somehow. I am unconvinced that this 90:10 ratio is
some cosmic law.

Also, you are buying into the original authors deeply flawed premise that work
that cannot be automated is tedious and distasteful.

I respectfully completely disagree. You have it exactly backwards. The tedious
and distasteful stuff is first in line for automation, not last. The stuff
that _cannot_ be automated is _always_ the most interesting stuff, because it
is the bit that requires you to engage your brain.

Example: automation can give you really really bad software design, but for
good software design nothing comes close to doing it 'by hand' (or rather 'by
brain').

------
neilk
Wow, I'm amazed how few people seem to be reacting the blog post the way I
read it. Did the text change?

His argument isn't about good or bad programmers, it's an observation about
programming: the annoying stuff can't be automated. Good programmers automate
whenever they can. Therefore, the author believes, annoying stuff will
eventually dominate their whole day.

Maybe your Ruby on Rails, or Scala on Snowshoes, or whatever, framework makes
it trivial to create a functional site from twelve keystrokes. That just means
most of your day is spent in customer meetings, gathering specs, and fixing
obscure bugs.

There's some truth to this, but I also enjoy designing and communicating,
which are also non-automatable. Perhaps the author doesn't like that stuff,
but if so, that's sad.

And there are always ways to automate more and more; you may have to leap to a
different paradigm to capture the essence of truly thorny problems, but this
should be fun for a good programmer.

~~~
aamar
Good point; I think this is the right response. The author is making a good
observation, but I'd suggest tweaking the statement of the paradox:

"An increase in programmer skill will increase the fraction of programming
time spent doing work that he or she hates, using tools and technologies that
he or she also hates."

This version reduces the suggestion that the paradox is a definition of a good
programmer. Both my statement and the original accept at face value that
programming skill leads to automating more successfully.

In my experience, this paradox is frequently, but not always, true depending
on the programmer's temperament and situation. This 2x2 illustrates:

    
    
                    |   Automatable   |  Not automatable
     | Pleasurable  |  Game/"Sudoku"  |   Deep problems
     | Unpleasant   |    Busywork     |    Frustration
    

An increase in programmer skill means shifting time from column 1 to column 2,
while not necessarily keeping it in the same row. That means a happy decrease
of busywork but also the loss of what I think of as Sudoku-work, the technical
work that feels sort of fun and pleasant mental exercise, but work that can be
easily automated or obviated given a decent language/working-environment.
Sudoku-work is technically productive, in a way, but certainly not optimally
valuable.

The paradox is based on the theory that Sudoku-work time is lost and turned
into frustration time. But there is a way out, one that seems partially
fixable via programmer mindset: deconstruct frustrating work into busywork
(and automate it) and deep problems -- the latter including deep technical
problems as well as gaining insight into clients/users, design, and so on.

edit: minor clarification, punctuation fix

~~~
neilk
Whoa, excellent insights. This could be an essay on its own. The ability to
enjoy Sudoku-work is something of a defining characteristic for programmers,
too.

There's a certain overlap here with the theory of "Flow":

<http://en.wikipedia.org/wiki/File:Challenge_vs_skill.svg>

<http://en.wikipedia.org/wiki/Flow_%28psychology%29>

It seems to me that programming is special, because repeated tasks tend to be
automated away. If you ever achieve Flow doing one task, you'll never achieve
Flow there again because you won't _need_ to.

------
jcr
The general observation of the author is correct, but his claimed reasons for
it are flawed; it has little to do with breaking concentration.

I was at a party with a dear friend of mine, Pat Davis, who is an expert PCB
layout engineer at Millennium Design (www.millenniumdes.com). When I was
healthier, I used to do some occasional PCB and automation work for his
company. While the two of us were talking with another mutual friend, I said,
"Working with me can be, ummm, interesting. I have zero patience for poorly
built machines and programs." And then I asked Pat for affirmation from his
own experience working with me.

Pat replied, "Sure, you can be tough to work with, but there is a real upside
to it; if I never want to do something manually again, I just make Jon do it
three times. He'll lose his patience and automate the living snot out of it."

As much as I hate to admit it, his answer was completely truthful.

~~~
wccrawford
Put me in the 'automate the crap out of it' category, too.

We had a sysadmin that spent most of his time on 2 tasks, 1 was an internal
maintenance task, and the other was email.

When he quit, those tasks fell to me, the sole programmer at the time. THe
internal maintenance task, I did it for about a week before I decided that it
needed to go away. So I wrote scripts and an interface for it, tested it for a
while, and turned it over to customer service, which was the department that
was requesting it anyhow. They were ecstatic because it meant they no longer
had to request it and wait... They could just do it instantly. Much better for
everyone.

The email was easier. We just moved to Google Apps.

The remaining tasks that the sysadmin did were now just a tiny fraction of my
time.

As the company grew, those tasks increased until management realized they
needed an actual sysadmin again, and they hired one.

~~~
jcr
Very much agreed. The real trouble is, the automate-everything mentality is
unfortunately very rare due to the simple fact that if successful, you've just
made your own job unnecessary. I actually want to work my way out of a job,
but most people are far more concerned with keeping their job security.

~~~
bane
It can be true, but I've found as often as not that work can grow to fill your
time like a gas. No matter how much of it you can automate away, it simply
means you can do more stuff.

~~~
maxawaytoolong
The key to avoiding this is to stop thinking like a programmer, and start
thinking like a sysadmin.

A programmer will automate some tedious repetitive task, and then boast about
the automation to his programmer peers and managers. The programmer will then
be rewarded with more tedious tasks to deal with.

A sysadmin automates a boring task, and tells nobody. The only way anyone can
request work done is to file a ticket in the ticket tracker. The work gets
done, but nobody besides the sysadmin knows how it was done. If the sysadmin
is good at his job, he is rarely seen at his desk, and often seen playing
foosball.

~~~
neworbit
No wonder people's impressions of sysadmins are not overwhelmingly positive.
(I'm envisioning Wally from Dilbert here.)

If you can replace someone's job with the proverbial small shell script (or
apt-get install $OPENSOURCEPROJECT), sooner or later that is what will happen.

------
axod
I disagree. A bad programmer is the guy that says "I will only use lisp!
Everything else is terrible".

A good programmer just wants the thing to work and will use whatever is best
for the task.

The solution is for programmers to learn more technologies and stop being so
fanboish with his favorite. Rise above the language/tool. They're nothing to
do with real programming, which is solving real world problems.

Good programmers don't think about problems in a specific language, they don't
read code in a specific language. It's all just code. Languages are just a
crappy way for humans to write machine code. That's all.

~~~
evilduck
Not to rant or single you out, but I'm not sure that "programmer" is a
sufficient descriptor in this thread anymore since everyone seems to have
their own little definition, and I'd argue that solving real world problems is
only one of many attributes of "real programming".

Is game programming solving a problem? What about music visualizers? They're
both fun distractions, but to call them "problems" is stretching the
definition. What about malicious programming efforts? What about using
programming languages for the challenge of it (brainfuck, whitespace, etc)?

Refusing to use anything but lisp may make you a poor candidate as an employee
or teammate or entrepreneur or whatever, but says nothing about your ability
to use lisp effectively.

~~~
axod
Game programming, music visualizers solve the problem "Make something people
want".

> What about using programming languages for the challenge of it (brainfuck,
> whitespace, etc)?

I think this is one reason lots of programmers get into a rut where they only
want to use a specific language and get depressed using anything else. Using a
programming language for the challenge of it is good fun, but it's not really
a worthwhile use of effort often.

I agree, 'programmer' has many definitions, but at the end of the day, we
"program" to make a product that we or others can use. How that product is
made and what tools were used isn't that important. It's the fact it's made at
all that matters.

------
PaulHoule
When my family sees me working I'm afraid they'd get the impression that I
hate my work. It seems like I'm always dealing with something that doesn't
quite work right.

I had a conversation with the salespeople in my organization that got me
thinking about it too; as a developer, I was always talking about what was
wrong or missing in my product, not what was right about it.

If you're the kind of programmer that the "buck stops with" you'll always be
frustrated with what you've got because your whole job is to make it better.

------
Nervetattoo
"Ignorance is bliss". The problem is rather universal. Put a good "X
professional" to do job Y and a bad X to do the same job, it will be more
miserable for he who is good because he has more knowledge and is better able
to realize all the reasons why job/task X is a miserable sorry job.

On the other side you could argue that a good programmer would be able to
abstract away a proportional amount of miserable work to a bad programmer,
thus leaving them equally miserable.

In the end its your attitude as a person that decides if a task makes you
miserable. If your attitude makes everything you do into a job — some piece of
tedious labour you do at someone elses whim — then no wonder you are
miserable.

------
Swizec
>I wish I could end this essay with some sort of moral, or an idea for solving
this problem, but there’s not much. All we can do is try to cut as many bad
technologies as possible out of our lives, and just tolerate the rest.

The author said they can't extract a good moral, but the moral is really self-
evident. Don't distract programmers!

------
waivej
A miserable programmer is a miserable person and won't be happy with good
tools and technologies. Likewise, a "skillful" programmer could be a like a
skillful mechanic that builds a good set of tools.

The best mechanics I've known ended up working in jobs where they like the
cars they maintain (Honda, Porsche, classic car restoration). The less
skillful ones seem "trapped" working with cheap parts or cobbling things
together and not doing a great job of it. They tend to have excuses about why
the cars or parts are engineered poorly. I've also known very skillful
mechanics that are proud of their ability to keep beat up old cars running
inexpensively regardless of the brand and making their own tools rather than
buying fancy ones.

Likewise, a good programmer will know the limits of the systems they are
working in and will strive to not do quick fixes that cause problems later.
They may also make their own tools and libraries...and they will move to
companies with better politics or projects when opportunities arise.

------
ZeroMinx
I think another point is that programming is generally started as a hobby,
because you love it. You might spend years and years doing it for fun - you
are in control what you're doing, you do things that you like.

Then, it becomes a profession. Suddenly, even if you're doing the same things,
it's now something you _have_ to do. All of a sudden it's not that much fun
any more.

~~~
thret
True of all hobbies turned professional.

~~~
ZeroMinx
Very true indeed, but this particular post was about programmers :)

------
Raphael_Amiard
I think that in today's technology world, if you really care about the
technologies you are using, they are plenty of positions with reasonably good
languages, where tools are not imposed because you're working on a web
product.

I use python professionnally, but there are a lot of ruby openings to, some
scala and clojure now, and even C# is really far from being a bad language.

I do hate to program in Java, but there is plenty of room in the tech world to
let me not program any line of Java, and let that to people who like it, or
simply don't mind.

So my question is i guess, if you are indeed a good programmer, and hate your
tools, what stops you from learning one of the "bearable" languages that are
plenty used in the industry today, and find a job where you can use it ?

~~~
cool-RR
Even in my open-source project, where I make all the decisions, I have to work
with technologies that annoy me. If you want to actually ship working
software, you have to make some compromises.

------
swombat
I must sadly agree. Woobius is written in Flex (Java-like ActionScript 3) and
Rails. Guess which I've spent most time in over the last 3 years?

~~~
merijnv
This is of course a case of the True Scotsman fallacy, because the author can
argue (following the posts logic):

\- Good programmers are stuck spending most of their time on - tools they
hate. \- You do not spend most time on a tool you hate. Therefore, you must
need be a good programmer.

And tada, irrefutable argument and a linkbait blog post subject.

------
agavin
I would like to add a corollary to this theorem. The best programmers are also
willing to work with the annoying technologies -- although they will gripe
(healthily) about them. It's all a balancing act.

There is a type of programmer who could be great, but is merely good because
they refuse to use or deal with annoying techs. Sometimes, you just have no
choice, and brute forcing past the annoying is by far the most efficient
solution.

You all know this type. They are the kind of guy that exclusively used the
Next (or Amiga), and ran windows in VM Ware on a bleeding edge Debian install
in 2000 -- When the software they were being paid to develop was compiled in
Visual Studio!

That's the thing. I hate MSFT. I hate Windows. I'll avoid using it. But... if
the task requires that a regular PC GUI app be written, I'll use Visual
Studio. Although I have been known to ask less PC phobic employees to handle
this part of the job :-)

Of course, I also decided for an entire series of Playstation 2 games to
forego doing the normal thing (using C/C++) and wrote my own Scheme
compiler/debugger/environment. And of course, this quickly spiraled into
several man years of work (on my part). It did come out pretty well. We did
four games, and they sold tens of millions of copies -- dozens of programmers
used the system and it was pretty cool. But boy was it stressful. I really
should have figured out a way of solving my problems that didn't require so
much custom programming to solve a sub-problem of my task.

My main task was: Program an absolutely fantastic 3D action game in 2-3 years.

I got sidetracked by the theory that "C/C++ sucks for writing highly
customized game code" and that the compile/link/run cycle of 10+ minutes made
iterative testing (which is the mother of all good gameplay) slow. This is all
true, but it's a less focused problem than making a great game.

Avoiding this or that technology, or using this or that more fun one is not
the same as getting the job done.

That being said, programming in Lisp, or Ruby feels like walking on water
compared to C/C++'s being wrapped in chains and pushed off the boat. You just
don't want to stroll off toward Australia when your gig is in Japan.

------
Luyt
When I find myself spending time over and over again on repetitive tasks like
fighting an inadequate editor or tools, or writing CRUD-like code over and
over, I take some time to automate away the nuisance, for example, switching
to a more appropriate editor, or refactoring out common code and only describe
the differences from there on, or maybe find a better tool for the job.

I have experienced coworkers who didn't do this. They figured out how to do
certain tasks and keep repeating it into eternity, without giving a thought to
how they could work more efficiently.

------
ebiester
How much of this discussion would have been easily replaced by "most?"

Most programmers above entry-level try to automate away tedious pieces.
However, there's an easier explanation for the miserable programmer paradox:
most programmers get good at the pieces they enjoy; thus, those pieces take
less time. We tend not to get as good at technologies we dislike (Pure human
nature -- if you don't like it, you're not going to play around with it as
much). Thus, they take longer.

------
CrLf
The example of going directly to a particular line is not a good one, IMHO.

I find myselft often doing stuff that I know that my editor can do faster.
Scrolling is one, but there are other examples. This not only doesn't
interrupt my train of though, but it also works as "time to think".

------
JonnieCache
Yeah.

If nagios were written in ruby/python/javascript I'd be a lot less miserable
right now.

I demand executable, object oriented configuration files! It is my basic right
as part of the pursuit of human happiness, as the OP has now shown
conclusively.

------
didip
I felt like the post isn't quite finished. He could have gone into greater
depth.

That said, to me, a great programmer can be measured by how much fun he is
having when programming. Not by how much he hate particular tasks.

Programming requires constant act of learning, that gets old real fast if you
don't love what you do.

I admire programmers who can take shitty tools that he hates, cleverly hack
it, and make something cool with it.

------
techiferous
This is similar to what I experienced on a bike tour: even though on hilly
terrain the distance you are pedaling uphill equals the distance you are
traveling downhill, you are spending 90% of your _time_ going uphill.

------
rfugger
This is why I hate long bike trips: you spend 90% of your time pedaling
uphill.

------
ahoyhere
There's a moral, but the author couldn't find it. The moral is not "don't
distract programmers" but:

Programmers need to grow up.

If you program just to program, or "solve problems" as so many put it, then of
course, you will be miserable when your pure programming is disrupted.

But work on something bigger - something you believe in - which gives you a
big-picture outlook, you'll look at programming as the means to your end. And
you will suffer a lot less. Even with no technical change. Because the
pleasure you experience will be mostly to do with achieving your bigger goal,
rather than (just) crafting the most beautiful line of code.

So many of the programmers I know set themselves up to be used as tools by
people with ideas, because they claim they only care about "solving problems."
So they get themselves hired by people with "interesting problems." And then,
because they end up miserable, they quit in short order (1 year, 2 years, or
less)... and repeat the cycle again.

It's not that jobs are inherently terrible, but that the way certain groups of
very smart and capable people LOOK at jobs.

Granted, lots of people who want to "do a startup" do the same thing. They
look for an interesting problem, not a bigger thing they want to achieve in
the world. Their work is all about "the problem," and so they get stuck... and
work on it for 1 year or 2, before they quit and look for another, more
interesting problem.

~~~
cool-RR
_"But work on something bigger - something you believe in - which gives you a
big-picture outlook"_

What is this "something bigger" and how is it any better than the goal of
"living a nice, comfortable, intellectually-satisfying life"?

~~~
mechanical_fish
That's something bigger. And if having to (e.g.) constantly code HTML in IE6
interferes with that goal... you need to stop constantly coding HTML in IE6.

The underlying fallacy of the original essay is that the life of a programmer
consists of a semi-infinite series of programming tasks that only stop when
you die. Well, yeah, if you think like an assembly-line worker this is true,
but you need not do so. In real-world projects, you do have to deal with HTML,
but you don't have to do so _to an infinite extent_. You only have to write
enough HTML to make all the web pages you need to fuel the rest of your
comfortable, intellectually-satisfying life. (For some, this is "near zero":
PG wrote all the HTML for this site five years ago, probably in less than two
hours, and it's working out just fine.)

HN has been over this many times. If you don't enjoy ritual grunt work, get
out of the piecework business and solve a problem. [1] Nobody really knows how
much work, say, the folks behind the scenes at 37signals do. We don't pay them
by the hour, or by the line of code. We pay them for their product. If they
can produce the product in half the time, they get the other half of the time
to invest in other things. They don't have to spend that extra time
programming in systems they hate.

\---

[1] This ties into my Programmer Aging Hypothesis: Old "programmers" are hard
to find not because their brains wither and die, or whatever, but because most
of them have learned enough to brand their work as something other than
"programming".

