

Why Software Projects Are Terrible And How Not To Fix Them - drewcrawford
http://sealedabstract.com/rants/why-software-projects-are-terrible-and-how-not-to-fix-them/

======
nhashem
A couple years ago, when I was at a previous employer, I was lamenting the
typical corporate frustrations to a friend. The executives seemed to have a
new "strategy" every quarter. All our projects had unrealistically given
deadlines. Even when we successfully released a new product, it was basically
DOA, because they were never things we would use ourselves. It was basically
impossible to pivot our aging flagship product, because it has grown so
complex that it was impossible to get any more incremental gains without a
huge commitment in resources. At least once a month I'd be asked, "Well can we
make this change to the product quickly?" and I would always say, "No. There
are no quick changes anymore. You can either commit the staff to getting the
product back into a state where we can do quick changes again (ie. refactoring
some of it), or you can just put the product in maintenance mode and have the
staff do something else." And yet no matter how many times I explained why,
using as many MBA-friendly concepts and analogies as possible, they never
understood.

My friend asked if any of the executives and managers had an engineering
background, and I literally laughed out loud. Of course they didn't. If they
did, they would have understood concepts in this article. Instead they were
exactly like that political douchebag in The Wire video.

One of my greatest hopes for the current proliferation of startups is that
some of the technical talent there actually does end up as leaders corporate
America. I see the fundamental problem of corporations led by execs with the
classic MBA-educated, McKinsey/Bain-trained background is they've never
created or produced a damn thing in their lives, and just fundamentally don't
understand why their decisions are so dysfunctional.

~~~
sixbrx
If not understanding you is the worst that happens I'd say you're lucky, but
watch out...

My buddy and I have been resisting "quick changes" on projects that have a lot
of technical debt for a number of years, and what has happened is that
managers have just gone to some other less experienced developers who have no
scruples to get their changes done without the "fuss". Our reputations have
taken a bit of a hit as well - we're "too careful" or don't see that the
"perfect is the enemy of the good" etc. That's what they say in front of us,
I'm sure they're more colorful when we aren't listening. I've learned to
mitigate this by working with a couple of managers that aren't clueless but
it's a danger when that isn't an option...

~~~
tawman
Technical debt is a major red flag especially in a savvy organization that
realizes it's accumulating it. Especially if much of the early success relied
on it.

I was talking with a prospective client that had spent the past three years
rushing a SaaS solution to market. They even referenced the technical debt as
an area they need help with overcoming as they move forward.

Second order change is difficult for any organization or established project,
but remaining consistent with your message can drive the process in that
direction. Sounds like a challenge for the two of you, but try to bring
solutions / recommendations instead of minor resistance that might get
misinterpreted. Not easy I know.

~~~
unobfuscate
How do you spend 3 years rushing a product to market? What type of system were
they building?

~~~
tawman
This company operates in the e-medical records space and works with hospitals,
physicians, and other providers. It is more than just records management and
deals with referrals for service along with analytical capabilities etc.

Hard space to penetrated but with a lot of federally funded initiatives the
past few years. New regulations around EHR is driving the space.

------
gregschlom
>Turning around a software project is like turning around a tugboat: it takes
a lot of time and energy, and tremendous momentum.

Off-topic sailors nitpick: tugboats are highly maneuverable, what takes a lof
of time and energy is turning around a very large container ship _with_ a
tugboat

See for example: <http://www.youtube.com/watch?v=rkCqDLR2s78>

~~~
jimfl
Indeed few boats that size are as maneuverable as a tug. It's a good metaphor
for a skilled developer: even with that kind of agility, it can take a while
to change the direction of something with a huge momentum. And a software
project has momentum vectors shooting off in high dimensional space.

------
yason
You can't really fix things like software publicly. That'd never fly in a
typical company, startups excluded of course. The only thing you _can_ do is
do it secretly and there are practically two ways to accomplish that.

The first way is hard to do if there's a large group of programmers working on
the project but it's very suitable for older projects that is still under
development but with lower staffing.

If possible, do things efficiently locally (use source control, use
python/jython, generate code, do automated builds, do automated testing,
reuse, and only deliver what you must) but leave your advantageous practices
unadvertised. Then, slowly abuse your edge to steal maintenance and
refactoring time from actual to-be-done tasks and features. Eventually you can
ease your own future work and you'll get more time to work on the system
instead of the features which become easier to add.

The second way is to employ a targeted attack against a whole malfunctional
group or team at once: skunkworks projects. Do the damn thing from the scratch
and do it right, then replace the old steaming pile of junk with it before
anyone gets to say no, and weasel your way back to the surface by toting
things were just changed a bit, and never replaced.

Of course, this comes with the assumption that the evil is in the system and
your colleagues aren't actively demolishing everything you build.

~~~
ido

         Do the damn thing from the scratch and do it right, 
         then replace the old steaming pile of junk with it 
    

This reminds me of the many attempts to rewrite nethack in a higher level
language, like ruby or python.

They all fail miserably, leaving nothing but a smoking crater and piles of
code that is never to be used.

Turns out all those millions of lines of c code written over a period of 25
years actually do quite a lot.

~~~
gbog
Yes, this point of view is well known and well documented.

But the opposite may have some weight too. I think I read somewhere that
Google codebase had been rewritten from scratch. Maybe it was early enough,
because I suppose rewriting it today would be painful. Are there any other
successful rewrite from scratch stories? Maybe vim?

~~~
robfig
Google doesn't "rewrite" things in the traditional sense.. they develop new
versions of old systems when the old system has reached the limit (or is just
grossly inefficient). The new systems generally are not at all architecturally
similar. I don't think I've ever heard of a rewrite "because the code was
messy" or anything other "it can't do X and we need X".

------
pagekalisedown
Original source of the dilbert comic in the article:

<http://dilbert.com/strips/comic/2006-11-05/>

dilbert can be painfully accurate, to the point where you have to stop reading
it to avoid depression.

------
Natsu
People who have too many problems dump them onto someone else. Such people in
management make life miserable for anyone below them. Problems tend to move
around in circles and pointing out an issue is a good way to have it boomerang
back to where it becomes yours to solve.

It's much better if you have coworkers and especially management with whom you
can cooperate. If there's at least enough flexibility to improve things, it's
possible to have things snowball in reverse, as it were, where each person
tackles a piece of the problem and it gets taken care of.

But it's hard if you work with people who don't care. For example, it can be
frustrating to get blamed for causing problems that have always existed, with
no one noticing them, simply because people were too far away from the
consequences and those who suffered the consequences had no idea about their
cause.

And yes, even simple things like "source control" can be a tough sell when the
devs insist that keeping the .c file in the folder with the .exe counts as
source control. For a nearly 30-year-old DOS program, written with ASCII menus
(think printf, not ncurses), uses only statically assigned globals (malloc?
what's that do?), deeeeply nested ifs, that directly writes outputs to
hardware I'm not sure anyone at the company still understands, and there's a
custom build for each install? I'm just grateful I have nothing to do with
"maintaining" that source and that DOS makes it easy to do backups via xcopy
and a network drive. In retrospect, I might have been happier not knowing what
was in those .c files.

------
wundie
UX comment on this website: I like to click and highlight words as I read. I
know I'm not the only one who does this. On this site when I do this, the
article keeps drawing focus to the top of the page and its...annoying. Why
would someone do this? I've never experienced this issue on any other site
before.

~~~
civilian
I thought I was alone in that preference! A designer was watching me read
through an article and he was like ".... why are you clicking everywhere?" It
helps me keep track of my place, and I'm ready to control-copy even faster.

------
cateye
There is a strong correlation between terrible projects and the value that the
project will deliver.

The lower the future value and the uncertainty that it will bring the value,
the more problems the project will experience.

So, if you as a developer ask "how is this going to deliver value (make
money)?" and you don't get a convincing and logical answer the more problems
you will experience.

The easy part is, the bad projects are really easy to identify because no one
on the project can tell something argumentative.

\- "You have to build a video website in 1 week!"

* How is this going to make money?

\- We will charge every user for every video they upload

* So, it is like Youtube but paid. Why will people use this instead?

\- BSOD...

------
mattmanser
Has this guy ever even read Joel's stuff?

The acronym TDD keeps popping up time and again.

Joel's not a TDD fan last I heard him talking about it (admittedly
stackoverflow podcasts a couple of years back). He certainly wasn't a fan of
it when the Joel test came out because that pre-dated TDD.

Fact is, while I agree with some of the points in the article, there are
things that aren't accepted practice and TDD is one of them.

Personally I think it's a colossal waste of time apart from some complicated
methods.

And don't even get me started on the esoteric 'scrum' or 'agile' where
everyone has their own pet definition of what it _really_ means.

~~~
yannickt
He never said or implied that Joel is a TDD fan in the article. He also
concedes that TDD might be a controversial practice in the first paragraph. In
any case, the merits of any particular practice have little to do with the
main thesis of this blog post.

