
Why Does Software Rot? - nhaliday
http://www.overcomingbias.com/2016/06/why-does-software-rot.html
======
fallous
In most of the cases discussed in the article, it's not the software that rots
but instead the users and/or organization that decays. When a system is
created, it is generally designed to solve a known set of problems encountered
by an existing population of users. Over time, the tool creators, original
users, and the current problem domains change but often the old system is
modified to solve these changes. A flexible system may be adaptable to certain
amounts of change, but only if the current populace of creators/maintainers as
well as users understand the limits. If they do not, the software often ends
up less useful than if it had never been touched.

~~~
specialist
_"...it's not the software that rots but instead the users and/or organization
that decays."_

Orgs decay, sure.

But I believe the OC's point was that orgs become more brittle over time as
they specialize. Conway's Law would suggest that software architecture mirrors
org structure, so it too would become more brittle.

Applicable cliches: "victim of one's own success", "the complexity
catastrophe".

Going the full meta here, I'm okay with the death-rebirth cycle for orgs. Yes,
a lot of knowledge (experience) is lost. But forgetfulness is also crucial for
learning, adaptation.

~~~
nickpsecurity
It was nice up to this: "But forgetfulness is also crucial for learning,
adaptation."

In IT, it's the opposite: history repeats itself endlessly with same flaws,
same missed opportunities, and same techniques recreated due to often-willful
ignorance of the past. One of the things I do here is get old or even current
work to people it might benefit. The number of times the old stuff applies to
current problems, but was never handed down to those people by predecessors,
shows the problem we need to work on is maintaining, packaging, and delivering
prior wisdom. Interestingly, the other engineering disciplines already do that
a lot better with IT remaining pretty stubborn. To put loss into perspective,
it took some groups 50+ years to reinvent benefits of Burroughs B5000 and
ALGOL. Just one example among many.

~~~
specialist
I agree with you.

By "forgetfulness", I also mean the stubborn clinging to the old ways, because
reasons. The necessary dying of the old guard, a la Kuhn's Structure of
Scientific Revolutions.

But I also mean that each new cohort apparently needs to rediscover the deep
truths (first principles) for themselves, to earn the wisdom vs blindly
accepting received knowledge. I have no idea why this seems to be the case.

    
    
      When I was a boy of fourteen, my father was so ignorant I could 
      hardly stand to have the old man around. But when I got to be 
      twenty-one, I was astonished at how much the old man had learned 
      in seven years.
    
        -- Mark Twain (disputed)
    

Perhaps forgetfulness is the wrong word. I'd don't yet have another. Some
trite phrase for the culling of the herd, gouging out the dryrot, burning off
the grasslands, aggressive RIFFing of the tentured... That also conveys that
we must give people the room and opportunity to learn things for themselves.

Maybe what I'm advocating is the Socratic Method. Versus telling people "just
because".

Thank you.

~~~
nickpsecurity
"By "forgetfulness", I also mean the stubborn clinging to the old ways,
because reasons. The necessary dying of the old guard, a la Kuhn's Structure
of Scientific Revolutions."

Well, that's true. I think we might need to distinguish between what ideas are
lost or are not what a new group is clinging to. Often comes with new people,
young or old, coming into an organization. Maybe also distinguish between
"current thoughts on what's best method" vs "known methods along with their
benefits and problems."

"Maybe what I'm advocating is the Socratic Method. Versus telling people "just
because"."

Well, there's how we publish the knowledge and how they learn it. I'm for
traditional reporting with empirical focus, less silos, easier search, and
easier distribution. Far as _learning_ , Socratic Method is one among many
methods that might have potential. It's too big a topic for me outside brief
work I did on using various parts of brain simultaneously to increase recall,
incremental problem solving of increasing complexity, matching training to
realistic scenarios, and side-projects that are open-ended to match fun &
curiosity. Don't think I did anything else in research on the topic. Well, a
M.C. test generator and some expert system prototypes but they were crap
honestly. :)

------
brownbat
Note, article is not talking about bit rot, I think that's confusing a lot of
people.

The point might be clearer as:

"Adaptability and efficiency are opposing priorities."

Ecosystems face this too. A stable environment will lead to adaptations that
improve efficiency, while creating new dependencies on everything staying the
same. In a sense, species are constantly competing to make the ecosystem more
fragile.

If this holds, there are broad impacts to information systems outside of
software. broader impact of this. We like to fantasize about the mind being
immortal. Maybe we could fix the telemere thing, figure out cancer, hop our
brain to a clone, or upload our consciousness to some cloud.

But in my experience, being mentally alive involves some mix of plasticity and
progressive refinement. You can't have both forever.

~~~
SilasX
There's a similar relationship between "regularity/predictability of an
environment" and "how much optimization is possible".

~~~
TeMPOraL
Not sure which way you say the relationship goes.

My first guess would be that the more regular/predictable an environment is,
the more optimizations are possible.

~~~
SilasX
That's correct. Optimization isn't really possible in a "white noise
universe".

The less regular the universe is, the more general and less specialized your
solution can be.

------
jussaskin
Software rots if you can no longer set up the exact tool chain and environment
(compiler, build tool, OS version, external database, etc) required to build
and run it. Even interpreted languages suffer from this problem as features
are subtlely changed - by design or accident. It doesn't matter if your
project is using an ancient version of VC++ or a two year old version of
NodeJS. If it doesn't keep up with the latest releases of the tools then it's
already got one foot in the grave.

~~~
ArkyBeagle
Through the miracle of virtual machines, this is much less so. VMs are the NAT
of computer systems.

~~~
jussaskin
Less so, I agree. But give it time - older VM image and container formats will
probably rot as well.

~~~
smilekzs
VM formats are easier to migrate than the actual software they run.

On the other hand this is a valid concern for containers.

------
boznz
Unless your sandboxed in a self contained hardware and software environment
(such as an embedded system) you will eventually be screwed. (rot sounds like
a gradual degradation its generally not in my position)

Standards changes and OS Updates are the biggest culprit; The Vista update
broke a couple of my old programs (written for Windows 95) due to the user
access control changes, another was broke due to the fact it interfaced to a
piece of hardware on the parallel port and the parallel port and the
manufacturer went the way of the dodo.

I have seen a stand-alone DOS 6 program running a machine at a factory. The PC
has been replaced three times now is only a few years old but the operator
says it still does the job, I also have a 8051 powered clock I built in 1987
that still happily ticks along if I plug it in.

~~~
ArkyBeagle
This is because there's little glory in simply making things work. There is
glory in making things that look shiny and new. As they say, "Absolutem
Obsoletum".

And congratulations on understanding exactly why I prefer embedded systems -
you don't have to keep chasing the dragon of the fashionable new thing ( which
is invariably old wine in new skins ). It is not that there is no new wine it
is just that proportionally, it's less than all new things on offer.

FWIW, if you look carefully there will always be a motherboard available with
a working parallel port. They're not cheap, and at some point the thing
plugged into that parallel port wears out.

~~~
nickysielicki
semi-related: a couple months ago I found a large industrial shipping scale in
my parents basement. I grabbed an FTDI USB -> RS232 cable and a DB25 -> DB9
cable, plugged it into a Raspberry Pi, and hacked this together:

[https://gist.github.com/sielickin/8cc79f0cb6a4b4c229b9786dff...](https://gist.github.com/sielickin/8cc79f0cb6a4b4c229b9786dffcabdbe)

There are some nasty hacks in there and probably a few bugs that I could fix
up at some point, but it works just fine!

I love how in a little more than 100 lines of Go code, I can so easily have a
program with a dedicated thread to reading data from the serial port, a
dedicated thread to serving up the initial http page to clients, a dedicated
thread for sending data to websocket clients, and all of it works together so
cleanly with messaging through channels.

I copy-pasted some d3.js code from around the web, retrofitted it, and in
maybe 2 hours of work I have something pretty cool.

My Mom runs an ecommerce website and uses another scale for shipping, I wonder
how hard it would be to write a driver for Windows that consumes data over the
network and would interface with her shipping label software.

~~~
ArkyBeagle
That's amazing - thanks for showing me that. I have much to learn that-a-way.

------
lifeisstillgood
Three reasons:

1\. The code is the same but the people using it forget / never knew the
reasons it was built that way - and so it looks rotten for the job

2\. Because requirements change and people try to make the old code do new
things without cleaning up / refactoring correctly - so it now does neither
job well, and looks rotten.

3\. Because the environment / platform changes, the FTP server is moved to a
new data center and the timeouts kill the jobs etc. It looks rotten.

"Rot" more accurately is just not keeping the code inside the code base up
with entropy _outside_ the code base

~~~
cpeterso
Or when the operating system or build tools change or are no longer available.
Your code hasn't changed and suddenly it no longer compiles.

------
WorldMaker
An interesting simile here is that software rots in the same way that
Encyclopedias do.

(This is admittedly a fitting simile partly because the simile itself is being
rotted by software like Wikipedia.)

In the world where new Encyclopedias (and Almanacs and Recipe Books) were
printed and sold on an annual basis, the question was often why do we need
"this year's Encyclopedia" when the old one is still perfectly valid. Books in
general decay pretty slowly and have a long shelf life, but the facts and the
views in the world inside them are frozen and possibly. Changes from year to
year of an Encyclopedia are somewhat hard to notice, but in Middle School in
the 90s I recall having to compare articles from a tobacco yellowed
Encyclopedia set from the 70s to trips to the same articles from very early
predecessors of Wikipedia. The worlds contained in those two sorts of
Encyclopedias were very interestingly diverging. The yellowed Encyclopedia's
facts were almost all still valid and "worked", but there were things that
didn't hold up and lots of new facts that needed to be inserted in various
places. If I were to edit an Encyclopedia, I'm not sure I would start from the
version in that yellowed Encyclopedia if I could find a more recent set. Some
of the predecessors to Wikipedia were direct descendants of that yellowed
Encyclopedia and yet for various reasons historical and technical, Wikipedia
itself did not inherit directly from that set in any meaningful way.

(It's interesting to note too that the physical media of software to date has
a much shorter shelf life than the pulp medium of books, tobacco-smoke-filled
library aging included, so argument exists that software rots worse than
Encyclopedias physically, at least.)

~~~
joe_the_user
It's a nice analogy but there are important differences here.

The thing is that the process of encyclopedias roting/going-out-of-date is
obvious because we understand that an encyclopedia is a collection of
assertions about the world. It is not as obvious that software is also a set
of assertions about the world - and it's not as easy to determine _which_
assertions about the world the software depends on.

Just as much, an encyclopedia is a static list of supposed facts and its
change is somewhat predictable by in terms of how our understanding of the
world changes. We have some idea what will be valid or invalid in an
encyclopedia X years old.

Software depends on obscure facts about OSes, about networks, about UI
expectations, about time, and so-forth and moreover, software involves chains
of dependencies so it's harder to predict what would or wouldn't break
software X years old.

~~~
WorldMaker
Thanks Joe. I think the difference you point out only strengthen analogy, and
which I tried to colorfully explore in the original post, but expand here
because it is interesting.

«It is not as obvious that software is also a set of assertions about the
world - and it's not as easy to determine which assertions about the world the
software depends on.»

I like that definition: "Software is a set of assertions about the world". It
may be less obvious than an Encyclopedia, but again, perhaps that only makes
it a better analogy.

«An encyclopedia is a static list of supposed facts and its change is somewhat
predictable by in terms of how our understanding of the world changes.»

Ah. This is where Wikipedia actually strengthens the analogy. An Encyclopedia
has always been loosely hyperlinked with cross-references. Entire complex webs
of assumptions and assertions about the state of the world. Refactors of
Encyclopedia content always could have interesting complex relationships with
other content.

Certainly there is an extent to which Encyclopedias are known to predictably
change: new elected officials each cycle, for example. But there is still
plenty of unpredictable changes: new red letter days in history, new
evolutionary scientific shifts in consensus, new lenses in which a culture
views itself.

An example of the lattermost, in that yellowed 70s Encyclopedia you could
still spot fragments in the process of being excised of what we now consider
to be racist views of culture and cultural history. That among other things
was a source of fascination to this teen in the 90s.

The often unspoken truth is that as much as Encyclopedias try very hard to be
objective, they are still a product of their culture. (So too do we often want
to think that Software is objective and nothing but mathematical assertions
about facts about our world, but here too they are products of our [corporate]
cultures and moored in ways both obvious and not obvious to the culture that
built them.)

Wikipedia shows us that complexity of Encyclopedias much more directly, but
those bones were always there in Encyclopedias even when we couldn't see them.
A link to click is much faster than shuffling between volumes of an
encyclopedia to follow cross-references, but the cross-references have always
been necessary to build up a view of the assertions about the world.

More interestingly, Wikipedia strives to give a better, more transparent view,
into the cultural biases of its contents; all of the [citation needed] tags
and "More Research Required" templates and Talk pages of endless discussion.
The novelty here is that they've opened up all of that to the common user, but
the essence of that has always been in Encyclopedias, just locked away by the
librarians and researchers in their own notes.

------
jussaskin
I find that integrated tests in software projects go a long way to reducing
rot. When something eventually invariably breaks due to some external factor,
the test suite greatly reduces the time to identify and fix the problem.

~~~
narrator
If you can build a piece of software and run unit and integration tests on it,
it is more or less rot proof. That's because all those tests lets one delete
or replace components of the system without having to worry too much that the
whole thing is going to break in some unknown way that won't be caught until
it's in production.

Without a good suite, the larger the system gets, the more people get scared
of doing radical things to it because they are worried about breaking some
critical functionality. As far as the not being able to build aspect goes, the
main threat here is relying to much on closed source software and/or libraries
that go out of support that no one can find the license, or critical
documentation for, or worse manifests some critical unfixable bug.

------
textmode
"... the arrogance and self-indulgence of youth."

s/youth/younger programmers/

How can they make work for themselves?

1\. Do what's already been done, not even knowing it's already been done. 2\.
Declare "rot" or some similar claim of obsolescence and proceed to redo what's
already been done.

There's nothing necessarily awful about this unless they fail to do a better
job than the earlier effort.

Alas, this is too often the case. For a variety of reasons.

In the early days, portability was a higher priority. Not to mention
longevity. Because everything was expensive.

Today's software "rots" a lot faster than the software from the early days of
computing, IMO.

And so the younger programmers have lots of "work" to do.

Yet I do not see much progress being made. Because I do not measure progress
by productivity alone.

Programmers who can churn out code in a dozen different languages to do the
same old things are a dime a dozen.

As a user, I do not want software that needs to be updated every week. Poorly
written software and gratuitous use of network bandwidth.

But I can see how programmers who love writing code would enjoy this state of
affairs.

~~~
simula67
Replace younger programmers with older programmers and people would scream
ageism.

------
notacoward
I think _rot_ is not quite the correct metaphor. In my experience it's more
likely to ossify, become sclerotic, build up scar tissue. As features are
added or performance is tweaked, individual pieces become more complex and the
connections between them multiply. If specific action (refactoring) isn't
taken to fight this tendency, later developers will react to one piece being
maintainable by making even more spurious connections and workarounds in
adjacent pieces. That fixes the immediate problem, but makes things worse
overall in the long term. Ultimately everything turns into the kind of tangled
mess that everyone who has worked on an old multi-person project can
recognize.

Unfortunately, a good refactoring requires understanding greater than the
original author's[1], and therein lies another whole essay. ;)

[1] Related to [http://www.linusakesson.net/programming/kernighans-
lever/](http://www.linusakesson.net/programming/kernighans-lever/)

------
ensiferum
Wheres the connection to software rot?

------
jaytaylor
"Software rot" stems from the world around the software changing in a way the
software is unable to adapt to, and breaking it.

~~~
bloaf
I feel like "rot" is the wrong term, and the right term might come from that
old parable about the man who built his house on the sand rather than the
rock.

------
Spooky23
Dependencies.

Big old hairy mainframe apps have no dependencies, and last decades. Anything
new expects constant updating.

------
dcre
Software rots because everything does.

------
hermanradtke
> Apache, the most important web server software today, is an old piece of
> technology whose name is a play on words (“a patched server”) indicating
> that it has been massively patched.

Is this true? I have never heard that Apache was a play on of words.

~~~
kazagistar
Apparently its a myth, and the original intention was based on the name of the
tribe. [http://www.linux-mag.com/id/472/](http://www.linux-mag.com/id/472/)

------
vonnik
There is an analogy to be drawn between software and societies, and they way
their early adaptations to one environment block their later adaptions to
another.

------
carsongross
What does rot even mean, in terms of software?

As far as I can tell, he means "Why is complex software hard to _change_ "
which is a reasonable, though fairly easy to answer, question.

Software doesn't rot. New features or refactors screw it up. New needs or
technologies may make it obsolete. But it doesn't rot, and people who talk
that way are often busy-body rewriters who want to pitch the existing
implementations, with all their Chesterton Fences[1], and begin anew.

[1] [http://www.chesterton.org/taking-a-fence-
down/](http://www.chesterton.org/taking-a-fence-down/)

~~~
wvenable
> Software doesn't rot.

On the face it, that's true. But the reality is that the environment that
software runs in is constantly evolving and software that doesn't also change
slowly starts working worse until it stops working entirely. We have plenty of
software at my work that hasn't changed fundamentally in 10 years and can no
longer be used.

Lots of really old software requires equally unchanged old environments and
old hardware and that hardware slowly rots too.

~~~
johannes1234321
... add to this that user expectations change. Applications not matching that
feel rotted. Just look at websites from 10 years back to get an idea only on
very surface levels.

------
tacos
Stewart Brand wrote a book called "How Buildings Learn" and in many ways it's
a better version of this post.

[https://en.wikipedia.org/wiki/Shearing_layers](https://en.wikipedia.org/wiki/Shearing_layers)

------
serge2k
> Newer programming languages are often interesting, but they are typically
> less flexible at first than older languages. Everything else being equal,
> older languages perform better and are faster

At the cost of more difficulty in writing, or other tradeoffs (e.g. security).

> Programmers, especially young programmers, often prefer to start from
> scratch. .. In part because it is much more fun to write code than to read
> code, while both are equally hard.

No way! You need to NPM literally everything.

