
How Software Gets Bloated: From Telephony to Bitcoin - bootload
http://hackingdistributed.com/2016/04/05/how-software-gets-bloated/
======
adekok
> software bloat almost always comes from smart, often the smartest, devs who
> are technically the most competent.

That's not my experience.

My experience is that software bloat comes from people who _think_ they know
what they're doing, but actually don't.

Competent programmers tend to be much more cynical about their own ability to
write good code. They've been burned too many times by bad code, including
their own.

The cases discussed here are not about competence or brains. They're about
upgrading systems, which maintaining compatibility with existing systems. That
is very, very, hard for anyone to do.

~~~
awinter-py
This is the classic 'curse of the gifted' flame war between ESR and linus

[https://lwn.net/2000/0824/a/esr-
sharing.php3](https://lwn.net/2000/0824/a/esr-sharing.php3)

Robert Frost and TS Eliot had a similar conversation. Frost wrote eliot saying
'the problem with you is that you speak 13 languages and know nothing about
real life'. Eliot responds 'the problem with you is that you're stupid'.

Age old debate, never fails to entertain.

~~~
AndyMcConachie
Thanks for reminding me that ESR used to be a reasonable person. I mean this
in all seriousness, I had forgotten.

~~~
masklinn
> Thanks for reminding me that ESR used to be a reasonable person.

[http://www.catb.org/esr/writings/dancing.html](http://www.catb.org/esr/writings/dancing.html)
predates that by 5 years, so "could be reasonable" might be a better
description than "used to be a reasonable person".

------
CM30
My experience is that it's often not the developers at all that cause a
program to get bloated, but constantly changing client/user expectations. Or
in other words, a program is developed and it starts out simple. Then, one of
the following happens:

1\. The user/client comes across edge cases, and has a very specific thing
they want done in that situation. Unfortunately, it's the complete opposite of
what the system would usually do, so said case has to be specifically patched
to act in a certain way despite making the codebase a confusing mess to figure
out. So one tiny part of a certain web page gets different padding or line
height because the client/user thought it looked better in that one situation,
and no one thought to say it was a bad idea.

2\. The project's spec was wrong and hence it was the wrong thing to be
building in the first place, but no one liked the idea of starting from
scratch. So you end up with a program that's meant to do one thing but has
been crudely hacked into doing something completely different and hence half
the features are utterly useless.

So it's not the developers fault in these cases, its management not knowing
what the team is actually meant to be building or spending a lot of time
trying to provide exceedingly random and useless 'nice to have' touches.

~~~
drawkbox
Completely agree with this assessment.

The way software is completed is pretty crazy today, little time for research
and development and a lack of focus on quality, but focused on short term
changes that cost long term technical debt. Estimation is hard and project
plans are only ever about 60-70% right.

There is a big problem with not allowing sliding dates as well. When software
is completed one day early that is a pile of crap it is lauded short term and
painful longer term submerged in technical debt, then when more time is taken
up front for quality it is short term pain for long term gain later on the
maintenance phase. The problem is the latter is hard to do with development
schedules not set by developers.

Note: I accidentally hit downvote as my mouse glitched and wish I could reset
that.

~~~
collyw
Agile

------
PaulHoule
I wouldn't just blame devs, I'd blame business, as much in terms of the
dynamics as well as the individuals.

From a marketing perspective it is easy to see that "Adding Feature A will
help us sell to customer B." It is not clear (and probably not true) that
feature A will cause us to lose a sale to customer C, but the cumulative
effect of a huge number of features can lead to "bloat" and "unreliability"
that can cost sales to customers Y and Z and drive the cost and risk of
development so high that it stops.

I think of Adobe PDF as an example. People criticize it as being "closed",
which is not true, it is one of the few ISO standards which you can download
for free and Adobe even scrupulously documents how their products deviate from
the standard. However, it is insane how much stuff has been added to it beyond
a document format (i.e. exchange of 3D models) such that when I am trying to
control the size of the C: of machines I control, Acrobat is the Creative
Cloud product that is the most universally useful AND one of the absolutely
largest thus making it the hardest decision.

The arguments against it being "closed" and "bloated" then crowd out the real
criticism you'd have if you actually read the ISO standard, such as the fact
that you can't draw a circle in PDF. (I don't know what happens if you 3d
print a wheel made out of 4 bezier curves that look like a circle, but I am
sure some of you will find out.)

~~~
teddyh
> _when I am trying to control the size of the C: of machines I control,
> Acrobat is the Creative Cloud product that is the most universally useful
> AND one of the absolutely largest thus making it the hardest decision._

Alternatives here: [https://pdfreaders.org/](https://pdfreaders.org/)

------
drauh
"Every program attempts to expand until it can read mail. Those programs which
cannot so expand are replaced by ones which can." -jwz

Thanks to Wikipedia, I found that this may have been an earlier one-liner by
Greg Kuperberg:

Every program in development at MIT expands until it can read mail.

[http://groups.google.com/group/rec.humor.funny/browse_thread...](http://groups.google.com/group/rec.humor.funny/browse_thread/thread/2195f8fdf5402dbd/786a67b37348bc23)

------
titzer
A lot of others have mentioned clear reasons that bloat happens in the normal
trajectory of software lifetime: increase in requirements, bugfixes, etc.

But sometimes constant employment on a project just creates bloat, too.
Refactorings that are intended from the beginning to reduce complexity either
don't end up delivering, or they are never completed, leaving systems in a
half-migrated state. Most systems I am familiar with are in a perpetual state
of transition between the old way and the new way. The more complex the
system, the harder a migration is. The harder a migration is, the more likely
it is not to be fully done. The more incomplete migrations, the more
complexity. On and on.

It's why eventually we should keep trying to reboot software systems with
from-scratch, feature-lean replacements.

~~~
sievebrain
People are very bad at deciding they need to be made redundant. Hence the
phenomenon that "software is never done". It actually can be done, but you
won't get the developers to agree to that!

------
henrik_w
Software that becomes "bloated" is software that is successful and in use.
Typically, the users want more and more functionality, because they use the
system for more and more. This is not bad.

Software that is not used (for what ever reason) is abandoned, and therefore
does not grow.

So there is selection bias too - the most notable software is the successful
cases, and they become "bloated" because they are successful.

(what the author talks about is not bloat, it's clever hacks that also make it
much harder to understand the program)

~~~
WalterGR
"Bloated" is a just a pejorative term that means nothing more interesting than
"Contains feature(s) that I don't personally use."

~~~
Retra
.. which means it is slower than it needs to be and has a more cluttered
interface than it needs to have for that user.

If those problems don't interest you, then it's hard to imagine what your idea
of good software is.

~~~
clockwerx
Does it really though? Those are trivial problems to solve: dont show users
tools they dont use via roles/permissions; dont execute code that is unrelated
to the current purpose.

~~~
Retra
Roles/permission code _is_ unrelated to the current purpose.

Really though, it's more of a systemic issue. There're so many ways you can
make decisions to make your software make decisions to conveniently pretend
you're giving the user exactly what they want without having asked them what
they want. Maybe they don't want to download and install code they don't plan
or know how to run. Bloat is problem for users because it has machine costs,
which users have to pay for in time, money, and attention.

------
bobwaycott
I doubt I'm the only one who spots that "bloat" is highly subjective, thus
defining and categorizing bloat depends on who is doing it.

As a _developer_ , I frequently categorize and define bloat as large amounts
of extraneous code from libraries and other tools that are not directly used
by any part of a given application, but are nonetheless imported or installed
by a project. Here, I typically go pointing the finger at requirements files,
sloppy imports, and other dependency-related smells.

As a _manager_ , I may define bloat as anything in the above dependency realm
when I'm working with developers. _Alternatively_ , anything in the realm of
features and functionality that do not have a defensible position as being
built for and used by a majority of users is sure to get some dirty and
suspicious looks from me as bloat. This is typically triggered when a client
or other project stakeholder suggests Feature X be added so Person Y (or
Customer Z) doesn't have to use some other Tool B. I push back hard on such
suggestions until there's a demonstrative case to be made that this kind of
bloat won't become a major liability in the code from a long view, or a
stumbling block for the majority of users.

As a _user_ , I often define bloat as any number of features I don't
personally use and, through various interactions and use-cases, feel get in my
way, clutter up my UI, whatever.

Everyone involved (except users) is responsible for the bloat in their own
domain.

~~~
collyw
Sure it is subjective, but there are certain places where it is obvious (like
the stuff i am refactoring just now). We have 5 different types of fees -
course fees, special fees, add-on fees, payment fees, extra fees (maybe I
missed some). These are split between various database tables, and classes in
the application, as a result there is way more code than is necessary. I am
going to try and put them all in one table, with a "type", and reduce the code
at the same time.

------
binarymax
Tautology time: If software was ever 'done' it wouldn't grow and become
bloated.

Bloat comes from the combination of endless feature requests combined with the
need for quick development cycles to implement those features.

Developer competence only slows the process.

------
Udo
Bloat happens when programmers don't care about (or don't understand) what
code actually does and what the tradeoffs down the line are going to be. Bloat
happens when code is produced cheaply and quickly. Bloat happens when there is
a cultural stereotype that having produced a bloated piece of code means
you're smart.

Bloat is not the same as complexity, but they are related. Bloat is not always
_bad_ , or even perceivable as such by the consumer of the code, but at the
same time it's also never _good_. Bloat is a natural accumulation tradeoff
decisions.

Anybody who writes code produces bloat, because writing software is an
exploratory activity towards solving a problem set. There is no way to always
know where the bottlenecks are going to be, and where your chosen abstractions
will break down. The difference between doing good work and bad is that the
programmer doing good work will revisit decisions and refactor things
ruthlessly down the line. The programmer doing bad work simply doesn't care,
is too pressed for time, or writes code in a way that makes refactoring
prohibitive.

Personally, I think a good litmus test for bloat is how much the code you're
writing actually achieves for the effort you put into it. If you find yourself
constantly massaging and circumventing the limitations of your boundary
layers, that's a big warning sign. If you are using a library or framework
that prolongs and complicates the effort of solving your problem compared to
the expected effort of doing it from scratch, then that library is clearly not
useful. Using it regardlessly is a deficiency in the critical thinking
required to make good decisions in programming.

------
sickbeard
The word 'bloated' is subjective. It's 'bloated' because it was designed for
something more but ended up being much less.

In other cases we also end up spaghetti code software full of hacks and bad
designs because it wasn't well thought out or designed from the start.

In the end designing software that matches expectations is the biggest culprit
of 'bloatware' vs 'spaghetti monster' codebases.

------
giveaway
I couldnt agree more on what is said by adekok and PaulHoule . Working on a
project right now and the product was lean and WORKED ! now thanks to un
necessary trackers and unwanted features it has bloated like anything .

------
alwaysdownvoted
"... software bloat almost always comes from smart, often the smartest, devs
who are technically the most competent."

I am quite happy with software written by people who who do not view
themselves as "devs" who are "smart", "the smartest", or the "most technically
competent".

I'm also happy with software written by telephone company employees in the
1970's, a grad student in the 1980's and a university maths teacher in the
1990's. Software written by "security researchers" is usually not bad either.

I like the small programs that win the IOCC; if I were asked to name a test
for "competence" (my definition), the IOCC would be high on my list.

I have no reason to question that the "devs" writing the bloat, e.g., at
Microsoft in the 2000's or any number of companies today, are _brilliant_.

However I have little interest in that class of software. I think bloat is
stupid. Not to mention unnecessary for my purposes. If I cannot pick the code
apart and recompile it myself, then the software is nothing but a liability to
me, not an asset.

------
brashrat
software bloat occurs exactly the way this discussion bloated.

read thru it... think we could rewrite it smaller and capture all the wisdom
and make it more orderly? would you care to join the team and start doing
that? :)

------
kinsho
My experience in code bloat seems to tell me that it comes primarily from good
intentions that fail to pan out as intended.

The road to hell is paved with good intentions....

------
awqrre
Most popular commercial programs get bloated or out of touch over the years...
I'm just a user but I was thinking that it is this way because developers need
something to do and don't like to waste time doing nothing.

------
kazinator
One reason why bloat happens (not the only one) is that when we increase the
number of requirements from N to 2N, the program size S does not simply go to
2S.

Sometimes when we add a requirement, numerous complications cause new lines of
code to be added throughout the program. So the requirement doesn't have a
fixed cost in terms of additional size; the bigger the program _already_ is,
the more lines are added by the new requirement.

Someone wants rational numbers in the numeric tower. Okay, so we have to
implement addition of a rational to every existing type, including itself.
Multiplication of a rational by every existing type as well as itself.
Subtraction, division, ad nauseum. The cost in terms of code size for the
requirement to have rationals depends on how many numeric types we already
have.

Development by accretion of third-party code is the major cause of bloat.
Third party components are already getting bloated over time due to the global
bloat effect. And here you are, just mashing these components together to make
your program. You don't use anywhere near the full functionality of these
components. You link that entire five megabyte shared library because it has
three useful functions. The seventy-seven others you don't use have to stay
there because it's a shared library; some other program might attach to it and
use a different subset.

These components get upgraded over time. Security fixes, enhancements, loss of
support for old versions. Several years later, you're _still_ just using only
three useful functions in that shared library, and none of them have changed.
But it's no longer five megabytes; it's now ten and has 157 other functions
that you're not using, not 77. You hadn't even done any work to get that five
more megabytes of bloat; you just triggered some automatic update a bunch of
times over several years and ran some regression tests that passed.

Even the same code gets bigger when compiled. Thirty years ago, the functions
were packed together, as well as their internal basic blocks: there was no
advantage to any special alignment for branch targets. Loops were not unrolled
and code wasn't inlined much: caches weren't big enough so it was often
counterproductive. C++ templates came into the picture since then: let's
generate umpteen variants of the identical code into slightly different
machine language dependent on type.

Managed languages: these drag entire platforms with them. And the internals
are "not user serviceable": you generally don't rebuild these things to remove
what you don't want. You accept the platform with all its bells and whistles
and work your app into that. Their bloat is nothing new. In the 1980's, some
Lisp implementations were criticized for making 20 megabyte images (at a time
when that really hurt). The rest of the world caught up to that: we now have
non-Lisp languages that have lexical closures, garbage collection and ... huge
images, far in excess of 20 megabytes. Today's managed languages, being
platforms, have naturally copied the concept of package management from actual
operating system platforms. It's super easy to bloat up the image just
essentially by adding simple declarations of what you want in it, chosen from
a large online catalog.

~~~
hinkley
A good design will often increase in size and complexity as a factor of
nlog(n) of the number of features. A mediocre design by n^2. And a bad one
will increase by n!

The problem I've been wrestling with a lot lately is that developers with a
good design don't see the problem in accepting all of these change requests.
All these features are easy for them to add because Architecture!

What they discount is that architecture doesn't get you out of testing the
interactions. So that nlog(n) development effort often turns into n^2 testing
overhead, and occasionally worse.

------
xufi
Sometimes at least recently. I feel that sometimes software for example the
default apps on a phone are doing more harm than good by taking up much needed
space.

------
cloudjacker
I usually quit when the roadmap starts to include bloatware

------
toomim
I totally agree.

