
Towards an understanding of technical debt (2016) - kiyanwang
https://kellanem.com/notes/towards-an-understanding-of-technical-debt
======
justinjlynn
Technical debt isn't debt. Debt has interest, and you can pay it back. You can
use debt to grow. In a sense, debt is a good thing - you can timeshift
revenue. Businesses are comfortable with debt.

But that's not what we're doing when we create technical "debt".

What we call technical debt is really best analogous to an uncovered short
sale. We sold something we didn't own (i.e. technical sophistication and
proper design) to get something of value (a shipping product) in the hopes
we'd never have to cover the asset (fix the bug, upgrade the dependency, etc.
- i.e. cash out before the code hits the fan).

If you win the bet and you never have to cover the position - it's progress
for nothing! Awesome! But, if you do have to cover it - disaster! You're
forced to pay any price to cover the short or abandon the feature - maybe the
whole project. This can happen at any time. Insanity.

In finance we generally don't permit uncovered short positions for a reason -
they destabilise markets, and quickly. Transitive dependencies, i.e. systemic
risks, abound.

Companies have to start talking about software projects and practices in an
actuarial sense - and get insurance to cover their naked shorts and wild
technical bets.

~~~
AlexTWithBeard
But according to your description, technical debt _is a_ debt:

\- it has interest: you're paying extra in maintenance, debugging and manual
testing for all the shortcuts you've taken

\- it can be paid back by refactoring the bad module. The price of such a
refactoring is usually known and, unlike in short sale, won't get to infinity

\- it can be used to grow. Instead of working five years on a perfect solution
we build something plausible right now and then keep refining it

~~~
Vinnl
And that's why debt is a good metaphor: you have to have a plan to repay it,
and that interest you pay should be less than the profit you make by taking on
the debt.

Take, for example, something like Mozilla's Facebook Container extension [1].
It's fine if they take some shortcuts that allows them to launch that faster
if it turns out there's another Facebook scandal in the news - launching
faster allows riding the waves of publicity, resulting in more users. Even
though the total development effort is more when they have to clean up those
shortcuts after launch, the additional effort has been used to grow.

[1] [https://addons.mozilla.org/en-GB/firefox/addon/facebook-
cont...](https://addons.mozilla.org/en-GB/firefox/addon/facebook-container/)

~~~
mbostleman
Debt is also a good metaphor because, just like finance, it can be used as a
beneficial lever that propels you to the next level artificially fast, or it
can be used foolishly and end up burying your effort.

I see examples of the latter all the time in the experience of developers and
how they approach decisions. I have recently worked on largish (or at least in
the 10s of millions of dollars of investment to create) codebases in which
developers (recently - like 2013) wrote switch statements that became God
classes because they simply were not familiar (at least not in practice) with
patterns that avoided it. Consider that one example and imagine the type of
havoc that lack of experience creates across the entire codebase. You can say
this is a case of ignorance, which it is, but it's the same as the ignorance
of some to handle money in a sustainable way. It's like giving a credit card
to a kid.

------
_bxg1
Highlights:

> When we program we’re writing down a very explicit and exact description of
> a solution to our current best understanding of the problem we’re trying to
> solve. Over time both the problem we are trying to solve and our
> understanding of it will inevitably change, often dramatically. When that
> happens some percentage of our very explicit and exact description is now
> wrong, and a liability. Taken to the logical conclusion, given time, every
> line of code is wrong and a liability that will need to be dealt with.

> A little discussed and poorly understood design goal for code is
> disposability.

Criticisms:

\- I'm not sure I agree that these five buckets are very distinct, nor that
they encompass all things referred to as "technical debt". They're good
examples worth thinking about, but I don't think they're as archetypical as
the author does.

\- I've never seen it as an assumption that "technical debt" implies "the
people before us did a bad job", something the author spends a lot of effort
arguing against. The people before you lay out a runway, which helps determine
how quickly technical debt will build up, but even the very best codebases
accumulate it eventually.

~~~
halbritt
I was struggling with that last but as well. Sure, in some cases the people
before did a bad job, but in my experience that occurs way less often than
engineers simply developing for the requirements that they understand, which
evolve over time, that is, both the requirements and the understanding.

It is possible to create very extensible software architectures, I’ve met many
engineers that will spend a great deal of time anticipating future needs, but
in many cases this constitutes premature optimization.

~~~
_bxg1
Something I've only recently started to grasp is that good code isn't about
taking abstraction and extensibility to the very maximum, it's about picking
the right _amount_ of abstraction to fit the amount of change the code will
likely face in its lifetime. Abstraction isn't without its own cost.
Unfortunately, all of these design decisions - even the meta-decisions like
degree of abstraction - are relative to the project and many of them have to
be made at the outset, where you're only guessing what the future will bring.

------
scarface74
From the article.

 _You can address issues that are damaging to productivity, operability and
morale, but only way to “fix technical debt” is “rm -rf”._

I think he was saying this ironically, but this is a very real possibility
sometimes. There were often times I’ve found (and wrote) code dealing with
cross cutting concerns that weren’t core to our business and I ripped it out
and used a popular third party (open source) package.

On a higher level, I champion using managed services whenever possible over
doing “undifferentiated heavy lifting”.

~~~
BerislavLopac
Deleted code is debugged code.

~~~
vaylian
Mostly true. But removing something can also give you something in return. In
this case a simpler and potentially more reliable codebase.

You should throw out old code when any of these apply 1) The code becomes
irrelevant 2) The code becomes redundant (you found a way to generalize
things) 3) The code is so bad that maintaining it is simply not worth it any
more.

These things happen frequently enough that removing code is usually a
reasonable action.

------
pronouncedjerry
As a previous engineer and now turned product manager. I like to frame these
technical investment work items as a pair [Item 1] + [Item 2]. This has
drastically improved the understanding and commitment from the rest of the
business to fund and allow for "non-feature" work.

Item 1: Security, Performance, Stability

Item 2: Fix, Update, Upgrade, Improvement

Examples: "Security Fix", "Performance Improvement", "Stability Update", etc.

~~~
bjelkeman-again
What is the difference between Update and Upgrade?

~~~
BerislavLopac
I have usually used "apt-get" as a good guideline: "apt-get update" updates
the definitions of available packages, while "apt-get upgrade" retrieves and
installs the actual packages. In a nutshell, "update" doesn't change the
operation of the system, while "upgrade" probably does. In SemVer parlance, an
upgrade is the major version shift.

~~~
gizmo686
Apt-get upgrade should be the equivelent of a minor version shift. A major
version shift would be do-release-upgrade.

Apt-get update doesn't update the system at all, it updates apt.

~~~
BerislavLopac
Fair point, I stand corrected -- apt's update/upgrade does not directly map to
SemVer version numbers.

------
ilovecaching
I wanted to get angry at this article, but he made some good points. Still,
I’ve found some organizations are more concerned with technical excellence,
and others see it as mastrabatory acts that get in the way of being agile to
outlive competitors and putting resources only where it will obviously help
the business. Obviously a lot of us like to build things that are correct,
efficient, safe, and easy to maintain and not just “meets minimum requirements
to keep the ship afloat” after so many years in school learning to become good
engineers.

It also seems like markets and disasters can seem random from a SWE point of
view. You can build a great system and people just might not take to it, or
the whole thing could topple over because of a third party component, a really
determined hacker, etc.

~~~
menko99
There is a selfish balancing act as well. I've been in two jobs where I'd take
on more and more technical debt to deliver new features quickly so we could
capture marketshare. Eventually, the technical debt killed agility and project
sponsors didnt understand why every little thing took three days to do instead
of one.

This left me open to being ousted (e.g., the Product Manager "had a friend"
who could surely do everything much faster.) Seems easy from the outside. They
got rid of me, hired the friend, who both increased the team size and then re-
built from scratch.

Selfishly, it would have been better to more steadily deliver features and
balance speed with maintainability -- because sometimes higherups dont
understand the tradeoff they are making.

~~~
ilovecaching
Thanks for sharing your story. Did they factor the cost of a rewrite into
their calculus? I know sometimes even though a system becomes hard to move
forward it would require such an investment to reach feature parity it has a
sort of protection against total destruction.

------
jmkd
While I admire the author's attempt to bring structural clarity to a broad
concept, greater clarity can perhaps be found by broadening it further still.

Anyone who has refurbished a house will live and breathe examples of technical
debt embedded in the workmanship, and the material / design choices around
them.

As you discover past shortcuts made by hurried builders, asbestos components
or electrical cabling no longer safe to use, you quickly realise that you have
to address these issues in a way that doesn't come back to bite you down the
line.

As a regular homeowner you are transformed into the perfect dogfooder for
House 2.0, and this perspective should inform your (day job) approach to
technical debt as it is traditionally defined.

~~~
devonkim
I read of a clock tower several hundred years old that was in need of repair /
replacement and construction workers realized to match the wood the trees
they’d need were thousands of miles away and extremely rare. When they started
asking around to see if they could source it with substitutes the
groundskeeper said that the tree they needed seemed familiar and brought them
to an orchard a few hundred yards away. For generations, a small orchard was
maintained from the time the project was started for the day when the timber
would need to be replaced.

We don’t do this normally in most homes around the world, but families of
multi-generational wealth seem to be able to do something similar with their
homes because they both have means and will to build an estate that lasts
centuries.

------
zzzcpan
Essentially a technical debt is an unvalidated assumption that other
unvalidated assumptions are worth pursuing. A hunch about hunches of sorts.
The article is full of them describing every way to think about technical
debt. And it's practically impossible to neither prove nor disprove the claims
of technical debt without a lot of work, making it super easy to justify
anything with technical debt. Naturally it becomes a term of authority and
power to sneak in things unlikely to produce anything of value.

Could be a sign that things got too toxic in software development if
developers can't even talk openly about experimentation and research.

~~~
sonnyblarney
Technical debt concerns are not necessarily 'unvalidated' \- at least not to
the extent that almost everything is a little fuzzy in tech anyhow.

For example: you started a few feature with a few scripts, added a few new
things here and there ad-hoc. Now it's growing and full blown and nary
impossible to maintain. A more clear understanding of the problem space now
exists, and also that a refactor would make sense.

The continued inability to really move forward due to the rag-tag system is a
kind of debt, and the trade off/decision has to be made as to wether or not
the team needs to take a few steps back before moving forward.

This is really common.

I worked at BlackBerry during their heyday. If you are old enough to remember,
there used to be international headlines when the BB email service went down.
Everyone in the press was using BB and would harp on it. This is partly
because BlackBerry had made a bunch of acquisitions and gun-taped the systems
together, as opposed to properly integrating them. There was a real 'cost' to
this bad press. The only real and fundamental way to avoid those systematic
failures was to do a proper integration which in and of itself was a risky and
unwieldy proposition for the company at the time.

I'd argue that was a fairly material and validated form of technical debt.

------
micksabox
We need more definitions. I think it's helpful to distinguish between
technical debt patients and technical debt doctors. As a developer (patient),
you experience the _symptoms_ of technical debt - slow changes, dev
frustration, etc. As a scientist (doctor) you observe this as a naturally
occurring phenomena - with a hypotheses, measurement system and literature
review. A TD doctor might be your local architect, staff or principal
engineer.

There's quite a bit of published literature on technical debt, which appeared
quite recently (~5 or 6 years ago). One distinction is between normal
technical debt (code level), and architecture technical debt a.k.a ATD (system
level) [1]. So far, whether you like the term or not as a patient, it seems
useful for scientists to think collectively about the phenomena in these
terms.

Some commenters in previous thread say an ideal (reference frame) doesn't
exist, from which to calculate the delta of TD currently. The symptoms do
exist. That extra few hours it takes to onboard a new developer, fix a bug or
whatever are observable. If there exists a possibility to reduce that
observable, measurable effort, then that possible future timeline should be
the reference frame.

I've risked an appeal to authority in order to transfer better understanding
as a random stranger on the Internet. Technical debt seems a topic too fuzzy
to fully understand, and that's why I'm pointing out the way scientists are
thinking about it. I don't see any other clear path we're going to get
industry wide consensus or understanding, if not for the scientific method and
institutional support.

That being said, the article did a good job of communicating at least 5
different ways of thinking about TD.

Disclaimer: I'm building an architectural technical debt research tool called
RefactorKit.com

1.[https://www.researchgate.net/profile/Terese_Besker/publicati...](https://www.researchgate.net/profile/Terese_Besker/publication/320081646_Managing_Architectural_Technical_Debt_A_unified_model_and_systematic_literature_review/links/59d739150f7e9b42a6b068c0/Managing-
Architectural-Technical-Debt-A-unified-model-and-systematic-literature-
review.pdf)

------
beiller
I usually just see the term technical debt as a way to pedal the latest shiny
programming environment. Coming from someone who works with COBOL, Oracle
PL/SQL and very old java code (I also work with 3.x python and Rust) the
ability to fix code is easier than ever given the amazing tools we have today
like modern debuggers and decompilers. Its no doubt hard, as we see, for
programmers to write say an emulator for Nintendo, but it is definitely done
today and they have no code usually to operate on at all. Also look at game
modders who are injecting DLLs. There is entire subgroups of programmers whose
day to day is just modifying the worst possible outcome in terms of technical
debt (already compiled/optimized/obfuscated code). So anyone who mentions
technical debt as an up front refusal to use some technology x to solve a
problem, is really just doing premature optimization in a project and I think
that is a real problem. To put it another way, any attempt to understand
technical debt is to speculate on the future of Comp Sci and/or companies'
licensing changes / mergers and is basically a complete gamble.

~~~
anticodon
I think it's not that simple. Latest tools often increase productivity. Gamers
can afford to spend days and weeks to add some feature to their favorite game.
They don't have deadlines and they don't care about QA. It's another thing
when you have to fix a bug or add a feature in a limited time with limited
resources because business needs it.

~~~
beiller
I try to frame it a bit more with regards to project management, where I feel
any discussion of tech debt is bad for the project. To even start talking
about it you have to start making many assumptions, for starters the
assumption that the project will be successful to then later have to maintain.
Then there are many many other assumptions to follow, and I feel that it is
generally detrimental for it to even be discussed. It always involves some
upfront price/scope/time on a project in trade for some future unknown cost
savings. Fair point about the game hacking though they don't always have such
constraints, but they do help close the gap by inventing these great new
hacking tools!

------
phutchins
This article describes technical debt in code which isn’t the only kind of
technical debt. It is potentially much clearer from my perspective to
understand technical debt in architecture and infrastructure.

The amount of technical debt can be described here by identifying the quickest
shortcut as well as the most correct solution, choosing the solution between
those two on the spectrum that makes the most sense for current time and
staff, and taking the difference between where you landed and the most correct
solution. This difference is technical debt.

Taking on the debt does most times get you further faster, but in turn it
slows you down in the long run. The longer you wait to pay it off, the more
time in the interim you spend making minimum payments that get you nowhere but
consume time.

------
lukecameron
Previous discussion on this post:
[https://news.ycombinator.com/item?id=10887452](https://news.ycombinator.com/item?id=10887452)
(2016)

------
sonnyblarney
I would add risk, essentially security.

Some companies have gone light on this because the liability/cost of failure
was not so much.

But at some point, some of those risks become real.

The costs aren't always apparent, because that's the nature of 'black swans'
\- but the systematic risk is.

------
LaserToy
I will go even further - all code is bad and probably broken. Why bad -
because if you put 10 engineers in the room and ask them to look at it, they
will argue about personal preferences and not about what code actually does.
And you can always improve what you wrote, which means current state is far
from good.

STechnical debt is not a universal thing. You can have a debt free code, then
new hire will argue that whole thing is going to collapse and needs to be
rewritten. Even when rewriting it brings 0 ROI and it actually good as it is.
I’ve seen fair share of engineering initiatives that were complete waste of
time, caused by attempts to fight “tech debt”.

The only good use of technical debt is to manipulate business folks into
approving projects you want to work on. And I’m guilty of using it that way.

