

Ask HN: Do you have real-world experience of technical debt killing a project? - hugorodgerbrown

Explaining the impact of technical debt to someone non-technical who hasn&#x27;t experienced it first hand is almost impossible (e.g. &quot;why didn&#x27;t you just do it properly the first time?&quot;).<p>It would be great to get a thread here detailing real-world examples of debt killing a project, or more, that we can all point people to when they claim it&#x27;s just developers complaining.<p>Sample experiences worth sharing include:<p>* Have you been part of a project that ground to a halt because of technical debt?<p>* Have you ever had to abandon a project&#x2F;product and &quot;start again from scratch&quot; because of problems in the previous codebase.<p>* Have you ever left a job because the overhead of servicing technical debt made the job impossible?<p>* Have you ever worked for a company that went under because debt was holding them back?<p>I know it may not be possible, but real company names would definitely add value.
======
jdubya
I worked for one of the largest load balancer/application delivery controller
company for a grinder of a boss on a security team there.

My job was to hack shit, find vulnerabilities and write automation for attacks
that our systems could prevent. There were 4 pretty big code bases (>100k
lines) that were used for the same thing. There was no inter-team effort,
everyone wanted to roll their own rather than working together because the
bosses had huge egos and wanted their name in the brick.

I was tasked with building a new framework!

So yeah, five different automation frameworks, one for automating attacks
against devices and servers, one for smoke testing (pound the box with
traffic) etc. Mind you all of this shit could be done with a single framework
because many of the tasks were similar. So I created a way to build test
profiles from our test plans and consolidated the code bases. I learned a
shitload about dealing with existing code during this effort. It was awesome
in that respect, I learned how to deal with poor management decisions -
execute better than they ever did.

Long story short I refused to scrap everyone elses hard work even though my
boss told me to "build it from scratch and use it for the security team only".
The reason being is because starting from scratch would have incurred more
technical debt than the consolidation effort took. Mind you every other
department in eng could have used said framework. Anyhow, under the radar I
went about consolidating the other four frameworks and I gave my boss the
timeline without his knowledge of what I was doing.

Three months later every eng department was using this new framework using the
tools that they were used to using for deployment and verification, just a new
ui and a lot of code that was cleaned up and everyone was pretty happy with it
to be honest. Everyone that actually used it that is.

My boss, who never used any of the frameworks, got pissed off that I did not
build a sec. team only framework and asked me to build a version only for the
security team for "security reasons". This was the proverbial straw that broke
the camels back. This fucking moron wanted to satiate his own ego at the
expense of my peers time, health and the company. I don't really care too much
for the company but everyone I worked with was very competent and I did not
want to just throw the baby out with the bathwater. That would be stupid.

Long story short, I was interviewing at places at the time and got an offer
that very day so I decided to take the offer.

Moral of the story:

Don't have your engineers build 4 different automation frameworks and then ask
an engineer to "rebuild" a custom version for a bullshit reason. Don't build
shit from scratch unless you are learning/experimenting and at large
companies, they rarely reward experimentation.

I get that this might not exactly be technical debt but 4 frameworks and the
effort I had to put in to the consolidation was brutal at times. Improperly
used technology, poorly deployed interface supporting services and solutions
for transient failures of tests being "just reboot the test box" was too much
bullshit for me.

God I fucking hated that job. I will never work for a large, well established
company again if I can avoid it. My startup might be getting picked up soon by
a conglomerate and I have 6 months left on my vest. If we end up at the
conglomerate I am going to quit and take a few years off working.

That is my experience.

------
wikwocket
The best way to explain technical debt is by analogy: Technical debt is what
happens to your entertainment center when you buy a new Playstation. To do
things properly, you should buy another HDMI cable, run the cable through your
cabinet around to the TV, on a dedicated input.

But that's not how it happens. When you bought the Playstation you also got a
copy of Call of Battlfield Honor Brothers, and your friends aren't going to
wait half an hour while you do that. So you steal the AV cords from the Wii,
run them in front of the DVD player, and plug them into the XBox's TV port.

So now, the system still works, but it's not ideal. You have to move some
cords if you want to play a DVD. You have to unplug the Playstation if you
want to play Minecraft on XBox. And heaven help you if you want to play Wii
Bowling.

Now imagine this happens many times over several years. You get a new XBox.
And a Bluray player. And a new sound system. Each time you do what it takes to
get things working, and no more. Pretty soon, just to watch Netflix you have
to rewire a whole switchboard of cables.

At some point, you need to pay off your technical debt. In this case, by
taking a whole Saturday to unplug everything, buy some new cables, and re-
route all the connections in a clean, logical way. It's the last thing you
want to do while your friends are playing Band of Duty-bound Heroes IV, but if
you don't, you'll be rewiring Ethernet cables in order to watch Hulu until the
end of time.

~~~
hugorodgerbrown
Thanks for the analogy, but this is missing the point - this thread is about
the opposite - real, verifiable, experience.

~~~
wikwocket
No worries. When you said "Explaining the impact of technical debt ... is
almost impossible," I thought you (or others) might be interested in a good
way to explain it. This analogy has helped me, so I hope others may find it
useful.

------
G_rupture
I haven't heard of any company going down because of technical debt, but here
is a study that has some quantitative analysis of technical debt in the
beginning: [https://medium.com/building-
bowery/e5a006f7b724](https://medium.com/building-bowery/e5a006f7b724)

~~~
hugorodgerbrown
Thanks - that's an interesting article, although the key quote from the
original paper doesn't really help make the case:

> The only thing that can be said with any certainty, is that not doing any
> technical debt maintenance does impact them negatively. But there isn’t a
> lot of difference from that point on. This is slightly controversial, and
> could be unrelated altogether.

Judging by the lack of activity on this thread, and the above article, I think
I could effectively argue that technical debt either doesn't exist (outside
the minds of developers), or that it's 'a thing', but just not that important.

This isn't what I was hoping to see...

