Hacker News new | comments | show | ask | jobs | submit login
How to Leverage Technical Debt (medium.com)
102 points by mortimerwax on Jan 18, 2016 | hide | past | web | favorite | 28 comments

The biggest thing I think people need to know about technical debt is that it must be used to create growth. Consider the situation where you go to the bank and ask for a loan. You tell the loan manager that you will spend the money on pizza parties. Unless you are in the pizza business, you are unlikely to secure the loan. Loan managers want to know how you are going to use the money you gain to generate growth and ultimately pay the money back (with interest).

Now consider technical debt. When you go to your architect and say, "I want you to hack something so that we can get it out the door a month earlier", your architect should respond with, "What are you going to do with that time I give you?" If the answer is, "I just think we need to get to market as soon as possible", then you are doing the technical debt equivalent of spending borrowed money on pizza parties.

As a reasonable architect, I need assurances that I'm going to be paid interest on the loan. If I give you a month, I'm going to need a month and a half back. You need to explain to me how you are going to make that kind of time. If you can not, then the business plan needs rethinking. No respectable banker would say, "Oh, there is a 0.1% chance of success if I lend them the money, but a 0% chance of success if I don't. I'd better lend them the money!" He says, "Come back when you have a better business plan".

The other annoying issue is that complexity in software can grow exponentially. So the cost of repaying the debt can quickly outstrip your ability to pay. Imagine borrowing money and having an interest rate of 1% for the first month, doubling every month. Seems like a wonderful idea to borrow the money, but you better pay it back fast.

The problem with technical debt is not that you must avoid it at all cost, it's that you must think very hard about whether or not you can afford it. If you can't, you should avoid it. If that puts your company out of business, then you need a better business plan.

I don't disagree with the gist of your comment but time to market is generally massively undervalued (and particularly by technical folk). It's more often than not a great reason for taking on tech debt (not to mention that early releases also constrain scope to smaller chunks).

I totally agree that time to market is hugely important.

The dynamics of getting to market are complicated and I don't agree that tech folk undermine them to any special degree. If you're an engineer with a few years experience, chances are you've been smacked by being too conservative or too loose with tech debt. Good contextual use of debt is a key part of the "senior" in any senior engineer. I'm happy to be "sloppy" and fast at the right moments.

My experience has been that business people value time to market, but only if they can offload the costs to engineering by taking on tech debt. When asked to put their money where their mouth is and pay for more engineers or cut scope, often the business side will balk. It's an annoyance when non-technical leadership perennially sees tech debt as a way to get a free lunch. Ain't no free lunches.

Interesting to place the technical debt creditor/borrower relationship that way round: often it seems that it is seen as being the engineering team who are the debtors, and the debt is owed to some sort of abstract entity - possibly the codebase itself. The developers wind up going to the business and begging for the time to pay down some of the debt, like a college kid going begging to dad to help pay down the credit card after they ran up too much of a bill.

I much prefer this characterization, where the engineering team are the lender, and they can call the business in to confront them with the unpaid bills that are coming due and demand a structured plan for how they're going to clear down some of their debt.

This is a great way of putting it! It's true that many middle managers and business/sales people do not respect this line of reasoning, but that is their mistake rather than a genuine or well thought out business decision -- which is why it bites them in the end. I would take this further and say that it's often a matter of professional disrespect to pay the architect's skepticism such little heed as to override the architect's vote to constrain feature creep. Self-discipline is hard!

One of the most challenging aspects of this in real development jobs is that shrewd managers and/or business and sales workers are very good at deflecting blame and creating political situations in which they are able to pretend to have supported the "right" choice no matter what the outcome.

So, very often, when features get hacked into the system against the architect's advice and it predictably causes untenable debt growth and parts of the system fall over, what is the first thing you hear? That it is the engineers' fault for not building the system correctly.

It's very rare that an architect can successfully communicate the problem of excessive feature growth and convey after a system failure that it was not really the engineers' fault, rather the fault of everyone forcing their poorly conceived features to take priority.

In a healthy development environment, everyone should be hearing "no" very, very often when asking to add features, especially when they come to you as if it is a big emergency that some market phenomenon mandates that the feature must be added right now.

If you find yourself in an environment where your employer does not empower you, as a developer, to say "no" in that situation definitively, then rather than trying to fight any blame wars about whose fault a system failure was, just get your resume ready and leave. There's no need to put up with that sort of poor engineering culture, and plus, wouldn't you rather put your name on something of quality instead?

Sure, except when your architect has some sort of contention, whether it's time, patience, or ego. Last time I had a decent idea to pitch, I pitched it to the architect twice - first time, it got completely thrown down. Second time, practically same pitch, except it was approved. I genuinely feel like a month was lost because of that.

There's something to be said for "just doing it", and so so many times have architects completely gotten in the way of making good progress.

It is both fortunate and unfortunate that there is a conflict of interest between entrepreneur and architect. As an entrepreneur, your job is to make money. The product is actually secondary. Risk is totally acceptable because you can start 20 companies and as long as 1 of them makes a huge payback, you are in the black.

Architects, on the other hand make products. I once was interviewed by a firm that asked me a pointed question about what I did to stop a failed project from failing. My answer was that I gave the best information I could, but that I left it up to the business to decide whether to fail or not. I still think that's the right answer, but it lost me the job.

As architects/programmers, we do not succeed by making money. We succeed by writing software. Several times in my career I have been hired to write shells of programs -- demos that will secure funding, knowing that the underlying software was essentially a lie. It just needed to be impressive and potentially disruptive. Probably the company who would shell out for it would cancel it the first chance they got anyway. Is it the type of software I would like to work on? No (though it did pay well ;-) ).

In any case, that tension is bound to exist. As long as there is good communication and honesty, the tension can provide good results. One of the first questions I ask in a job interview at a start up is, "Do you intend to make a product?" The second question is, "Do you intend to make the product successful?" The questions about technical debt are only really relevant if the answers to those questions is "yes".

Honestly, I feel you're making a much larger abstraction than what exists in reality (or maybe what SHOULD exist). Not to say that the abstraction doesn't exist, just that you're a bit too far in the othering-realm for my tastes. There's definitely a middle ground in there where the semantic difference between entrepreneur and architect doesn't exist - where good work and good ideas "are", regardless of how far you swing on that spectrum. I agree that the distinction between the two types of individual is necessary, but an over-use of it is an easy way to lose out in the long run.

The question about what you did to stop a failing project is an odd one and probably completely ineffective. If that's the reason they didn't hire you, take it as a good thing. There's too much nuance for it to be considered useful without other considerations. My recent situation was that I pushed too hard to stop a failing project and was socially cast out. By the time the senior staff noticed that my suggestions actually made sense and weren't batshit extreme, the social dynamic was too much and I left.

On your last point, I'm not sure that would be a useful question, either. It's so incredibly for a linux lead to say, "Our infrastructure security has no technical debt", for example. It's easy to say, but impossible to prove without being at the company for a while, even for an in-the-mud admin. Hell, that's what happened at the place I just left.

It's great to able to articulate that, but in the real world this sort of thinking does not actually hold much weight. Launching a month early is a good thing.

not necessary. if you launch early a crappy feature with a ton of problems it is actually very harmful. time to market is just one dimension of a trade offs you need to balance if you want to succeed

I often tell my management, "If it doesn't need to work, I can get it to you any time you want." Everything other than that is a cost/benefit analysis. To be honest, it's hard enough writing a reasonable sized program and not ending up with a horrid mess when I'm doing my best. I'll always recommend giving up functionality over quality if early release is desired. It's easy to add functionality later. It's hard to add quality.

What happens when the architect doesn't have the luxury of pushing back? As in, the business will make the architect's boss bend over if the architect says no?

Ed Yourdan summed it up very well in one of his books. I don't actually remember which one, so I will try to summarize it, but the quote is worth looking up. If you think that the project you are going to work on will fail; if you think that the work you are doing is crap; vote with your feet. Your time is precious to you. You only have so many years in industry to work on interesting projects. Don't waste them.

But there are times when the situation is not perfect, but you still don't want to leave. These can be good times as well. If the project has little or no chance of success, why not experiment with some unconventional techniques? It could be your last chance to do something like that. Working in a group which is failure tolerant can be liberating as well.

Finally, make sure that you fully understand the business decisions before you go overboard. In the very first start up I worked in, the management pushed estimates that were wildly inaccurate. We'd say that it would take 4 months and they would promise 1 month to the customer. We failed every single time. When we finally got to a product launch, many years late, I asked the CEO if he wouldn't have rather known at the beginning how long it would take. He smiled and replied, "No, of course not. I would never have gotten funding if I told people that it would take this long."

There are things that we, as programmers, know very well. There are things that we do very well. This is also true of business people. Sometimes we, as programmers, can't understand what they are doing because we aren't business people. Sometimes in our hubris, we think we know better than them.

Knowing who to trust in these kinds of situations takes time and experience.

Deliberately misrepresenting the amount of work it will take to build something is unethical and perhaps even fraudulent. I hate to see people do this kind of thing because it gives our industry a bad rap and could eventually lead to cumbersome regulation or certification.

It's true that software budget estimations are by their nature educated guesses at best. But we should still make an effort to be honest about them.

You shouldn't stay in a company where there is no power balance between business and tech.

Usually the technical debt will accumulate to the point the product can no longer move, is stuck on an outdated stack, and dies.

The best way I learned to deal with technical debt is to package it into securitized collateral debt obligations and find a product manager who can structure them into traunches and then find a QA manager who can rate these traunches and set up a regulatory framework with in which project managers can provide liquidity necessary to deal with it. If that does not work try to find an executive who can issue an Asset backed commercial paper that developers love. Now at this point the debt is handled so well it no longer is an issue.

The Freddie Mac style of development. Awesome!

This made my day. Thanks.

I think technical debt in the right places is fine. You have to know when it's OK to cut corners and when it's not. Novice software developers will sometimes think that all technical decisions are equally important, but that's not the case. Some technical decisions are way more important than others.

For example, I've found that anything related to the general architecture of your app is very important and you should almost never cut corners in those areas - This includes decisions related to what framework you will use, the structure of your URLs/links, how your app will use/reuse components, database engines, etc... These are fundamental engineering decisions and if you rush those, you will likely suffer for many years.

On the other hand, I found that it's generally OK to hack together the contents/features of individual pages within an app - If you mess up one page, you can always rewrite it fairly quickly.

Software is like a tree - If some of the leaves become diseased and start falling off - That's OK; so long as the tree's trunk is healthy and sturdy, the leaves can grow back quickly. If the trunk is rotten and the whole tree falls down - You will have to start again from scratch... If it doesn't crush you on the way down.

I don't see technical debt as a problem, I see the lack of refactoring as part of a software teams culture as a problem.

Technical debt will be incurred, if you like it or not. Even if you spend a long time on architecture and abstractions, developing software is a process and the "fantastic" architecture you created a year ago is now hopelessly outdated. This happens because over time your insight into the problem domain increases which will give you a better understanding on how to approach the problem which will lead to better decisions on all sorts of things regarding the software you're writing.

Having refactoring as part of a software development team culture can help you manage technical debt. If you assume that the code itself isn't precious but the experience of the developers writing the code is, then you can move forward. You can have refactor sessions at regular intervals, e.g. every Friday afternoon or a whole Friday every fortnight, whatever works for you and your team.

In a refactor session the whole team can work together to make the software better; many eyes make all bugs shallow [1]. It also give an opportunity to create better synergy and banter between team members.

[1] https://en.wikipedia.org/wiki/Linus%27s_Law

> Technical debt will be incurred, if you like it or not.

Right on. I've seen discussions about feature toggles which suggested not using them because they led to an explosion of code paths. But that won't happen if you delete the toggle (and the toggled-off code) once the a/b test ends or the feature is rolled out or whatever. It takes discipline and a culture that encourages that by being willing to review / deploy those changes, though.

The financial equivalent is more like selling an unhedged option than borrowing - the cost of paying back will increase unpredictably based on what other people do, not at a steady rate like interest.

I enjoyed the article, although I disagreed on some minor points, some of the broad strokes seem solid to me. Very thought provoking.

I think it's interesting to contrast the thrust of the article with Ward Cunningham's original definition of technical debt: https://www.youtube.com/watch?v=pqeJFYwnkjE

Wards definition is much more about the differences between the current implementation of the software and your understanding of the problem, particularly as your understanding changes over time.

He explicitly says that "tehcnical debt" is NOT an excuse to write known-to-be-poor code, which is why I disagree with the //TOBECLEANED approach.

This change in perspective shifts the point of view quite a bit. In some ways this problem defines a startup, your biggest problem is that you don't understand your problem. It's somewhat likely that you won't even have the same problem you'll successfully solve later. The first points are salient here, focus your engineering efforts on the things you don't understand yet, and the minimum support it needs to be viable. Shorten your feedback loop as you navigate the things you do not yet understand.

In this light, a startup is actively SEEKING technical debt, because it represents gaining understanding about a problem, and exploring that space quickly. This doesn't mean you have to "cut corners" and do things you know to be flawed, but it suggests focusing on small 'forays' into the unknown, gaining understanding quickly, and being willing to throw away your exploration for a new one if it isn't working. Then once you find a problem you start to understand and seems like a viable direction, you have a body of 'technical debt' that is the work you need to do to bring the software to your new growing understanding.

It seems like being willing to throw away the exploration if the direction isn't working out is a key advantage of a startup, along with the quick iteration times and lean support for explorations. This is why I dislike the suggestions of writing known-bad code. You're slowing your exploration down, but you aren't gaining understanding of the problem. It's junk debt, not technical debt. You can carefully minimize the supporting infrastructure your exploration requires, and avoid writing extra code, but you can't just slap it together.

It's not enough to flail around building as fast as you can, you have to deliberately learn and explore problems and understanding as you go, and evaluate which ones to put more exploration into and which to abandon. To me, this is the core work of a startup that differentiates it from established companies who are executing on a reasonably well understood problem. The (sometimes gradual) transition happens when you stop exploring, and start focusing on clearing the technical debt from your current exploration.

Thanks for sharing the article, I enjoyed engaging with these ideas!

The development team at a company I worked for stated they had technical debt. They said they would spend the next year eradicating it. Then didn't even try.

You want to know what technical debt really is? The result of procrastination in the realm of tech.

It's not even worth giving it a funny name like technical debt that upper management doesn't really get. They understand what procrastination is.

You can't spin procrastination. It is what it is.


Procrastination is letting for tomorrow what you know you should be doing today.

Technical debt is what happens when you try to do today what is due for tomorrow but you know deep down it cannot be done properly in less than 3 days. Then you deliver and you pray that none will be looking close enough to notice the difference.

What the article says is that technical debt is actually a good thing if whatever you are doing today has more than 50% chance of being thrown out of the window by the end of the month. If is better to try 20 crappy things, discard 15 and keep 5, than to do 8 good things and keep 1 or 2.

But this is only true when people take next month to fix the 5 things that were found to work in the past "sprint". The reason we all hate technical debt is because virtually no one does that.

Is someone who takes a financial loan procrastinating paying their bills? Or are they solving a short term problem the best way they know how?

Procrastination may lead to technical debt, but it's not the only source. The biggest source of technical debt is insufficient planning for future product requirements, leading to old code blocking new features.

That's why the concept is called "debt," because it's as if you took out a loan on the first iteration of the product, and then many features later, you're still paying interest on that loan.

So no, technical debt is not purely the result of procrastination. Sure, procrastination, especially in the planning stage, can create technical debt. But even the most productive people, those mythical creatures who never procrastinate, are vulnerable to creating technical debt for themselves. In fact you might argue they're more likely to create technical debt, if they choose to skip the planning phase and jump right into coding.

I don't know if "debt" or "procrastination" are really the correct words. It seems to be somewhat in the middle of the two, where the problem comes from "We'll take care of that in the future." where the future never comes.

A good example of that might be in log refactoring. I recently quit a place for what amounted to a complete lack of interest in correcting bad logs. After spending 8 hours trying to fix some non-prod issues and being completely halted by timeout errors, I was told "Those errors are fine, they even happen in prod (!!!). We talked about fixing it, but agreed to do it way further down the line." So, not really procrastination or technical debt, but something similar since it's 'debt' and 'procrastination' are simply symptoms of a larger problem.

>> insufficient planning for future product requirements

This covers both low prioritization and procrastination, the former of which I think is what you're getting at, and the latter of which I think is what purpled_haze is talking about.

It's the difference between "we'd like to clean up this technical debt but these new features are too pressing" and "we have nothing else going on, we're going to clean up this technical debt..." and then never doing it. In my experience, the latter happens because no one ever generates a real execution plan to get it done, often because there's just no reward for doing so.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact