

Ask HN: How to tell someone you need to rewrite an entire project? - dicarsio

What should you do when you see a <i>very bad</i> codebase and feel that the best - and cost effective way to complete your tasks efficiently, is to rewrite the entire project and your boss is in a hurry and wants to create fixes?
======
ams6110
I wonder, and this is not an insult, how much experience you have as a
developer. I know that for a good five years, maybe more, as a developer in
corporate IT I frequently thought that I was dealing with a "very bad"
codebase and rewriting seemed like an easier task than trying to understand
the current system well enough to make the changes I needed to make.

I was wrong. Every time. Unless the system is quite small and single-purpose,
you very likely don't understand the full scope of what it does, why it is
written the way it is, which constituencies it serves, all of its interfaces,
etc.

The few times I have been involved in rewrites they always took longer than
expected, cost more than planned, and ended up with kludgy patches at the last
minute as features of the old system that nobody talked about or knew existed
suddenly became showstopper must-have requirements. In other words, they
turned out like almost any other corporate IT project.

------
16s
Quantify it. That's all most managers understand.

Customer X wants the product in 12 weeks or they're going with a competing
product. Customer X may be spending 6 or 7 figures with us over the next 5
years in upfront costs and ongoing maintenance and support fees.

You may be saying, "It'll just take 6 months (24 weeks) to re-write this from
scratch and do it _right_ " (managers are used to hearing programmers say this
sort of thing). For the most part, managers don't care about "doing it right"
from a technical perspective, they only care about the bottom line. How is
_doing it right_ going to make more money (especially if we loose customer X
in the process???)

Also, be careful when making this suggestion. Many programmers who suggest
this are simply uncomfortable with other people's code. They can write stuff
from scratch themselves, but can't maintain or extend code they've not written
(that's always a bad sign). And to make matters worse, they always use the
argument "stuff I did not write is horrible".

~~~
dicarsio
That's not the case, as there is another project in the same company with very
good codebase made by one team, however, the one written by another team,
sucks :-)

~~~
16s
Another risk... has a re-write been brought-up before and shot down? Did
everyone agree (reluctantly) to make the best of it and continue with the code
base? If that happened and you bring this up again, it may not end well.
Management may see it as you undermining the project, not "being on board",
etc.

So unless you feel really compelled to do this, and you can do a good,
objective quantification (here is why a re-write will make us more money) as
why it _must_ be done, don't bring it up.

Also, telling mgt that a whole team in the company produces bad code may not
be in your interest and it certainly won't make that team your friend going
forward.

~~~
dicarsio
The code was written by outsourced people. And yes - another developer already
talked about a rewrite, but it was postponed. Thing is, if we postpone and add
lots of features, it will cost more to do it later! We can always rewrite
portions of it, and make it more stable, but I truly believe in a rewrite -
while the project is "young".

Another factor is the lack of budget in the project =/

------
nostrademons
Also, be _very, very sure_ that a rewrite is the right thing to do. Probably
80-90% of the time, it isn't, and you'd be much better off with an incremental
refactor.

One major way to tell: can you clearly specify what the revised version should
be doing, in a way that doesn't involve "whatever the old version did"?
Another litmus test: do you fully understand the old code and why it sucks the
way that it does?

~~~
dicarsio
An incremental refactor is a possibility too, but almost everything must be
refactored.

~~~
nostrademons
It's usually better to do things incrementally, even if the whole thing
eventually needs to be rewritten and doing it incrementally takes twice as
just rewriting from scratch.

~~~
jtheory
Unless the project is really very small, this is always the case with very few
exceptions.

Mind you, refactoring can happen at a fairly high level -- if you identify the
worst, most broken part of the code and first lock down/simplify its
interfaces, you can write a drop-in replacement for that whole chunk. Just
remember that if writing your drop-in replacement is going to take weeks
(rather than days)... or if your estimate keeps sliding..., you've probably
chosen too large a chunk and should slice & dice a bit more.

I've had this discussion so many times that now I just have it with myself,
and generally only say aloud the correct answer (though it's disappointing).

Psychologically, it's a beautiful thing to wipe the slate clean and just do
everything fresh from the start. If only this worked as planned more often
(and budgets were hefty enough to support "doing it right", and there were not
already a user base who actually _wanted_ to keep some of the broken bits
because they knew how to use them already, and you didn't have to deal with a
few team members who seemed to always be writing more code just like the old
stuff, etc etc)...

------
ludwigvan
Send them this article:
<http://www.joelonsoftware.com/articles/fog0000000069.html>

Then, send it to yourself.

~~~
rbanffy
There are legitimate reasons for a full rewrite.

If, for instance, you can change the platform and do the rewrite significantly
faster than the original technology and implement changes faster, then the
rewrite may be the may to go.

Imagine you have an app built on J2EE, with lots of logic on templates and no
tests whatsoever. It's huge and maintaining it takes a huge effort. Every bug
takes weeks to fix and generates wrong data that has to be corrected manually.

Imagine you can write an equivalent app in Ruby on Rails in two weeks, with
tests.

Which way would you go?

~~~
jtheory
Your two "imagine" clauses are incompatible, though.

"Imagine you can write an equivalent app in Ruby on Rails in two weeks, with
tests."

You've already said that the existing app has "lots of logic" and is huge.
Every bug takes weeks to fix, which means the "lots of logic" is seriously
twisted up and not easy to decipher.

If that's the case -- figuring out the logic to fix _one_ bug takes weeks,
then it's inconceivable that you'll untangle _all_ of the logic in two weeks.
That's before you write a single line of Ruby -- if you want an "equivalent"
app, that means you have to know what the current one does, first.

This is the usually-fatal flaw in the "rewrite" argument. The existing app
needs to be rewritten because no one can understand it anymore (is is both
poorly-written and very large/complex). But because no one can understand it
anymore, it will be extremely difficult to rewrite, unless you work as if you
were creating a new, similar application.

This is also why refactoring works. You cut off a bit of the existing app,
figure out just that bit, write tests for it, and then replace just that bit
with new code.

You can even refactor an app into a completely different language, actually --
you have two apps running parallel, and gradually move over bits of
functionality from one to the other. In the process you'll need to gradually
enforce order (and separation of concerns...) on the original app, but you
need to do that either way.

~~~
rbanffy
No, they aren't incompatible.

First, a huge Java app is about 3 to 10 times smaller (in LoC) when ported to
Ruby or Python.

Second, there is a lot of boilerplate code and configuration in a typical Java
app. Not nearly that much with either Rails or Django.

And third, basic CRUD functionality, something that has tons of Java
boilerplate associated with it, is usually very simple with Rails (or Django).

As for the tangled logic, it's the way many apps evolve. They are designed at
their start and then refactored and patched into their current form. The
current app wasn't designed anymore - it evolved - and, in order to properly
maintain it, you'll have to understand how and why its evolution took place.

One more point - in order to reimplement it on top of a more modern framework,
you don't need to understand thoroughly what your mutated-beyond-recognition
Java app does - you only need to know what it should do. The fatal flaw you
point out may not exist.

If you don't know what the app should be doing, then I have to agree - you are
doomed from the start.

Refactoring will give you some time, as it reverses the mutations back into a
designed state - and improve things a bit, but it will not be able to bring
the orders-of-magnitude improvements you see when migrating from an old and
obsolete (but still widely used) infrastructure, to a more modern one. That's
when a rewrite would be warranted.

~~~
jtheory
It sounds like you're referring to two different large Java projects, though.
If you can port it by just stripping out the boilerplate -- if it's at heart
just a CRUD UI with 7 frameworks and "enterprise best practices" mixed in --
then this is not the twisted and unmaintainable monster you're talking about
afterwards. It's less fun, but I've worked on lots of sprawling & unwieldy
Java apps implementing simple requirements, and it's not hard (it's also not
easy to justify a rewrite, unfortunately). Bugfixes don't take weeks (or even
days), particularly not once you know your way around a bit.

The app with the twisted logic (where bugs take weeks to fix, that has mutated
over years and years of hacked-in change requests and quirky tweaks often for
specific users' needs) is the one that screams for rewriting, but this is also
the app where no-one knows exactly what it does anymore.

And if you don't know what it does, I have no idea how you could claim to know
what it should do.

Many of the odd features are there for customers who aren't even customers
anymore, but no one knows. Even the code doesn't always help much -- I've seen
processing where it fails in the mid-method every time for some foolish reason
(and logs an error then returns), but the code _after_ the failure is actually
direly broken and must never be executed.

I've tried (failed, actually) rewriting one of these as well as doing lots of
refactoring on these kinds of projects.

With the first kind of app, you still understand it, so bugs can be fixed, new
features are possible, and there's no clear case to rewrite it. With the
second kind of app, you can either do what I mentioned above (and maybe you're
talking about this, more than a rewrite?) -- start talking with/watching
users, take a lot of notes, and build a new similar application all the way
through, which will necessarily start off missing much of the functionality of
the existing one.

Refactoring isn't about giving you time; it's about paying off technical debt
and forcing the codebase back into a state where you can do progressively
larger refactorings. This includes modularizing the existing app so that you
can replace it piecemeal. New pieces do not need to operate within the old
framework. It's quite possible (and sometimes desirable) to refactor the
original codebase completely out of existence.

------
ibejoeb
I'm just going to attempt to answer your question rather than question your
motives.

This is the difference between software and most other forms of
"construction." _There are no material costs._ There is no scrap to reclaim.
Bits can't be reused like steel. Tearing down code is negative work. This is
also goes toward the odd feeling we often have that software shares so many
attributes the fine arts even though it shares so many attributes with
engineering disciplines.

First: realize that the _word_ rewrite, not the act, strikes fear into the
hearts of people. Just don't say it.

Second: realize that the difference between a "fix" and "rewrite" is nil. Are
you going to fix it without changing it? Not if the problem is really in the
code. You don't edit out the plot hole in your novel or unpaint your landscape
into a lounging nude.

Now, the answer: say you'll fix it. You're really talking about the same
thing. You asserted that it would be cost-effective and efficient. To me, that
implies that it will address the urgency.

It's almost always quick and easy to fix what you know is wrong. The part that
makes it hard, slow, and expensive is rolling it out. For instance, it may
have taken a week to design a better data model for your SaaS app vs. a month
to try to find and mitigate a slow query, but now you have to figure out how
to migrate all of your clients, and that's a bitch.

------
gersh
It always depends on the circumstances. If people don't like the term rewrite,
you can just call it re-factoring or adding unit testing. You are probably
using something from the old codebase, right. If anyone asks why something
that used to work is no longer there, you can explain that you found bugs, and
so you temporarily took it out to add more testing.

Although, rewriting can be a lot of work. While there may be a temptation to
rewrite everything overnight, it can be quite difficult to pull off. Although,
I think devoting some time to a rewrite attempt may help you gauge the total
time it will actually take. If it proves easy enough, you can go for it.

------
rawsyntax
I once did this. However it involved some in-depth analysis of problems with
the existing code, and new ideas on how to solve the same problems. This came
about because the existing code didn't really fit a feature they were trying
to add on

------
profitbaron
The way I would approach it is, to create a really simple table which,
contains the pros of fixing the current code on one side and doing a rewrite
on the other.

Then I would provide the benefits of doing both and then I'd present this to
the person and say having, looked at our current code I believe that we should
rewrite it because of X, Y and Z, in fact I'd even summarize this at the
bottom of the table as well incase they want to take the table away and think
about it.

~~~
dicarsio
I guess I'll try that. The code is poorly written and with possible bugs in
it, so I can show a lot of problems to prove my point.

~~~
triviatise
Code that is working and in use is worth roughly 10x your imaginary code that
doesnt exist. So unless you could prove to me that the savings or value is 10X
the cost to rewrite I wouldnt do it.

Can you be more specific about why the code is bad? There are a limited set of
architectural reasons for which I would support a complete rewrite. They
typically correspond to non-functional requirements: 1) scale - the system
architecture simply cannot support scaling 2) reliability - the system
architecture is built in such a way that the system often fails 3) multi
language - the system needs to be modified to support multiple languages 4)
security - the system needs to be modified to support a more robust security
access model etc (there are many items like this)

These things all touch every single bit of code and so could warrant a
rewrite. Im skeptical of most other reasons. In most cases you can just
modularize the most dangerous aspects and you will be fine.

~~~
dicarsio
One of the major problems within this project are poorly written, slow db
queries, MVC is not correctly implemented (although using an MVC-based
Framework, function names not intuitive. I can improve, refractor some aspects
of the code, but the time that will took to fix it can be more than the time
to recreate it in another framework. What do you think?

~~~
triviatise
for me the question is still, what problems is this causing? slow db queries
can easily be isolated and unless they have done something like every write to
the database creates an object which then gets converted to a sql transaction,
function names can be fixed with global search and replace. Not sure what you
mean by MVC is not correctly implemented

~~~
dicarsio
I mean that, model bits are inside controllers, for example :-)

