Speaking as someone who has done a fair number of rewrites as well as watching rewrites fail, conventional wisdom is somewhat wrong.
1. Do a rewrite. Don't try to add features, just replace the existing functionality. Avoid a moving target.
2. Rewrite the same project. Don't redesign the database schema at the same time you are rewriting. Try to keep the friction down to a manageable level.
3. Incremental rewrites are best. Pick part of the project, rewrite and release that, then get feedback while you work on rewriting the next chunk.
- remove one bolt and raise in the air
- slide out rev 30 "parts" from beneath bolt
- slide in rev 40 "upgrade parts" (fuselage, wings, engines, etc) underneath the bolt
- fly "upgraded plane" without a lot of pesky 'new plane" studies
Additionally, in parts of california, how to build a house:
- find existing house
- pick a wall
- remodel everything except for that wall
- reap tax benefits in your 99.9% new house.
Gradually re-written from Java (JBOSS) to Python over about 10 years. Basically the Python side knew what URLs it could handle and proxied the rest over to Javaland. We ended up shutting down the last Java bits early last year.
Partial rewrites based on Domain/Model/REST endpoint and just proxying to both webapps based on which was new already.
No breaking changes from the outside, either share the parts of the code that are still good (most of the business logic might be) or fork them (and then refactor and maybe you need to fix bugs twice) and after a while you can switch off the old part.
Works like a charm with added benefits of being in the same language to avoid wasting time. But the base idea works even if you use different languages.
It's also not for web apps, for example Apache Storm can do Bolts? (been a while) in several languages so you can also easily rewrite parts, if you can serialize your data in and out of it.
I would argue that if you already have a lot of in-house knowledge with Java then you might as well stick with it (and just not make whatever mistakes you made last time around) but Python seems like a reasonable alternative to me.
At the time the re-write started, Django hadn't yet seen a public release, Rails barely existed (and had zero traction, and Ruby-the-runtime was horrible), dotnet was barely a thing, Clojure and Go didn't exist yet, dotnet barely existed. 2005 was a weird time.
Realistically the only options at the time were Python, PHP (4, not 5), or Perl.
I didn't come in until much later, but I don't really think there was a better option at the time.
do you actually know what that would mean?
Chances are, that's the problem though. You have to re-write because your design is bad enough that it's necessary. The only time you'd re-write without changing the design is when changing languages or platforms.
Then, when that's ready and stable and working with the existing schema, it's way easier to do a refactor and adapt to the new schema too.
Attempting a design re-write without changing the data structures will just result in the same design. And you can't change the data design without then having to change the code to match.
I can't imagine doing that as an independent step towards a rewrite.
Sure you can. You decouple your data structures from your schema with an abstraction layer, facade, etc.
On the other hand, if you need to refactor the database schema, don't rewrite the code. Just do the minimum modifications to keep the app working.
Just don't do both at once.
Edit: Clarified quote of original rule.
It is more along the line of "don't try to do a big-bang release of everything" when you are initially writing an app.
(I think this article should be the new classic.)
> My takeaway from these stories is this: Once you’ve learned enough that there’s a certain distance between the current version of your product and the best version of that product you can imagine, then the right approach is not to replace your software with a new version, but to build something new next to it — without throwing away what you have.
Trello isn’t a rewrite of FogBugz. It’s a brand new product.
I can’t believe FogBugz didn’t share any code with CityDesk.
100%. rewriting all layers simultaneously is the first step to never finishing.
Rewrites are ALL about the schema.
Look, over time, your initial requirements will change. The initial schema will 100% not be what is ideal.
On a long enough time scale EVERYTHING becomes many to many.
I've found any rewrite of software without restructuring the schema has always been a waste of time.
Usually a quick glance at the schema will shine a bright light on top of all sorts of badness - denormalizations, excessive metadata-driven-ness etc. Rewriting code on top of crud like that would be epensive, and result in a lot of ugly code, which would need to be rewritten again.
They make little sense as soon as a project is completed, even if you rewrite, the software hasn't had time to point you in a direction, a 'nice' rewrite of the same wrong solution is just as useless as a bad implementation of the wrong solution.
They may make sense after a product has matured, since the actual scope of the software will be more defined. The 'correct' schema is starting to reveal itself, but usually it's rare that business wise it's worth rewriting at this stage, adding features is the better choice.
They definitely make sense once a product is fully matured, starting to gather legacy issues - since fighting around all the cruft and incorrect schema restrictions is wasting more time than redesigning it from ground up using the existing implementation as the guide.
That's perfectly compatible with the conventional wisdom you claims wrong, which, again, was “you should never rewrite your code from scratch” [emphasis added]
If your code is trying to get updated, you can usually refactor. If your code is having a paradigm shift, it's likely any incremental rewrite will take more time and have more duplication without benefit for far too long to be successful.
Refactoring is recoding a component while maintaining it's external interface so that external interactions with the component are unaffected.
Rewrites, incremental or otherwise, do not necessary preserve external interfaces (an incremental rewrite doesn't start clean slate, but instead progresses by gradual replacement of existing code.
The two ideas are not closely connected.
You can rewrite portions of the app and embed the older components in the new app until you rewrite them as well.
Some developers prefer to leave the company during the re-write (which usually happens after 3 years exciting hyper-growth).
In the first case, you have a system that mostly does what you want, but it has architectural issues that make it hard to change. You plan a rewrite. You get buy in from management. But the problem is that the business still needs those changes. If your rewrite takes more than a few months, then you're still going to have to do the changes.
Over time, you end up with 2 groups: legacy and greenfield. The legacy group is usually made up of crufty old salts that don't care what they work on. They just grind it out. The greenfield group contains usually younger people who what to do something new. They want to "get it right" this time. They care deeply about what they work on and so there is usually a fair amount of conflict on the team about how to approach just about anything.
The legacy team grinds along, slowly solving business problems and the greenfield team provides no business value until they are finished (while constantly promising that it will all be amazing as soon as they are done). But the business types see only that the legacy product is doing what they want and that the team is quietly plugging away. The greenfield product does not do what they want and the team seems to be doing a lot of excited talking, but it still isn't finished yet.
So they say, "Wouldn't it be better to move the greenfield team back on the legacy product so that we can get things done faster? After all, they said they couldn't move forward with the old architecture, but it's moving forward just fine. And with more resources it will go along a lot faster. Besides we have these business emergencies that we have to deal with, so let's just put the greenfield project on hold for a while until we can sort out exactly what's best". And the greenfield project is effectively buried.
Now, it doesn't have to turn out this way, but powerful forces point you in that direction and you have to be very careful not to have it happen to you.
The other main problem is that when people think about doing a rewrite the requirements document usually looks like "Do exactly the same thing as the last system". But the problem is nobody actually knows what the last system does precisely while simultaneously they all think they know precisely how the old system works.
You get a lot of push back from the business when you start gathering requirements because the only thing they really want to say is "Just do it like the old system". Only, after months and months of development you end up realising that there were a lot of corner cases that you missed in the rewrite. Oh... and it's not possible in the new architecture to do that without jumping through a lot of hoops.
So, incremental rewrites are best, but only if you can get the business to actually use the greenfield project. Normally they will completely ignore it because their paycheque and their promotion and their happiness depends on being able to do their job at least as well as they could with the old system. And the new system doesn't do everything that it needs to do (because we've released incrementally). What's more the developers keep asking stupid questions like "What do you want it to do" and the business people are responding with "Why can't you listen to me? I've told you one hundred times to do it the same as the old system!"
And so the new system is shunned by the business. It becomes radio active. Unless some upper management type sends a mandate down to force the business to use the new system, nobody will touch it. The upper management, in the meantime are wondering, "Why are we doing a rewrite again? The old system seems to do what we want, while everyone is complaining about the old system".
Yep, it can be done, but it requires considerably amount of help from upper management. You should avoid it at all costs unless you are sure upper management is supporting you all the way.
But even when you are successful, it may just be the beginning of the end. You've managed to stave off cancellation. You've managed to keep user engagement and work through the requirements so that the new system is equivalent. But it turns out that there is some small detail or decision that makes the new product non-viable.
For examples of products I've been personally involved with: Word Perfect 5 was written in assembly code and was not based on Windows. Word Perfect 6 was rewritten in C++ and was based on Windows. The team that did the rewrite were very proud of their work. But they through out all the keybindings in WP 5 (because GUI is way better!). The new rewrite was also very, very slow compared to WP 5. This was the beginning of the end for WP, even though one could definitely say that a rewrite was necessary. They just made the wrong choices.
Similarly, I once worked for Nortel and they had a telephone switch that they sold for $10 million a pop. It had 31 million lines of code (and stinky, stinky, stinky code at that). They realised that they could rewrite it in a fraction of that amount of code. They had 3000 developers working on the rewrite and after a few years they succeeded. Only... it didn't work with all of the business equipment that the old switch worked on. And it turned out that nobody wanted it unless it worked with that equipment. And.. the new architecture was not conducive to making it work with the business equipment. In the end, they gave a few switches to China before they abandoned it.
Rewrites are hard even when you've made the right choice to do it.
That completely fits my experience. If the new code is being used early and can grow iteratively while being relevant and useful, its gonna work out fine.
Having the codebases running side by side somehow is the best choice all around, if it can be managed.