Something I’ve repeatedly observed in my career: the rewrite is started, but it never gets finished, and then the rewrite and the original coexist long-term
“Never gets finished” takes different forms. Two I’ve observed: Form (1): half the product gets rewritten in the new language/framework/stack, but the other half stays in the original - for whatever reason, rewriting the other half never happens. I once worked on a product where the UI was a mix of Java Swing, Struts, and Angular. The Angular UI opened the Struts screens in new browser windows. I wanted to switch to IFRAMEs to make it more seamless but never happened. For the Swing UI, you had to download and install it - I daydreamed about embedding it in the web UI using a JavaScript-based VNC client (e.g noVNC) but that never got beyond daydreaming. I suppose it would have made more sense to just rewrite Swing+Struts in Angular but nobody was volunteering for that painful grunt work.
Form (2): new customers go on the rewrite but existing ones stay on the original. Sometimes the easy customers get migrated (small scale implementations with few customisations), but the massive customers with heaps of customisations don’t. “If it ain’t broke don’t fix it”, and people are scared that the migration will go bad, the customer will get upset and you’ll lose them. Eventually they’ll churn, or embark on some radical change in direction that requires reimplementing it anyway, or maybe even M&A with another customer and want to merge their instances.
Sometimes this is called the “lava layer anti-pattern”
Another common issue is the product team just going crazy with new features on the rewrite. Which makes sense from a business perspective but turns it into a Frankenstein-rewrite and dramatically increases the chances of "never getting finished".
Ah, yeah, I've had something similar to that situation as well: I worked a place where you could see the different layers of technology, like different strata in rocks (or layers of lava, to use your metaphor).
You could see the SQL Server 7 era database with tables names taken directly from AS/400 mainframe fixed field width text files. Then you could see the mid-to-late noughties era SQL Server 2005 tables, as part of the same actual database, with its more human readable table and column names, but it was effectively a copy of the old database. All the business logic was triggers and sprocs.
And then the latest iteration, which I was working on, had been broken into "micro"-services, with our service using a Couchbase cluster (which, by the way, was a bag of spanners, and which I will never use again if I can possibly help it) to store yet another partial copy of the same data, and then all the business logic at the "application layer" in the closest thing I've ever seen to an actual seven layer architecture. They were pretty keen on enforcing those layers so what you had was just a ton of boilerplate function calls. The cognitive load of understanding what was going on was completely unnecessary for a system that basically just stored names, email addresses, addresses and phone numbers. Actually it didn't store addresses: it stored references to addresses that were stored by a different micro-service. These kinds of decisions meant that most function calls that actually did anything were also RPCs. You can imagine how this all performed.
Latterly I discovered that the blasted AS/400 mainframe system also still existed when I had to work with some files it was spitting out, and had to write an EBCDIC to UTF-8 converter (not actually difficult at all) to do so with 100% reliability.
The main reason for all of this was dependencies: to retire any given iteration of the system you'd have to migrate all its dependencies (of which there were many) to use the new variant. This sometimes meant we still had to sometimes extend the capabilities of the old versions of the systems, even though almost nobody understood them. So yours truly ended up having to write a stored procedure that updated a bunch of both the SQL 2005 and SQL 7 era tables to get a particular business process to work, and this sproc was called by our new microservice.
Even with the number of people they had migrating dependencies was a huge undertaking that they were only just starting to look at when I decided to tap out. An interesting project and a valuable experience for someone, certainly, but something I realised I didn't have the patience or fortitude for. They weren't idiots though: they knew they had problems, and that they were being choked by complexity, but there's no easy route to getting out of that kind of situation.
“Never gets finished” takes different forms. Two I’ve observed: Form (1): half the product gets rewritten in the new language/framework/stack, but the other half stays in the original - for whatever reason, rewriting the other half never happens. I once worked on a product where the UI was a mix of Java Swing, Struts, and Angular. The Angular UI opened the Struts screens in new browser windows. I wanted to switch to IFRAMEs to make it more seamless but never happened. For the Swing UI, you had to download and install it - I daydreamed about embedding it in the web UI using a JavaScript-based VNC client (e.g noVNC) but that never got beyond daydreaming. I suppose it would have made more sense to just rewrite Swing+Struts in Angular but nobody was volunteering for that painful grunt work.
Form (2): new customers go on the rewrite but existing ones stay on the original. Sometimes the easy customers get migrated (small scale implementations with few customisations), but the massive customers with heaps of customisations don’t. “If it ain’t broke don’t fix it”, and people are scared that the migration will go bad, the customer will get upset and you’ll lose them. Eventually they’ll churn, or embark on some radical change in direction that requires reimplementing it anyway, or maybe even M&A with another customer and want to merge their instances.
Sometimes this is called the “lava layer anti-pattern”