This is really cool but I'm curious why Bloomberg would need this. Ie, what special needs does Bloomberg have that would lead to a primarily non-engineering company investing the resources to create this. Was there nothing off the shelf that would have fit their needs?
I don't mean that in a derogatory way, I'm just curious what motivated making this.
But to answer your question: Mike Bloomberg's autobiography talks about this. When they started in the 80's there wasn't as much great off-the-shelf software as there is today.
Their customers were (and always have been) insanely demanding when it comes to reliability and speed. The last thing Mike wanted to do was be caught sheepishly explaining to the CIO of Merrill Lynch, "Well, gee, our Oracle database has this bug they can't fix for the next two weeks..." or even "...this optimization they can't make for the next six months"
This is a company that invented its own layer-2 network protocols 35 years ago just to squeeze every last drop of performance and reliability out of the hardware. Of course they wrote their own database (actually two -- there was a comdb 1 of of course, too)
Just for some scale, we have 4k+ engineering employees and if we were a public company, we'd be somewhere around the 5th largest software company in the world by revenue. (Strange stat, but our primary product is software and accounts for over 3/4 revenue)
I definitely wouldn't call Bloomberg a
"non-engineering" company if you've ever worked with any of their trading / market data products. Pretty much true of any trading side finance company nowadays.
Bloomberg started as a technology company. Sure, they have expanded into media, but I'm pretty sure they are still first and foremost a provider of software (plus hardware) and data.
I would bet that the Terminals are responsible for something like 140% of their profits, and that the media empire is a huge loss leader / vanity project.
The whole point of Reuters news is to inform traders about things that will lead to more trades being done over the Reuters platform, so it's entirely likely Bloomberg operates the same model.
So. We did a few things better than the industry to this day. HA is a spectrum. Transparent masking of all failures in any state of arbitary SQL transaction is on the far right side of that spectrum.
To directly answer your question: go look at where 'replication' was on rdbms 14 years ago. PostgreSQL? MySQL?
>>what special needs does Bloomberg have that would lead to a primarily non-engineering company investing the resources to create this
They probably open sourced the database, and would do more -- to dispel the above notion, and therefore to attract more qualified recruits.
Financial services companies, and technology suppliers around them, have came up much of the 'modern day' tech, some times decade(s), before google's and facebook's of the world
-- have had NoSQL with stored procs(eg Goldman S, early to mid 90s ),
-- in-house grown programming lang (APL+ based in Morgan Stanley) emphasizing vector-based operations
-- Smart contracts
https://www.lexifi.com/product/technology/contract-description-language
(where a contract is represented as algorithm specified in a domain specific language)
). This was way before etherium, started, I believe, in Credit Swiss (but not sure)
-- one of the fastest time series database (kdb+)
-- Data science and machine learning (modeling risk and valuations)
Today's hedge funds manage petabytes of data.
So do many of the big investment banks...
>> We had several goals in mind. One was being wire-format compatible with an older internal product to allow developers to migrate applications easier. Another was to have a database that could be updated anywhere and stay in sync everywhere. The first goal could only be satisfied in-house. The second was either in its infancy for open source products, or available at high cost from commercial sources.
I've worked at midsize "non-engineering" companies that do boring stuff like logistics where they wrote their own RDBMS and even virtual machines for legacy hardware.
It's not that uncommon especially if you've been around a long time.
They were, but rediculously expensive at the time. $100k+ at the time to have a non-replicated version on Oracle running on a multicore server. Dealing with Oracle was like dealing with the mob.
I don't mean that in a derogatory way, I'm just curious what motivated making this.