
Four DB2 Code Bases? - fanf2
http://perspectives.mvdirona.com/2017/12/1187/
======
yeukhon
I am just going to drop a few quick comments. It's 6:39AM in NY, and I just
woke up. Maybe not relevant at all, but...

I love reading about biography like these. I own a couple books, for example,
"Where Wizards Stay Up Late: The Origins Of The Internet" did keep me awake
for a few days on an early train. This one, absolutely easy to digest, thank
you for sharing.

1\. So do the four code bases share code they reuse (think of a common
repository they depend on)?

2\. Are the query planners all different because they needed to be platform
specific?

3\. Is DB2 still on four code bases?

Would be awesome if we have one central catalog of stories like this one.

BTW, I didn't know much about the author, so I did some quick googling:
[http://mvdirona.com/jrh/work/](http://mvdirona.com/jrh/work/)

(I hope HN won't flood it to 404! Mr. James, is this on AWS!? It looks like it
might be.)

~~~
jhalstead
If you have time to list the other books that you own like this and "Where
Wizards Stay Up Late: The Origins Of The Internet", I'd definitely appreciate
it. I love reading these types of stories too!

~~~
lobster_johnson
"Showstopper!: The Breakneck Race to Create Windows NT and the Next Generation
at Microsof" by G. Pascal Zachary is great, about an interesting period of
Microsoft's life.

------
spenczar5
I’m one of those youngish programmers who has never really worked on anything
but Linux, so the idea of these IBM operating systems is pretty foreign to me.
The article describes System i as “very advanced” but I have never even really
heard of anyone using it or referencing it.

How do you write and deploy software for these things? I am so used to just
being able to, like, build a Go binary and run it on a Linux box. Are they
better in some ways that I am missing out on? Just curious.

~~~
snuxoll
IBM i is advanced in some interesting ways, but the design has plenty of
drawbacks. The database (DB2) is baked right into the operating system and is
an integral component virtually every application relies on it - you can
access tables via SQL or directly through record access (equivalent to
fopen(), but against database tables). In fact, the “traditional” UNIX like
filesystem (the IFS) on IBM I is an entirely different world, the object
storage subsystem (the database) is where 99% of everything live on IBM I
including most applications. Applications aren’t compiled to machine code
directly, outside the lowest level code needed to get the system running -
they’re all stored as bytecode and compiled to machines code on first run,
which is how IBM has taken advantage of newer POWER releases without requiring
customers beg vendors to upgrade or rebuild their code. There’s a hell of a
lot more, but these are two key components of the design.

As for writing software, you basically have no choice but to use tooling IBM
has support for. There is a emulated POSIX environment (PASE) but it’s mostly
used by IBM to port things from AIX - the traditional IBM i languages are RPG
plus C/C++ as a newer addition, but they have runtimes for Java, Python, Node
and a few others. The traditional languages are the only way you have to make
classic green screen apps, while the latter are there primarily to host web
applications.

~~~
krylon
> but the design has plenty of drawbacks

As somebody who has never been near an AS/400, what are those drawbacks?
Besides the environment being weird in the sense that it is unlike anything
else?

The only bad thing I have ever someone say about the AS/400 was a supervisor I
worked for during my training who had at one point written RPG programs, and
she said ... some pretty unfriendly things about RPG. Which is not say that's
the only problem; like I said, I have no knowledge of the system beyond what
Wikipedia tells me about it.

~~~
snuxoll
There’s extremely limited compatibility with POSIX software, PASE exists but
you aren’t just going to be porting applications with minimal or zero effort.
Honestly IBM i makes little sense as a modern application platform, and DB2
for i is wonky plus licensing for it sucks (if you’re connecting from anything
but Java where you can use open source drivers expect to pay through the node
for DB2 Connect unless your application also runs on i).

------
guitarbill
I have a lot of respect for databases in general, and DB2 is no different. But
if you've ever tried to install DB2, you'll know it's a monstrosity, and even
internally it was hard to get a license. The result of this is that most
IBMers I know would rather use e.g. Postgres for other projects, not DB2. Now
I'm not silly enough to think Postgres will replace DB2 at e.g. banks, but the
effect is that Postgres reputation and knowledge spreads, and DB2 reputation
and knowledge die. In the long run, I think this is a good thing.

Also, if you read between the lines, there's a ludicrous amount of infighting
in IBM, which is probably how they ended up with four code bases. Still
happens to this day.

~~~
gaius
_Now I 'm not silly enough to think Postgres will replace DB2 at e.g. banks_

You might be surprised then, because that's not silly at all. DB2 may hang on
in the Back Office, the least technically sophisticated bit of banking for
various reasons for a while longer, but it's long gone from the Front Office
and Postgres is one of the replacements. I don't think anyone is starting new
projects on DB2 even in BO.

~~~
ghshephard
Are there any databases successfully in production in high-value transactional
environments other than oracle or DB2? I have to believe a lot of the pain
coinbase is experiencing stems from their decision to use mongodb - which I
love and use daily, but would never, ever have chosen over Oracle for high
transaction bitcoin exchange.

I wonder if anybody has done a write up on this oracle/db2 versus “others” in
the super-high-value back office scenario.

~~~
dmd
_Are there any databases successfully in production in high-value
transactional environments other than oracle or DB2?_

Teradata!

And IMS, but that's not relational.

~~~
snaky
And ADABAS (officially declared fully supported through the year 2050 and
beyond). Most often considered non-relational too.

High-value transactional environments most often neither relational nor even
running on operating system (in modern sense at least), like zTPF "because a
real operating system is too high level and therefore too slow for real
transaction processing needs". Current users of TPF include Sabre, VISA,
American Airlines, American Express, HP SHARES (formerly EDS), Holiday Inn,
Alitalia, KLM, Amtrak, Marriott International, Travelport, Citibank,
Citifinancial, Air Canada, Delta Air Lines, Japan Airlines and many others.

------
CrLf
The article mentions Informix, which has also been an IBM product for a number
of years now. That makes it at least five major RDBMS code bases inside IBM.

I spent a few years managing DB2 instances at a previous employer and I've
seen it hold its own against mixed workloads (i.e. shitstorms of bad-behaved
applications and people running ad-hoc queries) that I don't think even a well
tuned PostgreSQL could handle today.

DB2 has some really annoying quirks(1) and that IBM'ish feel that's very hard
to explain, but it's also very understandable (from a tuning perspective) for
such a complex piece of software.

DB2 for Windows seemed mostly crap compared to Linux or AIX, though.

(1) For example, at least since I've last looked into it, DB2 had no decent
MVCC(2) and it was a real pain to deal with all the lock-related problems that
would arise from large write/update transactions running in parallel with
small reads.

(2) At some point (9.5, IIRC) they tried to simulate MVCC by letting
transactions read previous row versions from the transaction log. This seemed
to come with its own set of significant drawbacks.

~~~
geophile
I interviewed with Informix shortly before IBM acquired it, and after they
acquired Illustra (a commercialized Postgres, company started by Stonebraker).
My recollection was that Informix was a mess having decided to basically build
their next major version on the untested Illustra. Different locations, lots
of politics, two massive codebases. I remember thinking that Informix shot
themselves in the head, and that IBM then acquired them cheap.

~~~
derriz
That brings back memories. I was involved in trying to build webapps on
Illustra in the mid/late 90s. We (a small dev shop) ended up with it because
it was sold to our non-technical manager on the basis that it was the future
for the web and it was ridiculously expensive for the kind of bottom-feeder
projects we were doing - which was kind of galling given we were being paid
peanuts.

But even I was initially excited - the feature list looked cool - multimedia,
rich/extensible data types (called blades?), OO/table inheritance, built-in
dynamic web page generation, etc. But the performance was absolutely terrible
and all the advanced features turned out to be less than useful or slightly
broken. I remember my disapointment when I first tried out the "search for
images LIKE a reference image" feature and it matched a butterfly with a
racing car. We struggled for a few months with the general flakiness and
horrible performance of Illustra and eventually convinced them to allow us to
use Informix instead and went back to storing images as files served by HTTP
and using PHP or Perl for the dynamic HTML generation. The Informix sales
dudes seemed convinced that Illustra was the future.

The performance was much better but for the web apps we were writing, we could
have just stuck with mSQL or MySQl and saved a lot of time and effort.

~~~
gaius
_multimedia, rich /extensible data types (called blades?)_

Yeah, DataBlades. I too was burned by them in the 90s.

------
gaius
Because DB2, like Watson, is an overall brand name for a suite of products.
This is a standard IBM practice.

------
BrentOzar
The timing of the post is interesting given Microsoft SQL Server 2017's new
support for Linux. Microsoft's doing it with one code base to span Windows,
Linux, and Azure SQL DB, and it's interesting to read why IBM took a different
route.

My favorite line out of it: "so I ended up taking a long weekend and writing
support for a primitive approach to supporting greater-than-2GB tables." Wow.

~~~
megaman22
That line is especially great in the context of the preceding paragraph:

> At the time I was lead architect for the product and felt super-strongly
> that we needed to address the table size limitation of 2GB before we
> shipped. I was making that argument vociferously but the excellent counter
> argument was we were simply out of time. Any reasonable redesign would have
> delayed us significantly from our committed product ship dates. Estimates
> ranged from 9 to 12 months and many felt bigger slips were likely if we made
> changes of this magnitude to the storage engine.

>I still couldn’t live with the prospect of shipping a UNIX database product
with this scaling limitation so I ended up taking a long weekend and writing
support for a primitive approach to supporting greater-than-2GB tables.

It's amazing how often one person with a solid grasp of a problem can run
circles around teams that are shackled by process and cya and politics.

------
pavlov
_Db2 for z /OS will live on for the life of the IBM mainframe ..._

This raises a question which is interesting to me (not knowing anything about
mainframes).

How long is the life of the IBM mainframe going to be? Are there specific
plans already in place to replace these systems on a known schedule? Or are
they going to be around indefinitely?

~~~
delinka
I suggest they’ll be around indefinitely. While large customers are willing to
spend large sums of money to maintain their systems and avoid the risks of new
technology upgrades, IBM will continue to serve those customers.

~~~
mseebach
Forever is such a long time. There are no new projects started on mainframes
and there's a definite, if still quite flat, trajectory towards moving things
off the mainframe. So _eventually_ it will die, but it will be decades.

~~~
delinka
As we approach the limit (of zero) that tail will just continue to get longer
and longer. And by “indefinite” I do not mean “forever.” I mean that there’s
no definite date that the ‘mainframe industry‘ will end— IBM hasn’t published
any roadmap to end their use, and many institutional users are happy making
few changes to business operations.

