
Predictions on the future of databases - thewarrior
http://gigaom.com/2013/12/14/5-predictions-on-the-future-of-databases-from-a-guy-who-knows-databases/
======
siddboots
The future is a fickle thing. 15 years ago I, along with many others, would
have told you that this was "the future of databases":

[https://en.wikipedia.org/wiki/The_Third_Manifesto](https://en.wikipedia.org/wiki/The_Third_Manifesto)

The difficult problems since that time turned out to be scalability and
parallelism, neither of which were addressed by The Third Manifesto. So
instead of D, we got Map Reduce and NoSQL.

Who knows what the hard problems of the next 15 years will be.

~~~
lmm
The object-relational mismatch is still a huge problem. I'm quite interested
by this supposed solution. But I spent 5 minutes chasing links and couldn't
find an example of _what the syntax actually looks like_ , only a suggestion
that I _buy_ some book to understand the language. This "D" thing sounds
pretty cool, but it seriously needs some more user-friendly marketing (and a
non-colliding, googleable name).

~~~
dragonwriter
Most of theninterestin info is linked from the reference material section of
www.thethirdmanifesto.com; what you Semmelweis to be looking for is:

[http://www.dcs.warwick.ac.uk/~hugh/TTM/Tutorial%20D-2013-05-...](http://www.dcs.warwick.ac.uk/~hugh/TTM/Tutorial%20D-2013-05-23.pdf)

Which provides a description of Tutorial D, an implementation of the
prescriptions.

~~~
dragonwriter
> ... Semmelweis ...

Or "seem" \-- I have really no idea how that happened and didn't notice it
until too late to edit.

------
tsahyt
There's a lot going on and that's the way it has been for quite a while now
but as someone who's working with databases only marginally at the moment, I'm
left wondering: What's wrong with RDBMS? Sure it's nice to have a simple Key-
Value Store for applications that don't need much more than this but bottom
line, the relational model is very powerful, albeit quite abstract, and most
data we need to deal with can be made to fit into this model. We have a
working, declarative query language that Just Works(TM), for which we have
written very good optimizers. The various instances of such DMBS range from
small-scale use (for example, SQLite) to really-huge (Postgres, Oracle, etc).
So, to sum it up: Exactly _why_ should we abandon this?

~~~
VLM
Most boots on the ground DBAs don't know anything about rational DB design.
Codd Normal form? Whats that? Can't I just make one giant table? Indexes? Why
would I want indexes? I heard they take up disk space and CPU so I won't use
any, to increase performance. So they (poorly) implement key value stores
regardless what the problem actually requires (if the only tool you know how
to use is a nail...). Nothing funnier that watching a noob do a select * on a
large table transferring gigs of data and use his RoR app to implement the
WHERE using nested ruby if statements and string comparisons (or better
regexes without understanding regexes, LOL) and god help me a bubble sort
instead of an ORDER BY clause and of course hit another table once for each
row selected from the first table because he doesn't know JOINs even exist, or
heard they're "slow".

A RDBMS is a very slow key value store, so anything that can't do transactions
or can't do anything else, is always going to be faster at its extremely
limited set of abilities.

From an engineering perspective if you need 10 HP to run your 5 KW generator,
a RDBMS is like installing a 10000 HP marine diesel and then not having any
idea how to start or maintain it or even where to get the fuel. Obviously
every objective performance metric would be better for a 10 HP lawnmower
engine in that app, if all you'll ever plan to use is 10 HP and you have no
idea how to use anything more advanced anyway.

There are also very loudly trumpeted anecdotal situations or contrived thought
experiments where certain unusual technologies fit a unusual situation very
well. This is, oddly enough, unusual.

I've read and experimented with the "7 DBs in 7 weeks" book and it IS very
interesting but I can't find any business cases to actually use any of it,
which is somewhat frustrating. And my experience is how you end up with people
writing CRUD apps to store cooking recipes that none the less use NEO4J
because they really, really, want to add a line to their resume that they used
NEO4J, not because the app needed it.

~~~
StefanKarpinski
Condescend much?

~~~
VLM
When I'm right, yeah. Not using "engineer" as a title of authority but as a
problem solving technique, maybe the TLDR is its an engineering problem where
non-engineers mess up RDBMS incredibly often, and when its blame time, better
off blaming the tool than the "designer" so...

"I don't know what I'm doing, but someone who knows what they're doing
anecdotally solved a completely different problem using tool XYZ, so for lack
of any better idea, lets copy them".

If you're familiar with cargo cult science there is an enormous miasma of
cargo cult engineering fogging up the entire database arena not just nosql.

Another concept that needs to be in the discussion is the "no silver bullet"
rule from programming applies to database design, like it or not. Can't just
sprinkle magic nosql pixie dust on any old random problem and expect it to
work, any more than applying any random programing fad to any random problem
will work.

The (old and new) tools are actually pretty interesting, although often poorly
engineered (by the end user) and implemented. Its the persistent anti-patterns
and non-engineering design technique that I'm properly arrogant and
condescending toward.

Hammers are a cool new invention and have some great unusual new applications,
but they don't install deck screws any better than the legacy screwdriver.
Laborers on the job randomly mixing screws nails screwdrivers and hammers on
the job and then internet discussions about how hammers and/or screwdrivers
suck is nearly physically painful to watch.

~~~
StefanKarpinski
Regardless of whether you are right or not, your tone is very off-putting.
This is not an academic concern – this sort of attitude is a huge part of why
the tech community ends up being very exclusionary. Oh yes, watching someone
make mistakes while learning how to build a web application or learn regular
expressions is hilarious. Much better to laugh condescendingly while watching
them struggle than try to help. Concocting "straw man" noobs everywhere who do
things in obviously wrong ways doesn't make you more right. And it definitely
doesn't make you smarter. It just makes you seem like a jerk and makes it a
little harder for people to stomach getting into the tech field. Please stop.

------
VLM
The most popular corporate world database system at this time is Excel. And
its not a very good DBMS, and the world is full of spreadsheet guys who know
nothing about database theory, designing horrific abominations.

On a large scale I think the future will be interesting, will "real databases"
remain in IT land or will it move out in the world?

Don't laugh, I'm just barely old enough that when I started out, printing,
file management, physical media management, static network address
configuration (not just IP in the old days), and backups were done solely by
IT professionals in the machine room, and now for better or worse every noob
does it (or tries to do it) themselves. If in the future someone comes up with
software to automate away DBAs...

Or it might go the other way and the corporate business database of the future
will be super turbo hyper i-Excel.

------
JackMorgan
Anyone who cares about databases absolutely must check out Datomic. If you
want to know the future of relational databases for the next 15 years, I think
it is here. Practically infinite client side scaling, logic based querying,
ACID, runs on several foundations, supports joins, all data is permanent so it
stores the entire history of every fact. You can query "right now" or in the
past at any known time. The database lives on your application stack the same
way git checks out your code on a development machine, and you push and pull
data to bring your application up to date with other stacks.

It doesn't meet every single need you might ever have, but it sure meets a
whole lot more then any other database I've seen.
[http://www.datomic.com](http://www.datomic.com)

~~~
misframer
I'm really skeptical about Datomic. I think it has some great features, but I
have a hard time believing that ACID transactions is one of them.

I've asked about this on their Google Group:
[https://groups.google.com/forum/#!topic/datomic/zkMV50VKw2s](https://groups.google.com/forum/#!topic/datomic/zkMV50VKw2s)

As of right now, there are still questions left unanswered.

~~~
stuarthalloway
[http://docs.datomic.com/acid.html](http://docs.datomic.com/acid.html)

------
beachwood23
I still think its amazing that Facebook is running off MySQL. Such a
tremendous amount of user data, placed in a system that many deem inferior.

I'm not saying they made a bad choice, I'm just impressed at how they've
managed to make MySQL fit their needs.

~~~
RyanZAG
I was under the impression they use it more as a sharded "key-values" store?

ie. no joins - the part of the database that makes it relational.

~~~
gbog
I concur. In our shop we also use MySQL without the joins. That's like an
efficient key-value store, but there must be a better solution somewhere: all
the joins are done in client code, and caching too. That's a great amount of
error-prone code that probably shouldn't be necessary.

~~~
bello
"all the joins are done in client code"

What's the point of joining in the application level? If you're going to join,
why not do it in the database? That should be both faster and more convenient
(unless your schemas aren't relational _at all_ , in which case I don't see
why you'd use a relational database)

~~~
RyanZAG
Data for User A is on server 9, and data for User B is on server 43. Joins
across servers is not really supported in mysql.

Another issue is connections between DB servers. Generally you want your DB
server to be as fast as possible, so having it handle the connections to other
servers slows everything down. If you offload the DB server connections to
your web server, you can easily scale by just adding more web servers and
having each DB server handle only its own data.

The best solution would be some type of 'mysql proxy' that could run on every
web server that would transparently handle the joins between all the different
mysql data servers. I think I saw a project attempting to do this awhile back,
but didn't really keep track.

------
digitalzombie
It's just going to be different dbs in the backend working with what they do
best in tandem.

Lucence base for search. RMDB for money/transactions. And other noSQL type for
just data that isn't critical, don't care much about relation, and just need
it to be fast.

Of course, perhaps PostgreSQL and other RMDB can have noSQL attributes then
noSQL would have some good competition.

PostgreSQL if they can make it easier to clustered, like cassandra or
elasticsearch. And get a better fuzzy string and other text search capability
in there then now we're talking. But I guess it's just me dreaming.

~~~
ddorian43
postgresql XC is master-master, shardable, cross-shard transactions, data
partitioning across nodes, distributed queries

~~~
aquadrop
Yes, but it's side project so it doesn't get such attention\confidence as
postgresql. But I hope the project won't get abandoned and will evolve into
something very powerful.

------
CmonDev
"Obamacare, for better or for worse, is being launched on a NoSQL database
system" \- silver bullet did not help (along with HTML5 etc. whatever).

~~~
Ygg2
So are you implying that a regular DB would have fared better?

I think it was a failure because of the complex law background and interfacing
with all kinds of legacy systems it has to draw data from.

------
bsaul
Could it be that facebook complete lack of innovation is due to them being
stuck for years with the same issues on the db ?

~~~
ddebernardy
Best I recollect, FB is quite innovative in the data center.

Also, it would be nice of you to step in and contribute to an open source
database engine if your expertise is such that you can dismiss what FB did
with them as lacking innovation. I'd wager they'll be all ears open for your
explanation how to do multimaster replication and distributed joins
efficiently while remaining ACID compliant.

~~~
bsaul
I wasn't talking about facebook technical innovations, but rather end-user new
features, sorry i wasn't clear.

------
aquadrop
By the way, as long as we're speaking about DB future, did anyone try
foundationdb? Does it live up to expectations?

------
h2database
What about H2?

[http://www.h2database.com/html/main.html](http://www.h2database.com/html/main.html)

I can't think of a better reason NOT to use it. Can someone who knows a lot
about db please comment on this.

~~~
ddorian43
[http://www.slideshare.net/PostgresOpen/postgres-is-
different...](http://www.slideshare.net/PostgresOpen/postgres-is-
differentfrombetterthanyourrdbms)

