

The future of databases is RAM clouds and SSD - petewarden
http://www.igvita.com/2009/12/07/future-of-rdbms-is-ram-clouds-ssd

======
Andys
The problem for RethinkDB is that the firmware on Intel SSDs is advanced
enough that it can speed up the normal, free MySQL engine (or postgresql, or
oracle, ...) to almost the same sort of speeds.

I am yet to see any evidence that RethinkDB's approach is anything but a niche
for those who need to squeeze an extra 5-10% that advances in filesystems and
flash firmware cannot.

~~~
coffeemug
It's not that simple. The FTL addresses two fundamental problems with flash -
slow random write performance and wear leveling. It works beautifully on
consumer hardware, but performs rather poorly in an enterprise environment.
For some workloads you can squeeze a lot more than 5-10% improvement out of
the drives. But that's not the interesting part (most of today's workloads are
read-intensive anyway).

The interesting part comes from being able to execute existing operations in
fundamentally different ways. When you're not restricted to sequential reads
you can implement a huge chunk of database internals differently - different
concurrency models, different implementations of certain joins, etc. This is
where the bulk of the performance improvement comes from.

Once you do this, you realize that a traditional bottleneck (I/O) is
completely blown away. Suddenly, all kinds of things nobody ever bothered to
optimize become performance bottlenecks (for example, the query plan is
traditionally interpreted, not compiled to native code - not good enough once
you get an I/O subsystem that's 250 times faster than what you used to have).

Once you get rid of the software bottlenecks, you realize that you're limited
by the bus. Since SSDs are highly parallelized devices, you have to change the
way you interleave operations for optimal performance. That alone is another
order of magnitude.

In three years or so, the flash manufacturers will put chips on the SSDs that
will allow you to send down programs to them (not unlike the shaders in video
cards). That will likely be a bigger revolution than SSDs themselves. Where we
are now is just the beginning.

You're right that we haven't shown any evidence that our approach is anything
but a niche. We will. This stuff just takes a long time to build.

~~~
pedrocr
_In three years or so, the flash manufacturers will put chips on the SSDs that
will allow you to send down programs to them (not unlike the shaders in video
cards). That will likely be a bigger revolution than SSDs themselves. Where we
are now is just the beginning._

This sounds crazy. SSDs have all this firmware and processing power to make
them behave like hard drives. If we want to drive the underlying storage
directly we might as well rip all that out and let the OS drive the flash
directly from the main CPU. Why would we want to be sending programs into the
black box? What can the firmware do that the main CPU couldn't if we stopped
attaching flash as SATA and instead just created a direct connection that can
DMA back and forth to main memory?

Some Linux developers have made that point, that they would rather be
implementing their own filesystems directly on the flash than dealing with a
firmware that is trying to make it appear that the SSD is a hard drive but
that then needs TRIM to work around the fact that that is not true. These
firmwares are nice for Intel by letting them turn a commodity (flash) into a
differentiated (and higher margin) product based on the code in the firmware.
Considering Linux already has filesystems that can operate directly over flash
and that better ones can be written I'd say we would be better off just
getting direct access to the flash.

~~~
DougWebb
I agree with your point in general, but if coffeebus' assertion that SSD
access will become limited by the bus, it still makes sense to allow programs
to be sent to run directly on the SSD. Remember, video cards originally were
just big ram buffers that were accessed directly by the CPU, but that wasn't
fast enough so now we have shaders on the cards. Having something similar in
an SSD will allow a lot of data searching and crunching to happen on the SSD,
and it'll run in parallel with whatever else the CPU might be busy doing.

~~~
igrigorik
I see your point, but I'm not convinced that the amount of data (raw volume
size) and the read speeds of flash lead to the same behavior as shaders + GPU.
GPU's are processing an order (or several) of magnitude more data, which makes
shaders a useful feature: data and computation locality.

Flash is 10x slower than RAM (in an ideal case), and even then, we haven't had
the need to introduce 'shader like' features into RAM - yes, because
historically we haven't stored all that much in it, but even with modern 64Gb
configurations, same thing...

~~~
DougWebb
I don't think the comparison is Flash vs RAM, I think it's "directly-accessed
Flash" vs "Flash on the other side of a PCI-E bus". We don't need anything
like a shader for RAM because the bus RAM is attached to is fast. The question
at hand is whether the Flash on an SDD is best accessed as a SATA device, DMA
Flash over PCI-E (or some new slot technology), or through shader-like cpus on
the same card as the Flash.

------
fauigerzigerk
That's interesting and very promising. But what I miss in most discussions of
relational vs NoSQL is some kind of separation of data model related,
performance related and scalability related arguments. These are three rather
different issues.

There is no way to have a benchmark alone show me what it means to denormalize
my data model. Denormalization can help scalability and sometimes performance,
but it has huge implications for maintainability, data quality, code
complexity and even organisational matters.

I do understand the advantages of being able to run any algorithm on data and
not be limited by SQL. But abolishing normalized data models altogether and
replacing them with application code and custom data structures everywhere
makes me very skeptical as someone who had to deal with data quality issues
for many years.

So that's why I'm glad to see some performance issues of RDBMS improved by new
storage architectures and also the innovation going on in terms of recursive
queries to deal with hierarchical and graph data structures.

~~~
tom_b
The missing discussion point for me is more fundamental - how and when should
I break with my RDBMS data modeling habits?

I can't say for sure, but suspect some of the headlong rush into denormalized
NoSQL stores is motivated by the ease of the data model. I wonder if once you
go down that road, if your "program contains an ad hoc, informally-specified,
bug-ridden, slow implementation of half of" what you get in a decent RDBMS.
(Forgive my hijacking of Greenspun)

That said, I'm usually working in denormalized models anyway (star schemas in
data warehouses for handy-dandy slice/dice/drill), so I'm completely open to
different ways of approaching data processing and analysis problems.

I actually feel more free with SQL than limited though. What I would really
enjoy would be to have the fully expressive power of SQL embedded in a natural
feeling way in a higher level language. I'm interested in Q (of KDB) for this
reason, have also been baking my brain in prolog to tear down old data model
habits, and hacking some in Ruby and Common Lisp for the same reasons.

Of course, I'm probably breaking my career in the process, since I've made a
habit of getting paid to work on RDBMS walking the line between
hardware/software performance tuning. But because of that, I'm also glad to
see new approaches to RDBMS storage and innovation in the whole DB space.

~~~
fauigerzigerk
I do work with denormalized data structures and schemas as well, but that data
is usually derived from a normalized source and it's read-only. All the issues
related to redundancy don't exist in a DW/BI scenario.

One area where SQL is clearly limiting is text processing and I'm doing a lot
of that. There's just no way you could do text analytics in a regular SQL
database. I'm sure there are other areas like that, so I agree with you that
innovation in the database space is welcome, even if it has acquired that
slightly unfortunate name "NoSQL".

------
chime
One of the most actively-used databases at my company is about 1GB in size and
grows at about 100MB/month. It is a Windows-based DB and is very heavy on the
reads. I installed SuperVolume
<http://www.superspeed.com/desktop/supervolume.php> on the DB server and
created a 4GB disk partition to store the DB files. SuperVolume mirrors the
entire 4GB disk partition in RAM and works flawlessly and transparently.

I was going to get an SSD for performance enhancement but I don't much see a
reason to do that anymore. All the reads happen at the speed of RAM and writes
are delayed and non-blocking. Nested queries that used to take 15 minutes now
take 45 seconds.

Right now the speed issues we have are solely due to unoptimized algorithms
and not due to hardware limitations. This is a wonderful spot for a coder to
be in because there is so much room for improvement now. If you have Windows
systems serving files and data on slow systems, I would highly recommend
giving SuperVolume and its related apps a shot before investing in expensive
SSDs.

------
Luyt
I put SSDs in all my development machines, and it indeed makes compiling
faster (about three times). When prices drop a little, I'll also equip our
database server with SSDs. Luckily the database is only 60G.

~~~
lsb
Interesting size; I was talking to guys at a gay dating site, and their
worldwide db was around 60 GB also; it could fit in RAM, on the 4XL amazon ec2
instances.

------
akamaka
I'd like to see a proper quantitative analysis done on this subject.

When you take money into account, are SSDs really worth it, or should I just
buy more RAM?

~~~
easp
Properly, Flash is another tier in the storage hierarchy. Faster than disk in
important measures, but more expensive. Slower than RAM, but cheaper, lower
power, and durable. Unfortunately, the infrastrucure for taking easy advantage
of Flash isn't very mature yet. SSDs make flash act like a hard disk, but OSs
and applications generally don't allow transparent fined-grained data
management to take advantage of it. I guess ZFS is best in this regard, since
it can migrate hot data to fast devices, but I haven't seen good options on
Linux, and I haven't seen hardware RAID controllers that could fill the gap.
In your DB, you can place tables on different devices, or use table
partitioning to separate recent data from cold, but that isn't as easy as
adding more RAM.

Ultimately though, cost effectiveness really depends on your workload. MySpace
is apparently finding it cost effective to use SSDs for their cache tier. A
few months back James Hamilton linked to a study that suggested that for some
common workloads, the case for SSDs had to include both price/performance and
performance/watt, since it didn't pencil out for each individually. That'll
change though, as prices drop and performance improves.

------
richcollins
"RDBMS systems are headed into main memory, which changes the playing field
all together"

It doesn't change the fact that RDBMSs don't do a good job of returning
representations of graphs.

~~~
fauigerzigerk
All the major RDBMS do recursive queries nowadays.

