

TokuMX 1.0: MongoDB with transactions, compression, MVCC, clustering indexes - ddorian43
http://www.tokutek.com/2013/06/announcing-tokumx-v1-0-tokumongo-you-can-have-it-all-2/

======
teraflop
I'll ask the same question that I had about RethinkDB last week [1]: how does
TokuMX handle secondary indexes on a sharded database?

Since it doesn't support cross-shard atomic operations, I'm assuming it takes
the same approach as RethinkDB, i.e. co-locating data with its index entries,
which means index lookups require querying every shard in the cluster. But I
don't see this explicitly stated in the user's guide, so it would be nice to
get some confirmation.

[1]
[https://news.ycombinator.com/item?id=5876168](https://news.ycombinator.com/item?id=5876168)

~~~
leif
The sharing architecture is the same as mongodb: data is partitioned based
according to a shard key and the secondary indexes are built separately on
each shard. So queries that use indexes other than the shard key must be
broadcast.

For 1.0 we wanted to match mongodb feature-wise as broadly as possible.

However, fractal trees can handle multiple clustered indexes very well, and
this extends to a few ideas I have for adding things to do sharding in what I
think will be a much better way. It's a bit up in the air right now but I can
tell you we plan to make some significant innovations here, and if you have
experience you can share, you should email us and we can talk it over and
maybe change what we end up doing.

------
netshade
I've been on the fence about trying this on Instrumental; we've been looking
at moving to DynamoDB, but I wanted to at least see if it would give drop-in
magic performance benefits, so I spun off a new SQS queue of our incoming data
this evening and did a write throughput test.

These are only initial impressions, but:

* It's definitely faster. Our write behavior is largely upserts against integers and doubles, and I'm seeing roughly 100% improvement against stock Mongo 2.4. The machine in question is an m2.2xlarge with a 1000 piops EBS volume attached, and it's doing about 7000 update operations a second. ( safe mode )

* I'm seeing consistently lower IO util than stock Mongo. Stock tends to vary wildly between 200-750 write ops, while under sustained write traffic, I see about 250 write ops.

* CPU usage is pretty well balanced against all cores, as opposed to stock's behavior.

* It's too early to say whether or not the storage savings will be as good as claimed, but at this point it seems that the TokuMX reprs are about 40% of the stock reprs. Like I said, most of our data is ints and doubles tho.

VERY LARGE CAVEAT I'm not running the database in a replica set because I'm
lazy. So, the write throughput numbers are likely the best case scenario of
what you'd actually be running in production.

(edit: line spacing)

If you've got to use MongoDB, it seems pretty nice. If it came with a tiny
person that maintained the database for you as well, it'd be a no brainer.

~~~
bretpiatt
Disclosure: Rackspace owns ObjectRocket and I work at Rackspace.

Have you tried ObjectRocket (if you're in US-East or US-West)?
[http://objectrocket.com/](http://objectrocket.com/) High performance Mongo
with replica sets.

~~~
netshade
I've definitely taken a look at Object Rocket, and it's a really nice looking
service. However, our dataset size would take us right into your custom quote
plan, and based on the rate of increase between the different plans, we'd
paying significantly more w/ ObjectRocket than we would w/ DynamoDB (if we do
indeed move to DynamoDB).

------
anonymous1983
Here are a few questions:

1\. Is this a drop in replacement if I'm on Mongo 2.4? 2\. If I want to
migrate back to Mongodb from TokuMX, what needs to be done? 3\. How quickly
does TokuMX integrate improvements from MongoDB? 4\. Is anyone using this in
their production deployment?

~~~
leif
1\. Yep 2. Dump and reload to get between mongodb and TokuMX. The storage
systems and replication systems are incompatible

~~~
leif
Wow sorry responding on my phone is hard. It's drop in if you're on 2.2. We do
not have some 2.4 features (FTS, geo).

------
mattparlane
This is pretty exciting. A couple of questions:

How are you (Tokutek) planning to keep up to date with the MongoDB tree?

Are you planning on talking to the MongoDB folks about upstreaming this? Or
will it be a pure fork with no sharing either way?

~~~
zardosht
Staying up to date is not an all-or-none proposition. We use feedback to drive
direction. For example, even though we used 2.2 as a base, user feedback
compelled us to include hash-based sharding, a 2.4 feature, in this release.

------
Scaevolus
Have you evaluated alternative compressors to QuickLZ? LZ4 and Snappy both
offer comparable compression speed and much faster decompression, which would
seem to be a big win for read-heavy workloads.

~~~
esmet
LZ4 is on our road-map for experimentation. I could see a faster decompressor
really helping when you have high-end flash storage.

------
ekyo777
I'll experiment with this a bit to see for myself but I would really
appreciate a list of downsides of using this over mongo; if anything else than
there may be less people supporting it.

~~~
leif
The simplest list is this: no full text search, geo indexes, or background
indexing yet.

~~~
ggoodale
Is there a feature roadmap somewhere detailing what order these and other
features will be added (actual schedule is great too!)?

~~~
leif
We're working on that, and we are hoping to base it on feedback, so if there's
something you want, definitely let us know.

~~~
wiremine
+1 on a roadmap. Excited to try this out!

------
kbd
I'd love to hear any opinions the RethinkDB guys have about this.

~~~
coffeemug
slava @ rethink here.

I have a few thoughts on this:

a) There is an astronomical difference between a neat technology and a
complete, usable, supported product (especially with really complex software
like databases). I can't tell yet how committed Toku folks are to this
project. Is this a research project that may or may not go somewhere, or are
they all in on the product? I think it's very important (for the customers and
the industry) to get a clarification on this point.

b) I love seeing engineering projects like these. Experimentation like this
(using a superb storage engine to power a popular db) is really exciting. I'd
love to see where this goes.

c) RethinkDB has its own state of the art storage engine (with a very
different architecture from Toku) that's tightly integrated into the full
system. That lets us do very interesting things (fast path code paths, btree-
aware caching system, etc.) The advantages and disadvantages of pluggable
storage engines are really interesting.

d) If TokuMX does turn into a complete product, it's really exciting. It's
nice to see the industry maturing.

~~~
zardosht
I work for Tokutek. Slava, thanks for the kind words. Just to clear up any
possible confusion, this is the product. We release and support TokuMX. The
research started roughly a year ago (the blog posts we've done are out there),
and it has led to this.

We consider this to be full featured in the sense that we have a feature set
that we feel users can deploy in production. As with any product, as users
give feedback on what more they would like to see, be it existing MongoDB
features or something else, we will use that feedback to enhance the product.

------
cheald
Can I put a TokuMX node into a Mongo 2.4 replica set cluster? It would be nice
to let it sync to the cluster and then compare it against live Mongo nodes.
Any concerns with sharding?

~~~
mrkurt
You can't have mixed replica sets, sadly.

------
ukd1
Do you think this would ever get merged in to the official release? Are / how
are you intending on commercializing it - MongoDB is AGPL?

~~~
esmet
There have been very informal talks with the folks at 10gen. The topic of
getting merged or not will ultimately come down to what users want.
Commercialization will start with support, much like what 10gen does for
regular MongoDB and what we do for TokuDB.

As for licensing, TokuMX embraces the spirit of open-source and we're
confident our open-source licence plays well with the AGPL.

------
HarrisonFisk
This really reminds me a lot of the progression of MySQL.

Originally it was used with ISAM/MyISAM and it was pretty popular. Then InnoDB
came around and it quickly revolutionized the MySQL world, allowing MySQL to
grow to the next level. Now InnoDB is by far the most commonly used storage
engine and the default on several distributions.

------
ggoodale
What's the storage multiplier for TokuMX vs. straight mongodb? Will my total
size on disk go up or down?

~~~
esmet
Most users should expect a big reduction in disk space. Depending on your data
and level of fragmentation with regular MongoDB, you could see ~4x-20x reduced
disk usage.

------
carterschonwald
the full source code is up at
[https://github.com/Tokutek/mongo](https://github.com/Tokutek/mongo)

