

FoundationDB Fault Tolerance Demo Video - nlavezzo
http://blog.foundationdb.com/post/50924738464/foundationdb-fault-tolerance-demo-video

======
nlavezzo
At 4:40 in the video we do a network partition - which seems to be relevant
today in light of the aphyr / Jepsen post.

Also covered are single and multiple machine hard power failures, rejoining
the cluster, and total cluster power loss and recovery, all in one take.

Happy to answer questions.

~~~
ericingram
Question: Does FDB have any sort of "find" method in its API? As a key/value
store I understand that I can set/get keys, but is there any way to find keys
based on nested values?

I.e. fbd.find({someproperty: "123"})...

~~~
voidmain
A database is going to do a query like that in one of two ways: scan over (all
of|a column family of) the data, or look in some kind of index on
"someproperty".

In our approach this kind of functionality is the responsibility of a higher
"layer" of the system above our key/value API. There are lots of ways of
organizing and indexing data with different tradeoffs between search and
update performance for different kinds of queries. We support pretty much all
of them-- not by building any of them into the core, but by providing a
general transaction facility that permits arbitrary indices to be updated.

For example: in a database like Mongo, queries like that one don't scale
linearly even if there is an index, because you have to ask every shard to
look in its local index. In FoundationDB, you would store the index in the
database as keys like (someproperty_index, "123", _id) and an index read would
only need to go to a single server.

It can be a hassle to create all the higher level functionality yourself,
though! Over time, we and others will be offering more "opinionated" layers to
save you time.

------
ericingram
What a great demo! We're currently building a critical service on MongoDB and
feel a bit leery about some of the issues others have expressed at scale.
Would be interested in testing FoundationDB for our purpose in the near
future.

~~~
nlavezzo
Thanks! We thought since it was a hit in person we should share it a bit
broader.

You mention scale - do know that what you're seeing in the video is probably
the weakest FoundationDB cluster ever assembled :) Our in-house testing
cluster is a 48 machine, 192 core setup, and it works great and scales almost
linearly. You can see our performance numbers at
www.foundationdb.com/performance (though those results are outdated now, the
tests were on a cluster half the size and several versions ago).

If you'd like to chat about your particular use case, send me an email at nick
at our domain.

------
dangerboysteve
Really interesting talk and I also just watched your "NYC Database Month"
video.

What are the node costs in your licensing model ? Also any plans to release
Flow as open source ?

~~~
nlavezzo
Thanks. We haven't published pricing yet but we are aiming to make it in the
ballpark of the hardware & operational costs of the machines required to run a
particular size cluster. I.e. if you are running a FoundationDB cluster at 50%
load then the licensing costs shouldn't be out of sync with the costs of
owning and operating, or renting, the machines in the cluster.

It does look likely that we will open source Flow at some point, but it will
take a good amount of work to do it properly and right now there is plenty of
it to go round. Always good to hear interest for it though!

------
burgreblast
Great demo. Really curious about the nuts and bolts of the set up. What OS,
and what is actually launching at startup? Again, very nice job.

~~~
nlavezzo
I'd have to ask one of the guys that put it together for specifics in the
morning, but the OS is Ubuntu Linux, and the applications auto launching are
of course the FoundationDB server which then connects to the rest of the
cluster and begins doing work, and also a client with a Firefox browser
running the testing / demo app which increments the counter, changes the
circles, and uses (I think) HTML 5 to render the images.

~~~
SirSkidmore
Kind of irrelevant, but just out of curiosity, but what are those little PCs
and monitors? They look very interesting!

~~~
nlavezzo
The boxes are by a company called Zotac. They're like $200 without drives and
we put cheap consumer SSD's in them. The monitors are USB powered, and driven
by USB - they are from a company called DoubleSight.

Funny story - given the amount of people at Disrupt that were not at all
interested in databases, we had probably just as many people ask us detailed
questions about "those cool little computers" as we did our database :)

------
djtriptych
Amazing demo. I'd love for aphyr to take a closer look...

