
An incomplete guide to Facebook thrift - rajeevk
http://www.avabodh.com/thrift/thrift.html
======
wwarren
We use Thrift a lot at my company, not just for internal communication, but
for our external facing API.

Larger companies like Evernote
([https://dev.evernote.com/doc/reference/](https://dev.evernote.com/doc/reference/))
have shown that Thrift can be operated successfully at an enormous scale.

Facebook also re-forked re-open-sourced Apache Thrift (originally a Facebook
project) as fbthrift:
[https://github.com/facebook/fbthrift](https://github.com/facebook/fbthrift)

~~~
Xorlev
The unfortunate part of public Thrift APIs is that you _must_ either provide
the IDL and hope your consumers are smart enough to work with Thrift, or you
have to take the Evernote route and provide SDKs for all the languages you
care to support. Admittedly, making SDKs for languages is a lot easier when
codegen takes care of 80% of the work.

~~~
lmm
Most options for an API offer you the same choice, no? If you're offering XML
either you offer a wsdl and hope consumers are smart enough, or you provide
SDKs for all the languages. If you're offering JSON you have to provide SDKs
for all the languages, since there's not even a standard IDL.

~~~
Xorlev
> If you're offering JSON you have to provide SDKs for all the languages,
> since there's not even a standard IDL.

I disagree. It's best to, but if someone wants to hack together something with
python/requests it's not hard to use the returned dictionary. Even with Java,
we'll frequently do some requests to get representative payloads, generate
models, clean them up and we're on our way. Or you could just use Map<String,
Object>.

------
chuckcode
Does anyone know of a data serialization framework that will do the usual
structs and arbitrary size matrices? I'd love to use something like thrift for
serializing matrices or other larger data over a wire or onto disk but last
time I looked thrift, protocol buffers and avro were all struct oriented and
there wasn't a good way to parcel up a matrix or large vector. Currently I've
just been using hdf5 as the default way to store reasonably large data
matrices, cubes, vectors, etc.

~~~
robrenaud
What's wrong with using a repeated float to represent a vector/matrix, perhaps
with a repeated int for indices if you are using a sparse representation?

~~~
chuckcode
Yes, I could take some time and write it myself. Take a look at hdf5[1] and
the features they have around meta information, data chunking, page level
compression for efficient reading of sub areas of data spaces. It is a non-
trivial amount of functionality but allows some pretty amazing performance,
for example see what pytables [2] has built on top of it. Now if they had just
made it easier to attach it to a wire as well as a file.

[1] [http://www.hdfgroup.org/HDF5/](http://www.hdfgroup.org/HDF5/) [2]
[http://www.pytables.org/moin](http://www.pytables.org/moin)

------
Xorlev
I found out the other day that Packetbeat supports decoding Thrift structs:

[http://packetbeat.com/docs/configuration.html#thrift-
configu...](http://packetbeat.com/docs/configuration.html#thrift-
configuration)

Which makes debugging Thrift-based architectures significantly easier.

------
GauntletWizard
This is honestly a better guide to Thrift than we have inside Facebook, though
I wish that said more.

~~~
rajeevk
Thanks you.. What more you expected in this?

------
m0skit0
Nice introduction. I selected Thrift as middleware for a business project for
the company I'm working on one year ago, so I'm pretty familiar with it. It
has some issues but for the most part it is reliable, fast, and extremely
flexible. Problem with Thrift is mainly the lack of documentation and
examples, so the more, the better.

~~~
tfigment
I have similar experiences and generally like it. We had to fix one ways in
Java at one point (now fixed properly) and not all languages have equal
support. We wanted SSL support on a .NET service but nothing official there
last I checked. FB updated and standardized SSL on C++ but left out other
languages by and large.

~~~
Xorlev
And a Scala one
([https://github.com/twitter/scrooge](https://github.com/twitter/scrooge),
idomatic Scala generated models/parsers) when paired with Finagle/Thrift is
pretty great.

------
wahsd
"Facebook re-open sourced the thrift..." cool, let me get right on that before
it's re-re-closed sourced.

------
zapf
Good to see this.

I was sold on Thrift but I think the popularity of REST made people take some
weird decisions.

Internal services talking to each other using REST seemed like a bit too much,
a place, where I though services like Thrift would have worked so well.

~~~
Xorlev
I think you mean HTTP+JSON. REST as a philosophy while applied mostly to HTTP
isn't really tied to it.

We personally decided to stick with HTTP (even though Thrift would be
"superior") since tools are numerous and simple. It's hard to beat curl and a
browser. We also found that services end up used in ways that are hard to
predict, especially in bash-written tools and the like. Thrift is a barrier to
entry.

Additionally, you _could_ do HTTP+Thrift. It's possible, it works, but you
lose some of the overhead benefits of Thrift at that point (IMO). In my
experience, fulfilling a Thrift request with a Thrift server has tighter
latencies at much lower CPU usage for similar request loads.

------
billowqiu
this is an interested introduction.

