Pickle, Marshal, and JSON are all ways of describing the kind of data structures used by Perl, Python, Javascript, and so on. They are way more flexible than Protocol Buffers -- you can have arbitrarily nested structures of arbitrary length. The format of the messages are basically self-describing. By which I mean that if they want to put an array there, they just put a '[' in and away we go.
I think XDR describes fixed-format structures. I don't know anything about DCE.
As described above, Thrift is a clone of Protocol Buffers.
Thrift and PB are a lot like XDR or god forbid, CORBA, but with a few twists. They define message templates for how to parse and emit relatively simple binary structures, which are then compiled into RPC clients and servers. PBs are cross-platform, at least by Google standards: they work with both C++ and Java very well. So you get the speed and convenience of making RPC calls with binary data structures in a mixed C++/Java environment.
But the best thing about PBs is that the message structure can evolve. If a message suddenly has a new field the receiver can't use, the receiver doesn't panic or read in garbage, unlike many other binary formats. So you can upgrade clients and servers in a gradual manner, without any downtime. This is what makes PBs especially suited to cloud computing.
You know, forwards compatibility is a feature of a great many binary protocols, including DNS, RADIUS, SNMP, and DHCP (itself a forward-compatible hack on BOOTP). So, PB is just another TLV encoding?
It seems like it. Before today I only had a vague idea how ASN.1 worked, and I never really delved into the wire format of PBs either. But yes, it seems to write a tag then a serialization of whatever the value is. Lengths are only required for strings because the format doesn't define any other type with variable length.
ASN.1/BER (you really mean BER) is the example most protocol dorks bring up when they argue against IDLs and structured protocols. It's incredibly complicated (though not as hard to implement as CORBA/IIOP).
indeed. i have used ASN.1 for a bunch of SNMP and GGSN accounting stuff, it is a pita. xdr on the other hand, is a breeze. also, xdr does support variable length fields.
