Hacker News new | past | comments | ask | show | jobs | submit login

how does it compare to protobuf,thrift ?



Quoting apc @ https://lobste.rs/s/7w6p95/msft_open_sources_production_seri...

The current offerings (Thrift, ProtoBuffs, Avro, etc.) tend to have similar opinions about things like schema versioning, and very different opinions about things like wire format, protocol, performance tradeoffs, etc. Bond is essentially a serialization framework that keeps the schema logic stuff the same, but making the tasks like wire format, protocol, etc., highly customizable and pluggable. The idea being that instead of deciding ProtoBuffs isn’t right for you, and tearing it down and starting Thrift from scratch, you just change the parts that you don’t like, but keep the underlying schema logic the same.

In theory, this means one team can hand another team a Bond schema, and if they don’t like how it’s serialized, fine, just change the protocol, but the schema doesn’t need to.

The way this works, roughly, is as follows. For most serialization systems, the workflow is: (1) you declare a schema, and (2) they generate a bunch of files with source code to de/serialize data, which you can add to a project and compile into programs that need to call functions that serialize and deserialize data.

In Bond, you (1) declare a schema, and then (2) instead of generating source files, Bond will generate a de/serializer using the metaprogramming facilities of your chosen language. So customizing your serializer is a matter of using the Bond metaprogramming APIs change the de/serializer you’re generating.


That's cool... but, from what I can tell (correct me if I'm wrong), Bond accomplishes this by using common classes for in-memory objects which have no relation to the wire format, and then simply invoking a pluggable wire format and parse/serialize time. This lets you plug in previous-generation serialization protocols like Protobuf, Thrift, or Avro but probably won't allow you to plug in a next-generation zero-copy protocol like Cap'n Proto, SBE, or FlatBuffers, where the in-memory data structure and the wire format are one and the same. If you want to try one of them, you'll still have to rewrite all your code, unfortunately.


Hey Kenton. I see Adam Sapek bumming around the thread, so maybe he'll chime in here, but Bond works essentially by: (1) inspecting the schematized type, and (2) generating code that will quickly walk over that type and write it to a stream. So yes, it would probably require some surgery to make Bond do what Cap'n Proto is doing.

It is interesting to think about how it might work, though...

If you want to follow up, I encourage you to email Adam (adamsap -at- microsoft) or you can ping me and I'll loop him in (aclemmer@microsoft.com).


Thrift has pluggable protocols. It comes with 'compact' (protobuf-like), 'dense', 'binary' and json out of the box. It also has pluggable transports and multiple server implementations (threaded, async, etc). I'm personally not seeing any innovation here... I think they just wanted their own version of Thrift such that they could ignore the languages they don't care about.


> I think they just wanted their own version of Thrift such that they could ignore the languages they don't care about.

I'll put it bluntly: you have no idea what you're talking about.

Bond v1 was started when Thrift was not production ready. This is Bond v3. There is no conspiracy to make Bond hard to use for technology we "don't care about." In general I'm fine with tempered speculation, but your conclusion here is just lazy, and we both know it. It contributes nothing to the conversation, and spreads FUD for no good reason. We can do better, agree?

Now, to address your comments about customization directly: pluggable protocols are an example. The metaprogramming facilities of Bond are dramatically more rich than those of Thrift. A good example of these facilities: using the standard metaprogramming API and a bit of magic we have been able to trick Bond into serializing a different serialization system's schema types. So, picture Bond inspecting some C# Thrift type (or something), and then populating the core Bond data structures with data it finds there, and then serializing it to the wire.

This is the kind of power you get when you construct the serializer in memory using metaprogramming, and then expose that to the user. The flexibility is frankly unmatched.


Microsoft could have pulled a Facebook (They improved the C++ server in FBThrift) and forked or rewritten Thrift, reusing the IDL and existing wire formats if nothing else, and dropped a kickass C# implementation. The result would have been D, Rust, Go, PHP, and Python developers, etc etc, wouldn't have had to go off and reinvent the wheel for the Nth time, for negligible gain in terms of tooling, and an almost certain regression in terms of ecosystem and acceptance. I don't think it's FUD to express a bit of cynicism here or cry NIH Syndrome.

> I'll put it bluntly: you have no idea what you're talking about. Bond v1 was started when Thrift was not production ready. This is Bond v3.

Let me put something bluntly. I don't care who started writing code first. Microsoft are well over half a decade late to the party. Thrift and Protobufs have been public domain since, what... 2007/8?

And frankly, at least on the C++ front, there's not much to get excited about with regard to metaprogramming here. The Avro C++ code generator already produces a small set of template specialisations for each record type, and they're trivial enough to write manually for any existing classes you wish to marshal against your schema. std namespace containers are already recognised through partial template specialisations. MsgPacks implementation also does this. Other more general metaprogramming solutions, like Boost Fusion, are also being used by many, in production, for completely bespoke marshalling.

Don't get me wrong, Bond looks really nice, particularly for C# programmers, and I have respect for the work being done, but I can't get excited about it. It's kind of like someone announcing yet another JSON library or some new web framework when what the industry needs is consensus on formats and APIs. Right now there are so many serialization frameworks that the de-facto standard will just continue to be schema-less JSON and robust tools will remain largely non-existent.


I think your points are valid, especially about wire format compatibility. Bond and Thrift are in fact close enough that providing Thrift compatibility is a real possibility. It wasn't high enough on our priority list to make the cut but we do in fact use some Thrift-based system internally. The impedance mismatch in the type systems are higher between Bond and Avro/Protobuf so we will have to see about those.

I hear you on the fragmentation. I know that this doesn't help the community as an explanation, but big companies like Facebook, Google and Microsoft really have a good reasons to control such fundamental pieces of their infrastructure as serialization. Case in point: Facebook has forked their own Thrift project because, I presume, having it as Apache project was too restraining.

FWIW, we plan to develop Bond in public and accept contributions from the community.


>> I'll put it bluntly: you have no idea what you're talking about. Bond v1 was started when Thrift was not production ready. This is Bond v3.

> Let me put something bluntly. I don't care who started writing code first. Microsoft are well over half a decade late to the party. Thrift and Protobufs have been public domain since, what... 2007/8?

I think your technical complaints are almost all good and valid. I agree with essentially all of them. I don't want to fork Adam's sibling response here, so I'll leave it at that.

My point is that asserting that Bond was developed so that MSFT could purposefully ignore certain languages is pointedly wrong, and irresponsible considering the dearth of evidence you have to support it. And here you have 3 authoritative comments to say so. I don't understand how you can possibly disagree with this, or be upset that someone would take issue here. It's ok to be wrong.


I previously worked on Thirft and have overseen the development of Bond with Adam as the lead developer.

Your characterization of Thrift is accurate and Bond actually has some of the same architectural roots as Thrift. Those features of Thrift were ones that I wanted to preserve in Bond. But we also wanted to expand that plugability to allow for even more flexibility than the core Thrift architecture would allow for -- for example, the ability to support Avro-like "untagged" protocols within the same framework. I believe that the core innovation is in how that gets implemented. Also, we believe that performance is a feature -- our testing has shown that Bond noticeably outperforms Thrift and Protocol Buffers in most cases.

There is no conspiracy or intent to "ignore languages" -- we will release additional languages as they are ready and as we can support them as first-class citizens. We also welcome community involvement.


One key differentiator is the limited set of languages Bond currently supports:

"By design Bond is language and platform independent and is currently supported for C++, C#, and Python on Linux, OS X and Windows."

Versus Thrift:

"language bindings - Thrift is supported in many languages and environments C++ C# Cocoa D Delphi Erlang Haskell Java OCaml Perl PHP Python Ruby Smalltalk"


After struggling with thrift's Go binding (it happily generated broken Go code with the Aurora project's thrift file), I'm now skeptical that any of the others really works. I've never encountered a more frustrating project in this space.


Cross-language support in such frameworks takes a lot of effort. Whenever you add a new language you really hit the classic 80-20 problem.

We have support for a few more languages that we are using internally but after having a hard look at the implementations we decided that they weren't up to par for the open source release yet. I hope that we will release more soon. And needless to say, we are open to contributions from the community.


Or something like CBOR or JSONB?




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: