
Using Protobuf with PyPy - the_drow
http://omerkatz.com/blog/2015/09/11/using-protobuf-with-pypy
======
haberman
I didn't quite follow how cppyy/Reflex works (and the Reflex link is broken).
But I work on Python Protobuf at Google and wanted to clear up a few things
about that.

> Protobuf's compiler generates Python code which relies on a CPython
> extension in order to interface with the Protobuf implementation and is I
> mentioned before this is not going to be as fast as it should be for PyPy.
> Google provides a pure python implementation of Protobuf in version 3.0
> which is not yet release and breaks compatibility in some aspects from
> Protobuf 2.x.

Protobuf 2.6 has both a pure-Python implementation and a C++-accelerated one,
just like Protobuf 3.0. The C++-accelerated one is only necessary if you want
the performance benefits.

Also, Protobuf 3.0 does not break any compatibility with Protobuf 2.x.
Protobuf 3.0 _does_ contain a new protocol version "proto3", which differs in
some ways from the previous protocol version "proto2". But Protobuf 3.0
supports both proto2 and proto3, and the two are completely interoperable with
each other. Basically, each message you define is either proto2 or proto3. Any
proto2 messages you define with Protobuf 3.0 are 100% compatible with proto2
messages in Protobuf 2.x.

~~~
graffitici
So would it be possible to use the pure-Python implementation with PyPy
without having to resort to Reflex et al?

~~~
the_drow
To my best understanding, the pure Python implementation is much slower. Even
on PyPy.

I did not know that Protobuf 3 is compatible with Protobuf 2.

~~~
maxerickson
I read that as Protobuf 3.0 supports proto2, which is different than proto3
being compatible with proto2.

------
fijal
Hi. We had tons of success using protobuf-c and wrapping the resulting wrapper
using cffi. The pure-python protobuf wrapper generates really terrible code -
don't use it for anything performance-sensitive.

~~~
the_drow
Yeh I know. We discussed this approach on the #PyPy irc channel. As far as I
recall the wrapper is not open source. We needed something that can be done
quickly and therefore Reflex code generation based approach worked better for
us.

~~~
fijal
there is no wrapper ;-) at least no wrapper generator, we just hand-wrapped
some stuff.

~~~
the_drow
Reflex generates the bindings for us. That's why we chose it instead of hand-
wrapping the C API ourselves.

