
Pybind11 – Seamless operability between C++11 and Python - jtravs
https://github.com/wjakob/pybind11
======
jboy
If you like to program in C++ and Python, the Nim language might appeal to
you: C++-like features (generic types, operator overloading, function
overloading, inline functions, optional O-O, optional data hiding, C pointers,
bitwise-compatibility with C, your choice of manual memory management or GC)
and C++-like run-time speed, combined with Python-like syntax and compile-time
Lisp-like macros.

I spent most of a decade mastering pre-11 C++, learning to apply the
recommended tricks & idioms (like the copy-swap idiom for strong exception
safety guarantees), learning to sidestep the gotchas. Meyers books, Sutter
books, GotW, "Modern C++", etc. Then, when C++11 came out, the language became
even more complicated, not less. That was a breaking point for me.

As a long-time programmer (and fan) of both C++ and Python, Nim offers the
best balance that I've yet found between C++'s ethos of thoughtful, precise
control and Python's user-friendliness.

(And if you ever happen to need seamless Nim-Python compatibility, including
native Nim support for Numpy arrays with C++-style iterators, my Pymod project
may be of assistance: [https://github.com/jboy/nim-
pymod](https://github.com/jboy/nim-pymod) )

~~~
coldtea
If only it didn't have some straight up bizarro choices, e.g. with regards to
case sensitivity

~~~
jboy
Nim has made 4 syntax choices that might seem bizarre to a C++ programmer: 1.
blocks by indentation instead of braces (aka "Off-Side Rule" syntax [0]); 2.
Uniform Function Call Syntax [1][2]; 3. case-insensitivity for identifiers;
and 4. dropping empty parens from a function call.

[0] [https://en.wikipedia.org/wiki/Off-
side_rule](https://en.wikipedia.org/wiki/Off-side_rule) , [1]
[https://en.wikipedia.org/wiki/Uniform_Function_Call_Syntax](https://en.wikipedia.org/wiki/Uniform_Function_Call_Syntax)
, [2] [http://nim-lang.org/docs/manual.html#procedures-method-
call-...](http://nim-lang.org/docs/manual.html#procedures-method-call-syntax)

Choice #2, Uniform Function Call Syntax (UFCS), allows you to write
`a.someFunc(b)` or `someFunc(a, b)` interchangeably.

Choice #3 is case-insensitivity for identifiers: You can write `a.to_lower()`,
`a.toLower()` or `a.tolower()` interchangeably.

Choice #4 is the ability to drop empty parens from the end of a function call:
`a.len()` can be written as `a.len`. Combined with UFCS, this allows you to
write `len(a)`, `a.len()` or `a.len` interchangeably.

I'd already been programming Python for years, so I wasn't surprised by choice
#1 -- in fact, I was pleased. After initially being highly skeptical of
Python's OSR syntax when I first encountered it, I've since come around
completely. The most common complaint against Python's OSR syntax is that it
allows both tabs & spaces, interchangeably, which has bitten just about
everyone who has ever used Python in a team. Nim avoids this problem by
allowing only spaces to be used for indentation, not tabs: [http://nim-
lang.org/docs/manual.html#lexical-analysis-indent...](http://nim-
lang.org/docs/manual.html#lexical-analysis-indentation)

My eyebrows certainly went up about #2, #3 and #4 when I first encountered
them. But you know what? Much like Python's OSR syntax, I've now come around
completely. Now I actually _prefer_ #2, #3 and #4 the way Nim does them. When
I'm back in Python, C++ or C, I wish they behaved the same way as Nim!

Think about it: How many stylistic debates have there been about whether an
operation in C++ should be a function or a method? How many times have you
pondered whether an object attribute should be a member or a method? It's just
a distracting detail with no benefit. And now you don't need to care!
Apparently Bjarne Stroustrup is a convert to UFCS too: [http://www.open-
std.org/jtc1/sc22/wg21/docs/papers/2014/n417...](http://www.open-
std.org/jtc1/sc22/wg21/docs/papers/2014/n4174.pdf)

Nim is, above all, a pragmatic language. I think that in another 5 or so
years, Nim's syntax choices #2, #3 and #4 will seem just as sensible as #1
seems to Python programmers today.

~~~
stewbrew
> Choice #3 is case-insensitivity for identifiers: You can write
> `a.to_lower()`, `a.toLower()` or `a.tolower()` interchangeably.

This looks like an enjoyable source for weird bugs: Write to functions with
similar names (e.g., to_me() and tome()) and be surprised when nim doesn't
distinguish between these two.

~~~
jboy
If your functions take any parameters, Nim will use the parameter types to
distinguish between them, just like C++ does when you overload functions.

OTOH, if your hypothetical is that programmer X writes `to_me()` while a
different programmer Y writes `tome()`, and the two functions just happen to
be identical in all parameter types... well, that can already happen anyway
where two programmers each independently write a function with the same name.

Nim has a simple, clear specification of how identifiers will be compared:
[http://nim-lang.org/docs/strutils.html#normalize,string](http://nim-
lang.org/docs/strutils.html#normalize,string)

There's no secret magic happening.

~~~
lomnakkus
It's doubtful that "convert it to lower case" (as documented for the function
you pointed to) actually means anything in this day and age, what with Unicode
and all...

Though, looking at the code for "normalize" it appears to only support ASCII.
That's _even worse_ since now "lowercase" doesn't even apply uniformly if your
identifiers have non-ASCII characters in them[0].

Doing a general normalization (as in Unicode) would probably also be bad, but
for different reasons: You probably don't want the validity of programs to
depend on the current locale of the machine you're compiling on. (See e.g. the
"Turkish I" problem.)

In short: Case-insensitive identifiers[1] are a terrible, terrible idea.

[0] Not generally a good idea, but it happens and there _are_ legitimate cases
for it.

[1] Or, rather, "doing anything non-trivial to identifiers before lookup", I
suppose.

------
justinsaccount
Looks neat, the tooling could use a little work though.

[http://docs.cython.org/src/quickstart/build.html](http://docs.cython.org/src/quickstart/build.html)
shows a few ways this could be integrated to be a little more... seamless.

~~~
fspeech
Using Cython for simple C++ binding is pretty great, esp. if one takes care to
use as much STL data structures at the API level as possible.

------
yzmtf2008
I can't even begin to describe how timely this project is to me. I'm working
on a research project that had chosen C++ over Python for performance reasons.
We started with boost and later decided to ditch it because, well, it's boost.

Later when we decided to provide a Python binding, I find myself in the same
place all over again -- we might have to use boost for the binding. Luckily I
haven't written anything yet and spotted this library on HN :)

~~~
entilzha
Similar situation here. Working on a research project that uses C++ for
MPI/RDMA/performance, but we would like to provide bindings in a more user
friendly language

~~~
lqdc13
Why not Swig2?

------
jordigh
As an experiment in doing something interesting with C++11, I attempted a
"literal translation" of an algorithm for the medcouple that I initially
prototyped in Python. I was very happy to see how remarkably easy this was:

[http://www.inversethought.com/hg/medcouple/file/default/medc...](http://www.inversethought.com/hg/medcouple/file/default/medcouple.py)

[http://www.inversethought.com/hg/medcouple/file/default/jmed...](http://www.inversethought.com/hg/medcouple/file/default/jmedcouple.c++)

Most Python idioms have an almost literal C++ translation. Even Python sugar
like a,b = v can be done with tie(a,b) = move(v). C++11 is an almost pleasant
language. If only it could completely break backwards compatibility and lose
all of the ugly parts... I suppose this is what D is.

------
opk
The rationale described there for pybind sums up my experiences with boost.
Now that much of the best parts of boost are available in C++11, boost ends up
being more trouble than it is worth. In general, I think it is best avoided if
you're writing software that you expect to still need supporting in a decade.
And you can always borrow ideas from it - we have our own lexical_cast like
template for example.

I also find from past experience that seamless interoperability is not always
a good thing because it obscures the interface boundaries. It can make it hard
to replace one side of the interface and can lead to programmers not familiar
with the full design of the software doing things inefficiently. I once worked
on a huge project using Corba and it wasn't always easy to know which calls
were remote.

~~~
paulddraper
Yes, anytime something makes it from a third-party library into the language,
it's not worth the hassle of a third-party dependency.

Of course, this "hassle" is off-loaded onto the language, ie all C++11 users
having to learn it.

------
daveguy
Very interesting. With numpy I haven't felt the need to C-ify anything in a
while, but making the process easier is always a plus (or plusplus). Does
anyone know how this compares with CFFI or Cython? Is there a hello pybind11?

~~~
kelvin0
[https://github.com/wjakob/pybind11/tree/master/example](https://github.com/wjakob/pybind11/tree/master/example)

------
ivan_ah
This looks very interesting. My friend has been telling me how C++ had
improved recently in terms of "pleasantness" to write, and now calling it from
Python makes it all the better:
[https://github.com/wjakob/pybind11/blob/master/docs/basics.r...](https://github.com/wjakob/pybind11/blob/master/docs/basics.rst#creating-
bindings-for-a-simple-function)

This bridge will be great for numerically-intensive projects like implementing
ML algos. I wonder how easy it would be to interop with scikit-learn data
structures from the C++ side.

------
sapek
Very interesting. We use Boost.Python to provide Python binding for Bond [0].
It works well but having something lighter weight would is great. One big
issue with Boost.Python is long compilation times. Does your library improve
on that?

[0]
[https://microsoft.github.io/bond/manual/bond_py.html](https://microsoft.github.io/bond/manual/bond_py.html)

~~~
bdash
The benchmark page of the pybind11 documentation [0] compares build times and
generated library size with Boost.Python.

[0]
[http://pybind11.readthedocs.org/en/latest/benchmark.html](http://pybind11.readthedocs.org/en/latest/benchmark.html)

------
santaclaus
Very cool, I'm a big fan of Wenzel Jakob's other major project, Mitsuba. Was
this developed to support Python bindings with Mitsuba?

~~~
wjakob
Yes, that's the plan! (eventually...) :)

------
dvdplm
Could Ruby have something thing like this too or is MRIs C api just too
limited? Rice is the closest thing I know of and it was pretty gnarly to use.

~~~
jameskilton
Maintainer of Rice here
([https://github.com/jasonroelofs/rice](https://github.com/jasonroelofs/rice)).
The problem set of trying to seamlessly integrate C++ and dynamic languages is
not a simple or trivial thing to accomplish. I know Rice isn't particularly
easy to use but it does have a lot of similarities to the OP's library and
Boost.Python API wise.

I'm curious what you found gnarly or difficult to use, maybe there are some
improvements that can be made?

Also as far as I know, Rice is the only library of its kind in the Ruby space
(ignoring SWIG, of course).

------
JulianWasTaken
(*CPython)

~~~
ant6n
no pypy?

~~~
opk
Does boost-python now support pypy?

