
Beautiful Native Libraries - obilgic
http://lucumr.pocoo.org/2013/8/18/beautiful-native-libraries/
======
angersock
This is an article I wish that every developer who writes libraries
(especially for use with games) in C/C++ would read. All of the advice about
cleaning up includes and exports and the whole thing is spot on, and is the
sort of thing that will cause me to curse your name a thousand times if you
forget it.

One minor point: if your library does file IO, let me override your file
operations with my own callbacks, or at least consider providing methods that
operate solely on blobs in memory (that way I can populate those blobs however
I see fit)--giving me a function that takes a const char* for the filename and
then does -~= _magic_ =~- is simply not okay.

~~~
masklinn
> One minor point: if your library does file IO, let me override your file
> operations with my own callbacks, or at least consider providing methods
> that operate solely on blobs in memory (that way I can populate those blobs
> however I see fit)--giving me a function that takes a const char* for the
> filename and then does -~=magic=~- is simply not okay.

Not only is that an excellent reminder, it's an excellent reminder for every
API in every language out there.

Though it's not necessarily fun, having to build a shim/layer over FILE* and
related functions all the time.

------
masklinn
> The reason for this is that Microsoft's C compiler is notoriously bad at
> receiving language updates and you would otherwise be stuck with C89.

And what Armin means by "notoriously bad at receiving language updates" is
that Microsoft has repeatedly and explicitly stated through Herb Stutter that
it does not want C, the C90 compiler is essentially an anomaly, it will remain
but it will not be updated to a more recent standard, and only the C++ subset
of standards more recent than C90 would ever be supported (by the C++
compiler, if you want pure C you're out of luck)

[http://herbsutter.com/2012/05/03/reader-qa-what-about-vc-
and...](http://herbsutter.com/2012/05/03/reader-qa-what-about-vc-and-c99/)

~~~
gliaskos
This year at build was reported that vs 2013 will have C99 support (not
complete).

[http://blogs.msdn.com/b/vcblog/archive/2013/07/19/c99-librar...](http://blogs.msdn.com/b/vcblog/archive/2013/07/19/c99-library-
support-in-visual-studio-2013.aspx)

~~~
masklinn
This is, as previously, support for the C standard library in its capacity as
a (qualified) subset of the C++ standard library.

VS will have no more C99 support that what was previously stated by Stutter:
it will support the common subset of C and C++.

~~~
plorkyeran
[http://blogs.msdn.com/b/somasegar/archive/2013/06/28/cpp-
con...](http://blogs.msdn.com/b/somasegar/archive/2013/06/28/cpp-conformance-
roadmap.aspx) lists some non-library C99 features not in C++.

------
huhtenberg

      YL_API void yl_set_allocators(void *(*f_malloc)(size_t),
                                    void *(*f_realloc)(void *, size_t),
                                    void (*f_free)(void *));
    

Technically, all you need is realloc(), because realloc(ptr, 0) is _free_ and
realloc(0, bytes) is _malloc_.

~~~
masklinn
> realloc(ptr, 0) is free

Not quite. POSIX states:

> If size is 0, either a null pointer or a unique pointer that can be
> successfully passed to free() shall be returned.

and although FreeBSD's man 3 realloc does not say _anything_ about a 0 size,
OpenBSD's man 3 realloc states:

> If size is zero and ptr is not a null pointer, the object it points to is
> freed and a new zero size object is returned.

And OSX's states:

> If size is zero and ptr is not NULL, a new, minimum sized object is
> allocated and the original object is freed.

So a minimal implementation would need realloc and free, to account for
realloc(ptr, 0) allocating.

~~~
huhtenberg
> _Not quite. POSIX_

Nitpicking without measure is a man's greatest pleasure ;)

You are not implementing POSIX. You are providing a hook for the library users
and _your library_ will call this hook with (ptr, 0) when it wants to free a
memory block. You just declare that your library works that way. Also, if your
library wants, it can test provided hook to see if it complies with library's
expected behavior and assert/fail if it doesn't.

Think about it this way - if you are exposing a way to override realloc()
calls in your library, then you _will_ have to specify what your library means
when it issues realloc(ptr, 0). So it's perfectly fine to declare that it
expects such call to do what free() does, in which case a single realloc hook
_is_ sufficient to override all memory operations.

~~~
masklinn
> Nitpicking without measure is a man's greatest pleasure ;)

Pointing out that your assertion is tragically wrong is not nitpicking.

> You are not implementing POSIX.

Of course not, you're not implementing an OS. You're _using_ POSIX.

> You are providing a hook for the library users and your library will call
> this hook with (ptr, 0) when it wants to free a memory block. You just
> declare that your library works that way.

This means it is not possible to pass through standard allocators to your
library without your user having to know precisely how they handle the realloc
case. This strikes me as a pretty significant annoyance for any user of said
library.

Oh, and your library can't rely on the platform's own allocators, so the user
_must_ provide an allocator for which he _must_ know how realloc precisely
behaves (and possibly wrap it) regardless of spec.

> Think about it this way - if you are exposing a way to override realloc()
> calls in your library, then you will have to specify what your library means
> when it issues realloc(ptr, 0).

If you stick with and expect POSIX semantics, as the name of the hook would
imply, you don't have to specify anything and you simplify your library user's
job.

And think about it this way: if what you want is not realloc, don't call it
realloc.

------
eliasmacpherson
Ulrich Drepper wrote a good guide on shared objects a long time ago and it is
more in depth and focused on linux.

[http://www.akkadia.org/drepper/dsohowto.pdf](http://www.akkadia.org/drepper/dsohowto.pdf)

------
adsche
Great read!

I have a question to section "Exporting a C API": All methods in class Task
are marked const. Yet only yl_task_is_pending in the C API takes a pointer to
const. Is there a reason for that?

~~~
the_mitsuhiko
No, just a stupid copy paste mistake. I fixed that, thanks.

~~~
adsche
Right, that makes more sense, could have guessed that tick() shouldn't be
const :)

(You missed one more detail though: In the implementation of
yl_task_get_result_string, it should be AS_CTYPE now.)

Another question, different topic, about the memory allocations: In your
example, why do you provide implementations for calloc and strdup instead of
letting them be set as well? I do agree with your stance on (not) dealing with
allocation failures in the library. But if I as a library user use (anything
like) standard malloc (and NULL pointer checks after e.g. yl_*_new()) I would
probably be annoyed at your implementations of calloc and strdup (which would
crash somewhere in your library when malloc returns NULL).

~~~
the_mitsuhiko
> why do you provide implementations for calloc and strdup instead of letting
> them be set as well.

For me personally it's because I never use calloc religiously. I implement
calloc so that I can forward those allocators to my dependencies (like curl).
I guess you have a point there.

With regards to strdup: My version changes the interface in that it's allowed
to pass NULL through it in my own code. I know I did not do that in the blog
post because I did not want to start a discussion about that API change :-)

~~~
adsche
Thanks for the answers. Great writeup altogether :-) test.py sounds
interesting, maybe I'll look into that some time.

------
pcunite
The sad state of affairs is that so many people assume native languages
(C/C++) to be so complicated when in fact it is the _environments and API_
that are commonly used with these languages that should bear the greater
blame.

I'm putting together a backend webserver and frankly, there is nothing
complicated about using C/C++ syntax over PHP or JavaScript. Using my own
objects and data abstractions is simple. However, as soon as I need to use
someone's library, that is when the complexity, confusion, and bugs begin.

I encourage anyone who wants to use system languages (for performance or
academic reasons) to take the time to find a good library (libuv on which
node.js is based for example). Program procedurally for a while to get a good
feel for things before endlessly wrapping logic in object or functional
abstractions.

Should you ever expose your API to others you'll do the world some good if you
make it enjoyable for others to use. Perhaps make two abstractions: one for
performance and one for simplicity.

------
rossy
> Obviously you could just use a different compiler on Windows but that causes
> a whole bunch of problems for the users of your library if they want to
> compile it themselves. Requiring a tool chain that is not native to the
> operating system is an easy way to alienate your developer audience.

I don't like this attitude. A lot of Windows developers like to pretend that
Windows only has one compiler and therefore doesn't support C99. Not only is
there MinGW GCC, there's also Pelles C and Clang which are all just as
"native" as MSVC (you don't need Cygwin or MSYS to use GCC or Clang.) As long
as you release a DLL and an import library, I don't think anyone will feel
alienated.

------
tomlu
Really solid advice throughout.

If you're using non-POD C++ and want to allow custom allocators (wise), I
would keep it simple and use placement new instead of messing about with per-
class operator new. If you must you can wrap it up in little internal template
functions to make it a little more palatable.

Eg:

    
    
      template <typename T>
      T *yl_new(Context *context) { 
          return new(context->malloc(sizeof(T)) T(); 
      }
      
      template <typename T>
      void yl_delete(Context *context, T *object) {
          object->~T();
          context->free(object);
      }

~~~
the_mitsuhiko
Then you have to stop using STL containers altogether as those will just use
operator new internally :(

~~~
tomlu
True, but that has more to do with wanting to place the allocators on a
context object. Even using custom allocators I don't think there's a way to
supply a non-global context to the STL.

------
crosvenir
If this is beautiful, I would hate to see ugly. I wish we could take the power
of C++ and just start over syntactically with a brand new language.

~~~
alipang
Why is everything beautiful after Steve Jobs? It's idiotic. Elegant, whatever,
but beautiful?

------
malkia
While xxx_API macroses are common, over the time I came to appreciate the .def
files more. It takes a bit more effort to keep them in sync, but then you no
longer need the xxx_API pattern, and you can ship both static and shared libs
this way.

For C++ interface though, .def files become an unreadble mess due to the
mangling, and in this case using the xxx_API is probably better choice

------
iam
Great post, very informative!

But why recommend changing operator new/delete by using a macro instead of
privately inheriting a class that does that?

------
jnbiche
Excellent article that I wish I had been able to read the first time I worked
on a C library last year.

And speaking of Rust's dynamic linking support, what is the current status of
Rust and shared object libraries? I think you can create them now if you
include the Rust runtime. Is this correct?

~~~
the_mitsuhiko
Rust can be used without a runtime now, but there are some shim functions that
need to be defined still. There is currently work being underway I believe to
make this easier.

------
winter_blue
It'd be good to see a guide on creating beautiful Python binding for native
C/C++ libraries.

