
Neat Rust Tricks: Passing Closures to C - rabidferret
https://blog.seantheprogrammer.com/neat-rust-tricks-passing-rust-closures-to-c
======
thenewwazoo
This is a really well-written article about a common way to trampoline Rust
methods into C embedded RTOS tasks, which often work in exactly this way. I've
done just this in the service of getting ChibiOS and uC/OS II running Rust
code.

------
cheez
Is this a neat trick or just standard operating procedure for calling C from
<your favorite lang>? As it was billed as a trick, I was expecting some sort
of runtime code generation to pass the data pointer and some jump instruction
to jump to the right spot and unpack the data pointer.

Maybe I just overcomplicate things ;-)

~~~
DougBTX
I’d say that “standard procedure” would be to do it the same way as it would
be done in C: define a struct, allocate one somewhere, then pass a pointer to
it as the data pointer. Using the anonymous struct which represents the
closure itself seems like skipping a step, the user doesn’t need to spell out
which values are stored in the struct.

~~~
zozbot234
If the language supports closures which capture variables from their
surrounding environment, there's no way around using "the closure itself" as
your data object. After all, "the user" is not expected to "know" what any
given closure is capturing from the environment; part of the point of closures
is implementing a sort of information hiding.

------
KenanSulayman
Interestingly this is very similar to how I implemented passing closures into
JavaScriptCore as hooks for JS class invocations ("function calls"). [0]

Essentially it's taking advantage of the fact that closures are static methods
with "implicit" data pointers. It should be fairly obvious that this is a
massive violation of safety and undefined behavior, and most likely to break
when debugging symbols etc. are inserted.

The safest way to do this until Rust has figured out a stable-enough-ABI for
closure passing would be a thread-local trampoline, I guess. Not very nice..

[0] [https://github.com/psychonautwiki/rust-
ul/blob/master/src/he...](https://github.com/psychonautwiki/rust-
ul/blob/master/src/helpers_internal.rs#L67-L118)

~~~
doomrobo
Where's the UB? It casts a boxed closure to a raw pointer, and then back to a
boxed closure. There's no tricky introspection being done here.

~~~
KenanSulayman
I'm not entirely sure you read the code I'm referring to. There's no box
there.

------
kazinator
You can pass closures to C as C functions in TXR Lisp, a language I created.

Example:

[http://rosettacode.org/wiki/Window_creation#Win32.2FWin64](http://rosettacode.org/wiki/Window_creation#Win32.2FWin64)

In this program, a translation of Microsoft's "Your First Windows Program"
from MSDN, _defun_ is used to define a WindowsProc callback. _defun_ generates
a _lambda_ under the hood, which carries a lexical scope.

The lambda is passed directly to Win32 as a callback, which is nicely called
for repainting the window. (Or at least, things appear that way to the
programmer.)

Setting this up requires a few steps. We need a target function, of course,
which can be any callable object.

Then there is this incantation:

    
    
      (deffi-cb wndproc-fn LRESULT (HWND UINT LPARAM WPARAM))
    

The _deffi-cb_ operator takes a name and some type specifications: return type
and parameters. The name is defined as a function; so here we get a function
called _wndproc-fn_. This function is a converter. If we pass it a Lisp
function, it gives back a FFI closure object.

Then in the program, we instantiate this closure object, and stick it into the
WNDPROC structure as required by the Windows API. Here we use the above
wndproc-fn converter to obtain WindowProc in the shape of a FFI closure:

    
    
      (let* ((hInstance (GetModuleHandle nil))
             (wc (new WNDCLASS
                      lpfnWndProc [wndproc-fn WindowProc]
             ...
    

The lpfnWndProc member of the WNDCLASS FFI structure is defined as having the
FFI type _closure_ ; that will correspond to a function pointer on the C side.
The rest is just Windows:

    
    
      (RegisterClass wc)
     

register the class, and then CreateWindow with that class by name and so on.

~~~
kazinator
Here is another example of callbacks at work from the TXR Lisp test suite:
using the C library funtion _qsort_ to sort a Lisp array of strings.

[http://www.kylheku.com/cgit/txr/tree/tests/017/qsort.tl](http://www.kylheku.com/cgit/txr/tree/tests/017/qsort.tl)

It's done in two ways, as UTF-8 char * strings and as wchar_t * strings.

What's used as the callback is the function cmp-str which is in TXR Lisp's
standard library. A lambda expression could be used instead.

Also tested is the perpetration of a non-local control transfer out of the
callback, instead of the normal return. This properly cleans up the temporary
memory allocated for the string conversions.

~~~
cellularmitosis
TXR looks interesting. Is there a project README?

~~~
kazinator
There is a boring and poorly maintained home page:
[http://www.nongnu.org/txr](http://www.nongnu.org/txr).

And big honkin' manual.

------
jgtrosh
How does this relate to nested functions in C? (And resulting “infectious
executable stacks”?)

[https://nullprogram.com/blog/2019/11/15/](https://nullprogram.com/blog/2019/11/15/)

~~~
richardwhiuk
The problems there don't apply I believe because Rust closures don't require
an executable stack.

~~~
mmastrac
That's correct - a Rust closure generally [1] can't be converted to a function
pointer as it requires both code and state.

[1] [https://github.com/rust-lang/rust/issues/39817](https://github.com/rust-
lang/rust/issues/39817)

~~~
twic
The whole point of jgtrosh's link is that there is a way to hide data behind a
function pointer, so Rust _could_ convert any closure to a function pointer.
But it requires writable-and-executable memory, so it's a pretty bad idea (in
GCC's implementation, that memory is on the stack, which is an extra bad idea,
but i don't think it needs to be).

~~~
mmastrac
Definitely.

Technically this can also be done via static code trampolines that are mmap'd
as well [1]. That approach has been used on iOS in the past to turn blocks
into raw function pointers.

If you have a platform that allows W+X on code (yikes!), you can do [2] as
well.

[1]
[https://github.com/plausiblelabs/plblockimp/blob/master/Sour...](https://github.com/plausiblelabs/plblockimp/blob/master/Source/trampoline_table.c)
[2] [https://www.mikeash.com/pyblog/friday-
qa-2010-02-12-trampoli...](https://www.mikeash.com/pyblog/friday-
qa-2010-02-12-trampolining-blocks-with-mutable-code.html)

~~~
rabidferret
Anything that doesn't require W+X would need an entire page allocated per
closure, wouldn't it?

~~~
devit
No, you can of course allocate W+X pages from the OS and put multiple closures
in them using a standard userspace memory allocator.

Or if the OS doesn't support W+X allocation at all, then you can have a bunch
of tightly packed pregenerated trampolines in the binary.

~~~
saagarjha
Right, this is how Objective-C's implementation works, except it keeps around
one page of trampolines and remaps that around when necessary to be able to
"create" more trampolines on the fly, I believe.

------
tedunangst
Now call qsort with a closure.

~~~
rabidferret
`void ( _)(void_ , everything_else)` is in my personal experience a much more
common API than that of `qsort` (probably for exactly the point you're
pedantically trying to make), so I chose to focus on that API for this
article.

There's really no reason to pass a rust closure to `qsort` instead of sorting
in Rust. That said, if there's demand for real world use cases that require
passing Rust closures to C APIs that take only a function pointer and not a
data pointer, I'll be happy to write a follow up.

~~~
kazinator
In any decent language, all functions are first-class, so if you want to use
them as callbacks, you need that to work.

That's still true even if the API takes a separate context pointer that is
given to your function as an argument.

There is still a function pointer there, and what you'd like to use as a
function pointer is a function in your local language, and that's an object
with an environment. Even if some instances of it have no environment, the FFI
mechanism might want to tack on its own. For instance, the FFI needs to be
able to route the callback to the appropriate function. Whatever function
pointer FFI gives to the C function, when the C library calls that function,
FFI has to dispatch it back to the original high level function. That requires
context. Now that context could be shoehorned into that context parameter, but
it could be inconvenient to do so; that parameter belongs to the program and
to the conversation that program is having with the C API.

~~~
comex
Generating native function pointers on the fly:

\- is inherently slow, because CPUs have separate data and instruction caches;

\- is extra slow in practice because you need a separate allocation for
executable memory (unless your stacks and heap are RWX, which is a terrible
idea);

\- is not portable, requiring architecture- and OS-specific code; and

\- is not supported at all in many environments (of varying levels of
braindeadness).

For a statically compiled language like Rust, it makes much more sense to use
the context pointer.

------
dmitrygr
> If you’re not familiar with C’s syntax, here’s the equivalent signature in
> Rust

Author is hilarious. Who is familiar with that but not c?

~~~
alkonaut
I came to Rust without writing C before. Most of my experience with C comes
from problems exactly like this. I doubt I'm alone in this.

------
jonny383
Rust is already doomed. The amount of literature being published about either
comparisons or compatibility with C is a strong indicator C is here to stay.

~~~
cellularmitosis
If Rust is intended to replace C, wouldn't you expect lots of this sort of
literature? i.e. isn't this actually a sign of its _success_?

~~~
steveklabnik
Also, being able to add Rust to an existing C or C++ codebase was an important
design consideration. Big projects like Firefox aren’t just going to re-write
millions of lines of code all at once.

~~~
The_rationalist
And this is why rust is not going to succeed. I has not a great compatibility
with c++.

