
Rustler is a library for writing Erlang NIFs in safe Rust code - bryanrasmussen
https://github.com/rusterlium/rustler
======
hinkley
I've been trying to decide between learning Elixir and learning Rust and
Elixir had a narrow lead because of employment potential and my growing
dissatisfaction with Docker (container orchestration contains an ad-hoc,
informally-specified, bug-ridden, slow implementation of half of Erlang).

Then I learned that the Achilles' heel of the BEAM VM is that if native code
crashes it takes the whole thing with it, so people have been using Rust for
native code (NIFs). So now the plan is Elixir _then_ Rust which is easing my
FOMO quite a bit.

~~~
davidw
> Then I learned that the Achilles' heel of the BEAM VM is that if native code
> crashes it takes the whole thing with it,

That's not really an 'Achilles heel' any more than most other languages that
allow you to load C code that could segfault or - maybe even worse - get stuck
in an infinite loop.

Indeed, the fact that BEAM focuses on and acknowledges this is an advantage
for people trying to create robust systems. They've had workarounds in place
for years, like running stuff in a separate system process.

~~~
andy_ppp
This is actually the point of Rustler, rust can make guarantees about crashes
and as such you can write safe low level code that never crashes the VM.

~~~
davidw
They ought to point out the notion of dirty NIFs a bit more prominently:
[https://bgmarx.com/2018/08/15/using-dirty-schedulers-with-
ru...](https://bgmarx.com/2018/08/15/using-dirty-schedulers-with-rustler/)

Not crashing is good, but not sufficient to play nicely with the BEAM
scheduler. You need to also ensure you're not using up a lot of time in some
long-running operation, unless it's a "dirty NIF".

~~~
fortran77
Yes. I write NIFs all the time, and it isn't crashing that's the problem for
me (I'm a good programmer!) -- it's taking too much time. We have a NIF that
does CUDA computations and we have to make sure we break them up right. Rust
won't help me on this. I need C++ for CUDA.

~~~
dnautics
1) I'm helping out with a project to bring some nice deep learning semantics
to the BEAM. Can you share the CUDA nifs?

2) I'm considering taking up Zig as a better (well simpler, because my poor
brain can't handle rust) safe language for NIFs. Do you think that zig might
make a better fit as it can easily integrate C headers into itself?

3) are dirty nifs not ok?

~~~
fortran77
We do plan to make the NIFs open source! Keep searching on github.

We use ours for signal processing, not "deep learning" so it may not be
completely applicable.

------
steveklabnik
For an example of this library used in production:
[https://blog.discordapp.com/using-rust-to-scale-elixir-
for-1...](https://blog.discordapp.com/using-rust-to-scale-elixir-
for-11-million-concurrent-users-c6f19fc029d3)

------
skosch
Related: I've found that to run time-consuming number-crunching code from
Elixir, the most convenient approach is to call a Python script via ports.
It's super easy and can't crash your VM.

To convert from/to Erlang terms in your Python script, you can use a library
like [https://github.com/skosch/python-
erlastic](https://github.com/skosch/python-erlastic) (a fork I maintain, the
original project was abandoned).

------
davidw

        fn add<'a>(env: Env<'a>, args: &[Term<'a>]) -> Result<Term<'a>, Error> {
    

As someone not familiar with Rust, I'm curious what all the symbols there
mean. What it's doing is of course obvious, just wondering about the details.

~~~
saghm
`'a` is a "lifetime parameter"; syntactically, it's like a generic parameter,
only it's use to denote how long a given reference stays alive. This is one of
the important mechanisms that Rust uses to ensure memory safety without a GC.
Every reference in Rust has a lifetime, but many of them can be elided.

`fn add<'a>` just means `define a function with a lifetime parameter called
`'a`. This is analogous to if you defined a function as `fn add<T>` with a
generic parameter T.

`Env<'a>` means that the type `Env` is parametrized by a lifetime. As before,
this is similar to a type being parametrized by a generic parameter. This is
usually done when a type contains a reference; Rust requires that references
stored in a type must have explicit lifetimes specified. In this case, we're
specifying that the lifetime that Env is parametrized over is the lifetime
parameter that the function defines. This doesn't mean anything in particular
on it's own, but when we use `'a` elsewhere in the signature, it means that
the two places use the same lifetime, which is necessary to specify sometimes
to clarify to the compiler that something is safe.

`&[Term<'a>]` is a slice of `Term<'a>` values; any type can go between the
`[]`, e.g. `&[i32]` for a slice of 32-bit integers. A slice is a view into a
sequence of values in memory. One of the things Rust does to aid the
programmer in low-level optimizations is define separate types for sequences,
namely slices, arrays, and vectors. A slice is a reference to another sequence
type (which could be another slice); they're very cheap to make, but cannot be
resized. An array is a value type (i.e. not a reference) and must be constant-
sized. Finally, a `Vec` is resizable and heap-allocated, so it's expensive to
create and copy (relative to slices).

Finally, `Result<Term<'a>, Error>` means that the function returns a `Result`
(i.e. either a success or an error); if no error occurs, it will return a
`Term<'a>; otherwise, it will return the type `Error` (which is generally
custom-defined in a given package, but could be an instance of a standard
library error like `std::io::Error`. This is roughly analogous to returning a
`Term<'a>` and marking that an exception of type `Error` may be thrown.

One thing to note is that Rust's return values with lifetimes tend be tied to
one of the parameters. In this case, the compiler's rules aren't sufficient to
infer which of `Env` and `Term` to tie it to, so the programmer needs to
disambiguate. The programmer chose to tie them all together, which presumably
indicates that the output will reference both parameters in some way (although
it's impossible to figure out more precisely how it will from the signature
alone).

Lifetimes are definitely the hardest part of Rust in my opinion; the important
thing to realize is that changing the lifetime parameters in a function will
never change the user-visible semantics of the code. If the lifetimes are
invalid, the compiler will error, and if they are valid but not optimal, then
the worst case is that that memory might be kept around longer than needed.
Unless you're programming something that is super performance critical or will
be long running, you generally won't have huge issues if you fix the lifetime
compiler errors with trial and error when they come up (which is much less
often than you might expect due to the compiler being able to infer them most
of the time).

~~~
cesarb
> the important thing to realize is that changing the lifetime parameters in a
> function will never change the user-visible semantics of the code

To reiterate: the lifetime parameters are used only to check the code for
errors, they are discarded after that and don't affect the code generation.
There's even an alternative compiler for Rust (mrustc) which completely
ignores them and the rest of the borrow checking stuff, assuming the code is
correct.

------
algaeontoast
Does anyone know of an example of a rust library or function that’s been
turned into a NIF? Ideally available on GitHub both before and after the
conversion?

Also, the “nerves” framework is basically 80% NIFS of embedded C code.

~~~
elcritch
Nerves overall has a much smaller portion of C nifs than 80%, probably < 10%.
Mainly for properly trapping and cleaning up subprocesses. There’s some nifs
too setup Linux specific sockets/ioctls. Still surprises me that almost every
Linux kernel interface essentially overloads `ioctl`.

Not sure of an example. Really just wrap any rust function with appropriate
code to convert Erlang terms to rust and back... There’s a few elixir rustler
howtos: [https://medium.com/@jacob.lerche/writing-rust-nifs-for-
your-...](https://medium.com/@jacob.lerche/writing-rust-nifs-for-your-elixir-
code-with-the-rustler-package-d884a7c0dbe3)

------
h91wka
As a professional Erlang developer I don't trust libraries that are heavy on
NIFs, and Rust isn't going to change this. Key feature of BEAM VM is process
preemption. It makes NIF development harder than it looks, and typically
performance of NIF-based libraries looks best in synthetic benchmarks.

