
Neon: Node plus Rust - aturon
http://calculist.org/blog/2015/12/23/neon-node-rust/
======
cmrx64
Wow! The integration with rayon for data parallelism is especially
interesting, and I imagine that could be a killer feature for some people
looking to accelerate compute-heavy node.js code without having to deal with
the nastiness of parallelism in C++, or C++ at all.

~~~
kibwen
I don't believe the blog post announcing Rayon ever made it to HN, so here's a
convenient link:
[http://smallcultfollowing.com/babysteps/blog/2015/12/18/rayo...](http://smallcultfollowing.com/babysteps/blog/2015/12/18/rayon-
data-parallelism-in-rust/)

~~~
paulmlewis
Only a handful of comments but it was posted a few days ago:
[https://news.ycombinator.com/item?id=10759434](https://news.ycombinator.com/item?id=10759434)

------
dalailambda
I can see this being useful in combination with Piston[1] to achieve scripting
for game engines, in the same way Lua is normally used. Rust as a core for a
game engine seems like a really great idea, considering the speed and safety
benefits it provides.

[1]
[https://github.com/PistonDevelopers/piston](https://github.com/PistonDevelopers/piston)

~~~
ZoFreX
Javascript seems like a great contender in this space to me. Btw, as well as
using nodejs / v8, there are also Rust bindings for SpiderMonkey (Mozilla's JS
engine): [https://github.com/servo/rust-mozjs](https://github.com/servo/rust-
mozjs)

(not super user-friendly yet, as this mainly exists for Servo - but there's
enough here to embed Javascript scripts in your Rust projects!)

------
steveklabnik
I've been very excited to see this project develop. I think it really speaks
to what we can do with Rust in the future: once we get some nice abstractions,
"FFI" barely even feels like it.

Node is actually the worst case here: V8 being in C++ makes this harder than
Ruby or Python, being implemented in C.

~~~
rubiquity
Did Yehuda ever open source or complete a similar abstraction like this but
for Ruby? I remember someone mentioning to me he was working on it. I have a
longstanding todo (though I have no motivation because I currently have no use
for Rust or Ruby) to try something similar though I remember my initial
attempts having trouble with how Ruby's C API expresses some values.

~~~
wycats
I'm actively working on it
([https://github.com/tildeio/turboruby](https://github.com/tildeio/turboruby))
and in fact was racing @dherman to finish. He beat me by a mile :( but I hope
to wrap up a usable first version soon.

~~~
dherman
Eat my dust! :P

------
Udo
There's an API thing from the JS side I don't understand, and I didn't find
anything documentation-wise (maybe it's just because I haven't had my coffee
yet):

    
    
      require('neon-bridge').load()
    

So this loads... which rs file, exactly? Where do I specify the Rust file to
be loaded? Or is there just a single entry point and you're supposed to have
one singular Rust module that is named by convention?

~~~
dherman
Rust is a compiled language, so the JS files don't directly load a .rs file
but instead they load a native Node extension, i.e. a .node file, which is
generated by compiling and linking the Rust sources. (The node-cli tooling
automates the whole build process.)

The line you cited is automatically locating the .node file, loading it, and
returning the module object associated with it.

On IRC a couple people have been questioning this part of the API, suggesting
that just using the standard npm module `bindings` might be better. It's a
reasonable idea and I'll probably open a discussion in a GitHub issue.

~~~
Udo
I realize that Rust is a compiled language, that's not the issue. The syntax
somehow suggested to me the modules could be compiled on demand, but that is
not the issue either.

My question is/was, given two or more modules written in Rust, how do I
address and require them by name? Due to its naming I assumed "neon-bridge"
would be the glue that handles this. Given the fact that you can require .node
files directly, it was not clear to me that "neon-bridge" is just the name of
an example module ( _or is it?_ ).

~~~
dherman
Ah, I see!

So neon-bridge is a utility npm package I published that automates loading
your Neon project (so it's not the name of an example module). And the
expectation is that you only have one Rust package ("crate" in Rust parlance)
per Node package, which is why you don't need to pass it the name of anything
-- it's looking for _the_ native module in the package, as opposed to _a_
native module.

However, that single Rust crate may have multiple modules in its
implementation. They live in the src/ directory of your Neon project and are
picked up by the Rust compiler and build tool.

And yes, deeeeefinitely need to get some docs going... :P

~~~
Udo
Cool, thanks for clearing that up. And thanks for working on this, I know with
the questions it can sometimes seem like it's not appreciated... :)

~~~
dherman
Not at all -- my goal is simplicity and clarity so a confused (potential) user
is a bug report!

------
KaterKarlo56
Honest question: Why would someone use Node instead of a Rust web framework
when writing part of the code in Rust anyway?

~~~
Filligree
\- Node has full garbage collection. Rust has some very clever memory-
handling, but it's not quite the same.

\- Node is javascript. You may want to run the same code on client and server,
and it's simpler than compiling Rust to JS.

\- There may be JS libraries you want to use.

I could go on.

~~~
vpkaihla
> Node is javascript. You may want to run the same code on client and server,
> and it's simpler than compiling Rust to JS.

What would be the reason to do that?

~~~
nailer
You need to do the same thing on the client and the server.

\- DOM manipulation for AJAX states so they can be reached directly from the
server without needing client side JS to do anything

\- Calculate discounts for products (obviously clients just POST the product
they want, but the discounts should be applied on the server consistently to
how they were shown to the user)

\- Your code is something generic, like recursively checking for a keypath on
an object, and totally useful for JS anywhere it runs.

All kinds of other reasons.

------
BinaryIdiot
This is fantastic! Great idea and job so far! One of the biggest disadvantages
with node is any type of complex manipulations / processing due to the single
threaded, scheduler paradigm but combining node with native modules makes it
much easier to pass off more complex pieces directly to C++.

Passing off to rust? Count me in! I'll be taking a look at this when linux is
supported (would also love windows support but mac os x and linux are
necessary for me).

~~~
steveklabnik

      >  I'll be taking a look at this when linux is supported 
    

[https://github.com/dherman/neon/pull/22](https://github.com/dherman/neon/pull/22)

[https://github.com/dherman/neon-cli/pull/6](https://github.com/dherman/neon-
cli/pull/6)

~~~
BinaryIdiot
I just saw :D

Damn people work fast. I hadn't even looked into what it would take to support
Linux (and honestly my rust knowledge at the moment is so limited I probably
wouldn't have figured it out in a reasonable time frame).

------
rvikmanis
Could be killer if combined with electron.

~~~
derefr
If the goal is having a fast "business-rules engine" coupled to a pretty
HTML5-based app-wrapper, one of my own projects is to combine Electron with
the Erlang VM.

Rather than trying to go whole-hog on linking the two worlds together at a
process level, I've just embedded an Erlang release into the Electron package,
which decompresses itself to the user's Library/APPDATA dir on Electron
startup, transparently registers as a service (so it can continue to run for
things like server sync when the app is quit), and then communicates with the
app using Unix domain sockets.

It's a bit like the feeling of having a Service Worker, but one that just
happens to be able to embed things like physics engines inside of it.

~~~
mmastrac
I'm doing the same thing with a JVM. I had to fix the bug where named pipes on
Windows were totally busted in Electron before I could make any progress.

~~~
derefr
Ah, I ran into that problem (I think?) and had been just using regular IP
sockets as a stop-gap, planning to switch back over after it was fixed. I'm
glad to hear someone with knowledge of Electron internals ran into it as well;
I feel bad that I didn't file it as a bug, though.

~~~
mmastrac
If you're curious, the fix was here:
[https://github.com/atom/electron/issues/1968](https://github.com/atom/electron/issues/1968)

It made it out into the regular release and I think it's fixed but TBH I
haven't tested for a few months.

My work is all on GH here as well:
[https://github.com/orbitaljs](https://github.com/orbitaljs) There might be
some synergy here between the projects.

------
netcraft
I wonder what makes it osx only? very cool project though, hope it takes off.
I wish there was a way on github to watch a project at a level where you could
just get notified of new releases.

~~~
steveklabnik
Only that Dave only has access to OS X. I plan on contributing Linux support
if nobody else beats me to it.

~~~
imrehg
I'll be very happy to test on Linux (don't think I have the skill to create it
just yet:)

~~~
dherman
Watching Christmas movies with the family right now but I'll file a bug with
some hints later. :)

~~~
gmtgmt
This is a nice Christmas present to the Node.js+Rust community. Thank you.

------
arthursilva
Or even better

line.split(' ').filter(|word| matches(word, search)).count()

~~~
sremani
I love the functional programming style embraced by Rust.

------
vvanders
That's really, really slick.

------
gbuk2013
Does anyone know what is the difference between using this neon vs using node-
ffi[1]?

[1][https://github.com/node-ffi/node-ffi](https://github.com/node-ffi/node-
ffi)

~~~
blaenk
Neon is for creating native Node modules in rust, which are libraries that use
the Node addon API [1].

node-ffi seems to be for adding a bridge between an existing non-Node module
dynamic library and node, which at the very least seems to incur some non-
negligible overhead; see the "Call Overhead" section of your link.

[1]: [https://nodejs.org/api/addons.html](https://nodejs.org/api/addons.html)

------
vkjv
Since both node and Rust use libuv for parallelism. Is it possible to provide
the node libuv pool to rust so that they share a single bound on parallel
operations?

Also, can I use Neon for async callbacks?

~~~
Matthias247
Rusts std lib does not utilize any longer. It only used in in some pre 1.0
versions when green threads were still a thing.

------
rictic
This is really cool. I'd love to see a system whereby pure computation modules
were also supported in the browser via web assembly.

~~~
pcwalton
Me too! :)

The possibilities of rustc (which has an LLVM backend) and Emscripten (which
consumes LLVM IR) are really interesting.

~~~
dherman
Agreed! In fact it would also be useful in Node: you could build an almost-
full-performance but portable version of a native module as a fallback for
clients who don't have the requisite software to build a Neon library (say,
one they only indirectly depend on) on their machine.

~~~
cmrx64
And, before you know it, we'll have realized the dream of Urbit, in JS ;)

Always bet on JS!

------
mcs_
nice to have access to rust libs from node.

------
advanderveer
Would this allow Rust to be run on AWS Lambda?

~~~
dikaiosune
I don't know the answer to how Neon would affect that, but AFAIK, you can
already do this:

[http://julienblanchard.com/2015/rust-on-aws-
lambda/](http://julienblanchard.com/2015/rust-on-aws-lambda/)

But that uses a shell command. There's also this way to do so using FFI
(article is about C++, but I'm pretty sure you could do the same with Rust):

[http://benchling.engineering/crispr-aws-
lambda/](http://benchling.engineering/crispr-aws-lambda/)

------
chrisdew
Many thanks for writing this.

------
smegel
Node + Rust ~= Go

~~~
__david__
Not even close. The expressiveness of rust alone far outclasses go in my
opinion. Add in the node ecosystem and it's in a different ballpark
altogether.

