
Writing a website in Rust - viraptor
http://blog.viraptor.info/post/i-wrote-a-website-in-rust-and-lived-to-tell-the-tale
======
kybernetyk
Well, IMHO, "the right hammer for the right nail".

Rust is a great systems language (besides the lack of bitfields) but I
wouldn't use it as a web language. Just as much as I wouldn't use C++ to
develop webapps.

Sure you can do it but then again there are far easier ways to achieve your
goal.

~~~
mangeletti
Out of curiosity (I've never done systems programming), what are some of the
things that make a language more suited to systems programming rather than web
(other than libraries and community support - I'm more interested in the
intrinsic qualities of the language)?

~~~
OneMoreIdol
Outside of userland tasks like file manipulation, it is impractical to use the
popular web application languages like Java or scripting languages such as
Javascript or PHP or Python for systems programming without employing
interfaces or hooks to libraries or programs written in languages that are
compiled to machine code, simply because programs in those languages are
commonly deployed using virtual machines or interpreters that have only
userland privileges. So good luck writing a driver. And I don't know ... if
your vm has heap allocated to it, are you able to address system memory out
side of that to address hardware? Also, the language runtimes usually add a
lot of overhead as well and employ automatic garbage collection, which is
generally something you don't want in performance-critical systems.

~~~
justincormack
That is not strictly true. Look at Snabb switch, written almost entirely in
LuaJIT for example
[https://github.com/SnabbCo/snabbswitch](https://github.com/SnabbCo/snabbswitch)
\- you need some kind of ffi to mmap device memory, but it is quite possible.
(LuaJIT is fast of course which helps for 10Gb ethernet).

~~~
OneMoreIdol
That's kinda what I was thinking when I commented, and there are sure to be
exceptions. But I don't see anything different with your Snabb switch example.
You can probably use Java and JNI for that matter to call mmap, but would you
be able to write an implementation of mmap in LuaJIT ?

~~~
justincormack
Yes you can call mmap with the ffi in LuaJIT just fine. It is not very
complicated, it just asks the kernel to do it. Implementing what the kernel
does, well that does need some assembly.

~~~
OneMoreIdol
Indeed we agree. Using any ffi for that matter.

------
barosl
Whenever someone writes bad things™ about some language, one of the language's
fanboys shows up and starts nitpicking! Yes, now is the time!

> Another bad part is Rust’s JSON handling. It badly needs macros which make
> things easier.

Actually this is not the end of the world. As you mentioned, Rust's JSON
library supports `ToJson` for primitive types, but also provides _compile-
time_ code generation for arbitrary `struct`s. Quoting the code from my
project:[1]

    
    
      #[derive(RustcDecodable, RustcEncodable)]
      struct Msg {
          cmd: String,
    
          id: i32,
          x: Option<i32>,
          y: Option<i32>,
    
          speed: Option<i32>,
      }
    

What the mystical `#[derive]` thing does is to direct the compiler to create
the boilerplate for converting the `struct` to/from a string automatically.
So, now you can do this:

    
    
      // Decoding a JSON string
      let text = r#"{"cmd": "move", "x": 5, "y": 10, "speed": 200}"#;
      let decoded: Msg = json::decode(text);
    
      // Encoding a Msg object to JSON
      let msg = Msg { cmd: "new", x: 10, y: 20 };
      let encoded = json::encode(msg);
    

The `Msg` struct is full of `Option`s because my project allowed many fields
to be missing, but if the JSON messages in your protocol are in fairly similar
format, you can eliminate most of them.

One more benefit of this approach is that it type checks. If the JSON you
received is missing some fields that are not defined as `Option`, the decoding
process produces an error! So you can be certain that you're handling a valid
JSON after the decoding stage. This is analogous to schemaless vs. schema-
enforced database design.

> Compared to many static languages, the handlers look tidy. Compared to
> dynamic languages, they’re terrible.

This is the property of the Iron framework, not of the Rust language itself!
Another web framework for Rust, namely Nickel.rs[2], provides much simpler
APIs.

I believe the author of Iron is trying to establish essential things first,
and build more "user friendly" APIs on top of them. (FWIW, the Rust project
itself follows the similar strategy.)

[1] [https://github.com/barosl/pgr21-online-
server/blob/master/sr...](https://github.com/barosl/pgr21-online-
server/blob/master/src/server.rs)

[2] [http://nickel.rs/](http://nickel.rs/)

~~~
wtetzner
I haven't actually gotten a chance to use Rust yet, but would something like
this work?

    
    
        #[derive(RustcDecodable, RustcEncodable)]
        enum Msg {
            Basic { cmd: String, id: i32 },
            Positioned { cmd: String, id: i32, x: i32, y: i32 },
            Speed { cmd: String, id: i32, speed: i32 },
            Full { cmd: String, id: i32, x: i32, y: i32, speed: i32 }
        }
    

That would allow you to prevent weird cases like x being provided but y not
being provided.

~~~
drbawb
Yup, you can encode enums just fine. ADTs get encoded with an explicit
`variant` tag at the front, and then an array of field values. For e.g:

    
    
        { "variant": "Basic", fields: ["cmd-value", 42] }
        { "variant": "Positioned", fields: ["cmd-value", 42, 0, 0] }
        //etc ...

------
neverminder
> Actually the compiler checks cover most of the things I’d normally unit-
> test, so this is probably the only non-trivial project I wrote without
> checks and I’m OK with that.

This is interesting. First time I read about Rust I had this gut feeling that
it's design might reduce the number of unit testing cases. Someone care to
comment on that?

~~~
viraptor
Just to add some context, here are the tests which I didn't write, but would
in Python/Flask usually:

\- Proper behaviour if I don't pass a parameter. Not needed, because it's an
explicit `Option<>` which I _have_ to handle.

\- Is results list constructed properly / what's the None-vs-empty behaviour.
Not needed, `Vec<>` is verified at type level and None is not possible.

\- What happens if various database functions don't find a result. Not needed,
because type signatures force either a returned `Entity` (if it's not there,
it's an implementation bug: panic + 500), or `Option<Entity>` which again
needs to be explicitly handled.

But these are only unit tests. If it was a critical app, I'd still write
functional / logic tests to make sure the right data goes all the way to the
database and back in known scenarios. Types can't guarantee that.

One kind of tests I'm tempted to write is for templates rendering properly,
because handlebars-iron takes parameters as Json - all type guarantees go out
of the window there. But it may be the same amount of effort to migrate to
some type-safe templating like Maud.

~~~
s_kilk
I've had a similar experience writing web-apps in Scala.

The Option type alone handles so many cases which would otherwise require
unit-tests in Python or another unityped language.

~~~
Cyph0n
I'm a Python/Flask dev who has knowledge of the basics of Scala. Which web
framework and ORM would you recommend to develop a (potentially) non-trivial
web app?

~~~
neverminder
If you want to use Scala, Play Framework 2.4 + Slick 3.0 would be the
deadliest combination in my opinion.

~~~
s_kilk
Second, I'd certainly recommend play framework.

------
eigenrick
Thanks for the detailed write-up. You are a pioneer in the rust webdev space
and have created a map (with the dragons labeled) for others :)

~~~
viraptor
Haha, I didn't intend to go for detailed :) I'm pretty sure there's a lot more
to tell about the details. I just wanted to at least list all the modules I've
been looking for / using in this webapp. Thank you for the feedback.

I intend to write another post with some very basic skeleton / hello-world of
a new Iron application using database with connection pooling, templates,
logger, parameter parsing. But that's for another day.

~~~
anonbanker
Please do so, and tell HN. Many of us are paying attention.

------
CyberShadow
I think D has shown that you can get a nice web stack in a compiled systems
programming language:

[http://vibed.org/](http://vibed.org/)

Lots of packages for it on code.dlang.org as well.

------
nickpsecurity
Interesting project. If Rust is to replace C++, then we need to see it
exercised in all domains C++ has succeeded in. As I told another commenter,
C++ with good frameworks does very well in web applications albeit with fewer
libraries & utilities. A team of Rust programmers could catch Rust up to one
of the C++ web frameworks and do a shootout between the two on realistic web
apps. The results in terms of efficiency, productivity, and maintainability
for each should tell us plenty about Rust's success in terms of its
objectives.

That experiment among others, of course.

------
Manishearth
Regarding compile time, you can track the passes with `-Z time-passes` and
wait for borrowck/lints to get over (or just typeck if you're only worried
about types).

There also was [this
plugin]([http://www.reddit.com/r/rust/comments/2krdbu/rest_easy_a_lin...](http://www.reddit.com/r/rust/comments/2krdbu/rest_easy_a_lint_to_tell_you_when_compilation/)),
but it's outdated at the moment (I can upgrade it later).

Regarding imports, `use foo::bar::*` works.

~~~
viraptor
For use, I meant that there are so many packages you just take space just
listing them. For basic iron you need: iron::prelude::*, iron::status,
staticfile, logger, router, handlebars_iron, and more. And these are all from
different crates.

~~~
Jweb_Guru
This is the same in even many dynamic languages, though, isn't it?

~~~
viraptor
Almost. In Rust to use a trait, you have to explicitly import it. That means
if you want to run `.to_json()` on something, you need to import trait
`ToJson`. Then again if you want to force a type of something (for example an
empty hashmap) you need to import the types which are included.

So what in a dynamic language could be:

    
    
        a.to_json() if a else {}.to_json()
    

In rust will start with:

    
    
        use rustc_serialize::json::{ToJson,Json};
        use std::collections::BTreeMap;
        ...

------
bliti
Very nice write up. As a Go programmer, Rust is really interesting. Can't wait
until you can build web APIs with it.

~~~
steveklabnik
Crates.io is built this way: Rust serving JSON on the back end, Ember
consuming it on the front end.

~~~
bliti
How mature could this setup be considered? Thanks for pointing it out.

~~~
steveklabnik
As mature as anything for a language which just hit 1.0. Alex, the main
developer, said he likes it a lot.
[https://m.reddit.com/r/rust/comments/2v1fe3/hows_rust_workin...](https://m.reddit.com/r/rust/comments/2v1fe3/hows_rust_working_out_as_the_backend_for_cratesio/)

I think it's a pretty good architecture overall. We'll see how it all shakes
out.

------
malandrew
The tooling for the web written in JavaScript is excellent, so I was curious,
are there any options there are for calling JavaScript from Rust code? I know
there is the ffi module for NodeJS that allows you to call Rust from
JavaScript, but I was wondering if there is something that works well in the
other direction.

~~~
barosl
I once experimented this, and it worked. I wanted to do this because I needed
a decent and fast CommonMark library, but Rust didn't have one. (Yes, I tried
to embed JavaScript in Rust just to parse some MarkDown text!)

There are two libraries that embed V8 in Rust, but all of them seem to be
abandoned for now. However if there become enough interests, a new project
will arise I think.

(FWIW, I ended up calling CommonMark library written in Python from Rust,
because embedding CPython was easier than embedding V8. Unfortunately
CPython's performance is worse than V8, but as I know Python better than
JavaScript it was a reasonable choice.)

~~~
thristian
You know there's a CommonMark reference implementation in C, right?

[https://github.com/jgm/cmark](https://github.com/jgm/cmark)

I haven't tried it, but I would expect it would be easier to link with Rust
than an entire other language VM.

~~~
barosl
Oh, yes. I had also tried that, but found some convenient features missing,
e.g. line breaks treated as `<br>` (not two consecutive blank lines), GitHub-
flavored tables, and the strikethrough syntax. I'll have to wait until the
official CommonMark spec includes them.

Also the third-party CommonMark libraries are usually easier to extend, with
your own syntax addition.

------
ffn
Hey OP, have you tried the other "future" languages like go or Elixir? If so,
how does it compare with your experiences in rust?

~~~
viraptor
No, sorry. I did some erlang (directly, not elixir), but not for websites. I
really missed strict types and compiler validation there. I know there's
dializer and others, but it's just not the same level of comfort.

I've done one website in C however and can tell you that Rust is _way_ more
predictable ;-)

------
bishop_mandible
63 dependency crates? That's insane.

~~~
steveklabnik
Last I checked, a 'rails new' gives you over 40 dependencies off the bat.

~~~
bishop_mandible
Yes, the Rails and Node communities are known for insanity.

~~~
AgentME
I put all node functions I might want to re-use in other projects on NPM by
themselves (though this also usually pushes me to make a lot of unit tests for
each of them, which is a nice benefit). I'm not really clear on what's insane
about re-using a lot of small modular parts. This is way better than copying
and pasting functions between codebases, and then trying to keep them (and
their tests) all in sync.

------
ilaksh
Here's something that most of you will probably learn the hard way: popularity
and merit are not the same thing.

Rust is very popular now. That doesn't mean it has a great design. One of the
very first design decision/belief they made was "no code can perform with
garbage collection". Well, garbage collection does cause quite a few
performance problems, but that doesn't mean it can't work if you engineer it
right.

Personally I think Go and Nim are better languages as a general statement,
both for applications and systems programming.

I know I am going out on a limb making a negative statement like that, and
previously Nim developers have tried to discourage me from saying negative
things about Rust. Those guys are smart and have social skills and they
realize they should be careful to be nice to Rust developers because
sophisticated developers looking for things like performance, type safety,
etc. with a C or C++ background are really going to benefit from Nim if they
give it a chance and Rust developers are prime candidates for Nim conversion.

So let me just say clearly that I don't associate with the main Nim community,
the core developers, or anyone really. Nothing I say here reflects on them I
hope.

I literally have no friends in fact.

I don't understand people, or pay much attention to them, or interact with
them very much. When I do interact I say what I really think, not what people
want to hear.

What I DO understand is technology. From a very early age I have been
programming in everything from different types of assembly language to C to
C++ to Ruby, Javascript/CoffeeScript/ES7, different variants of SQL, Rust,
Forth, OCaml, etc.

Mozilla is a leading technology organization with many contributors. Rust is a
new technology with quite a few people innovating on it.

Unfortunately, Rust starts with a bad design decision and never really
recovers from that design decision. The values and perspective are lacking a
modern, contemporary perspective.

The Rust worldview is trapped in the C++ era.

I have been primarily a JavaScript developer both on the front and back end
for many years now. Why? Because I like to make useful applications and I am
sane and paying attention.

But in a world where people were better informed and had better judgement and
the best things won out rather than the most popular, Rust would be an obscure
language being toyed with by a few academics for (perhaps?) implementing
certain parts of kernels, the new browser from Mozilla would be fully peer-to-
peer capable (using IPFS/gittorrent/swarm/ndn etc), written in Nim of course,
use JIT/VM/something Nim for scripting, and have thrown out JavaScript/ES6/ES7
AND CSS for good. Of course, in this ideal world there would be no google
monopolizing all advertizing and capturing most good engineering talent, since
semantic markup and p2p query would also be be built in to this browser,
making google irrelevant. The default mode for the browser would be virtual
reality, with the ability to render 2d operating systems or markdown/images on
arbitrary 3d surfaces.

But instead we have the world we have. All advertising must follow the
dictates of a giant all-powerful global corporation. Mozilla is trapped in a
pseudo-C++ mindset and spending most of its efforts trying to reproduce the
intractable mess of decades of CSS hacks, in the time the engineers have left
after going through in excruciating detail all of the possible ways to
'borrow' memory. CSS, a brilliant system that is a pain the in the ass not
only for programmers but also designers, so complex that only two computer
programs in the world are known to do a reasonable job of rendering it
accurately.

Its time to stop dragging the old tools, mindsets, and technical debt forward.
Stop judging things on the basis of momentum or authority
(Google/Mozilla/Microsoft) and start using your brain to select things
rationally.

Mozilla as an organization is not going to be capable of admitting they made a
mistake with Servo/Rust and recoding it in Nim. Just like the world is not
going to accept that we should throw out CSS and use any other simple system
that can reproduce graphic designs. And we are not getting rid of JavaScript
with its horrible threading model and garbage collection anytime. But we
should. In a sane world we would learn from our mistakes and throw all of that
out.

~~~
viraptor
If you're going to write a block of text like that, you could at least fact
check the main premise. Rust had GC. They started off with GC, so it wasn't
the very first decision that "no code can perform with garbage collection".

Here's a post about removing GC from core rust 2 years ago:
[http://pcwalton.github.io/blog/2013/06/02/removing-
garbage-c...](http://pcwalton.github.io/blog/2013/06/02/removing-garbage-
collection-from-the-rust-language/)

Next: they didn't kill GC. They removed it from core, because rust is capable
of having gc implemented as a library. Standard library still has two gc-
enabled pointers - Rc and Arc. You can use them in current code to have
garbage collected values.

~~~
filsmick
Rc and Arc aren't exactly GC types... they are just reference-counted
pointers, Rc being akin to C++'s shared_ptr. I get that in a sense, this is
garbage collection, but certainly not full-featured like the ones you find in
other languages.

~~~
pcwalton
You can use full GC in Rust--we use the SpiderMonkey GC to collect Rust DOM
objects, for example. It's not the most easy-to-use thing, however.

Most systems software gets by fine with a combination of thread-safe and
thread-local RC. Reference counting is a form of garbage collection that works
really well when it's used only for the subset of data that needs GC--which is
the style that Rust encourages anyhow.

~~~
ilaksh
Hmm.. Rust was created to support the new browser, browsers mainly manage DOM
objects.. I wonder why you are using a legacy garbage collector in order to
handle primary tasks rather than having that part of your core design?

~~~
viraptor
Spidermonkey uses generational gc with some kind of compacting these days. Why
do you think that's legacy? What models are strictly better than that?

~~~
ilaksh
Nim's garbage collector is better than Spidermonkey's.

~~~
Manishearth
Irrelevant here.

For Servo we need a javascript engine. We're already using Spidermonkey. It
has a GC for Javascript; we're already paying those costs. The Rust-side
representation of DOM objects is also managed by the GC; that makes sense
because these are tied to Javascript things.

We don't use the GC elsewhere. I think at some point we did, but the only
place now in Servo where GC is used is where the data is strongly connected to
data already managed by the SM GC.

Nim's GC is for general-purpose use in a language. Spidermonkey's GC is for
GCing javascript, which already has an extensive runtime (which the GC ties
into heavily). "Nim's GC is better than Spidermonkey's" is a statement of no
value (and oversimplifies the situation) unless the context is specified.
Using the SM GC to collect random Rust objects would be a bad idea. Somehow
rigging up spidermonkey to use a Nim-like GC in Rust (all other things being
the same) would also be a bad idea. Two different scenarios, two different
GCs.

