
Diesel: A Safe, Extensible ORM and Query Builder for Rust - steveklabnik
http://diesel.rs/
======
Animats
Why is

    
    
        Post::belonging_to(user).load(&connection)
    

an improvement over

    
    
        SELECT * FROM posts WHERE user_id = 1;
    

?

If you use this, it nails the database schema into your Rust code, because
this needs to know the table formats. If someone adds a new field to a row,
you have to rebuild all your Rust programs that use the database. Now your
deployment process just got much more complicated.

It's a neat hack, but seems an overly complex way to do something that's not
that complicated. Unless this has some huge performance gain over rust-
postgres[1], it's probably too much.

If rust-postgres is too slow, it would be worth having an optional generic for
rust-postgres to help with unmarshalling returned rows. Rust-postgres returns
row objects from which fields can be extracted. Converting that to a Rust
struct is the only operation which needs to go fast.

[1] [https://github.com/sfackler/rust-
postgres](https://github.com/sfackler/rust-postgres)

~~~
rabidferret
In that very specific example, you are right -- the benefits are pretty
minimal (mostly just the deserialization). However, you're actually able to
represent the relationship between those two models _however you want_ , but
still deal with relationships between types, and not what the exact join
clause is.

Additionally, SQL is extremely hard to compose. In my applications, I very
rarely have some specific one off query that I want to run, I have multiple
predicates that I want to be able to mix and match easily. String munging is
fragile, while this actually lets me build reasonable domain abstractions.

To your final point "Unless this has some huge performance gain over rust-
postgres"

Diesel performs 33% faster than rust-postgres, and will be competitive with if
not faster than ideomatic C.

~~~
BinaryIdiot
> Additionally, SQL is extremely hard to compose.

I've seen almost everyone who doesn't use SQL directly say this but I always
feel it's the exact opposite. SQL is easy to write once you do it enough. I've
gotten to the point where I compose zero SQL in any application's code and,
instead, do it all inside of stored procedures (of which they do not generate
dynamic SQL). Then my code just calls stored procedures, my database users for
the web application never have more access than executing a specific subset of
stored procedures, and if there is a bug found in a stored procedure I can
deploy them separately from the web application.

I'm guessing it's just different philosophies but almost every time I've used
something that abstracts an existing interface to another, completely
different interface, I always end up running into major issues.

~~~
rabidferret
Stored procedures are no more composeable than SQL strings are. Now it's
certainly easy to say that I haven't used SQL directly. But I did spend years
working that way (though of course this statement shouldn't give you much
confidence by itself).

Ultimately an ORM _can_ help you to build more maintainable and resilient (to
changing market conditions) software than direct SQL. Diesel in particular can
catch mistakes that SQL strings cannot.

I do understand why a lot of people have reservations about ORMs. I share a
lot of them. I've maintained Active Record for about 2 years now, and learned
a lot from doing so (note: This does not mean that I like AR or want this to
be anything like it).

TL;DR: Your points aren't wrong. Diesel is different. Maybe give it a shot. ;)

~~~
BinaryIdiot
> Stored procedures are no more composeable than SQL strings
> are[...]Ultimately an ORM can help you to build more maintainable and
> resilient (to changing market conditions) software than direct SQL.

I couldn't disagree more. Creating the separation between SQL DB and web
application allows you to maintain, deploy, fix and upgrade things
independently and it's easy (maybe not quite as easy as an ORM would make it
but not really harder either; you're going to have to know how SQL works
anyway if you want to effectively optimize and secure your data).

Not only that but you can provide far better security controls because you can
limit certain users access to certain procedures, never direct access to
tables and various other things and if you don't do any dynamic SQL in the
procedures itself you not only never have to worry about SQL injection but
unless there is a critical vulnerability in the SQL system itself it's going
to be impossible to conduct an injection attack.

Yeah you can separate your code to make it easier to deploy separately to get
a similar benefit and yeah your code can prevent SQL injections too but it's
so much more work, more possible points of failure and the differences between
databases can really bite you in the ass when you do everything generically.
SQL is fun :)

> TL;DR: Your points aren't wrong. Diesel is different. Maybe give it a shot.
> ;)

Yeah I'm not saying NEVER use ORMs; they're great for creating things quickly
that don't need to scale. But after my experience with them I'm highly
skeptical using them outside of that type of use case. Of course you can scale
ORMs but I've never seen it done well.

~~~
rabidferret
I think we're violently agreeing in a lot of ways. :)

I'm curious what you mean by scale. Are you referring to performance, or code
size? If you're referring to performance, you'll probably find it interesting
that we out-perform or are competitive with ideomatic C.

~~~
BinaryIdiot
By scale I mean performance though I'm not sure the application's language
matters as much as the SQL that's being generated and run against the
database. I've just never had good luck with ORMs generating the best, most
optimized SQL for the query I'm trying to run and I've known a few people who
used an ORM until they started getting heavy traffic, tested out talking
directly to the RDMS, found it to be more performant and end up dropping the
ORM.

Don't get me wrong ORMs can certainly be performant. I just haven't seen or
know anyone who used it in a manner that scaled to thousands of concurrent
users and, at the same time, didn't have a lot of pain trying to do that with
the ORM.

------
erickt
If anyone is interested in hearing from the creator, Sean Griffin will be
talking about Diesel at the next Bay Area Rust Meetup on February 18th [1],
which will be live streamed and eventually archived on Air Mozilla [2]:

[1]: [http://www.meetup.com/Rust-Bay-
Area/events/219697075/](http://www.meetup.com/Rust-Bay-Area/events/219697075/)

[2]: [https://air.mozilla.org/bay-area-rust-meetup-
february-2016/](https://air.mozilla.org/bay-area-rust-meetup-february-2016/)

~~~
dikaiosune
He also did a good podcast interview about Diesel and about Rust (and what
makes a Rust ORM special with compile time magic) a week or so ago:

[http://bikeshed.fm/49](http://bikeshed.fm/49)

------
steveklabnik
Sean Griffin (and others), the maintainer of Rails' ActiveRecord, has been
working on this ORM for a while now. It's not a port of ActiveRecord, but a
look at what an ORM that takes advantages of Rust's strengths would look like.

It makes heavy usage of Rust's type system features, particularly zero-sized
types, to provide a large degree of safety with no overhead. Sean has
mentioned several times that he even believes that he is faster than literal
SQL strings, due to pre-compiling them in some way, but that's a bit out of my
league.

He also wants to have features like "type-check all of your queries at compile
time and then query the database to make sure that the schemas all line up"
and other such compile-time safety shenanigans.

------
brandonbloom
Honest question: When would it ever really make sense to use Rust on the
server for a service that interacts with a SQL database heavily enough to
justify using such a library?

Wouldn't waiting on the network eat up any possible performance gain you could
expect from Rust vs something like the JVM or Go runtime? Before modern day
JavaScript JITs, relational database systems were the epitome of dynamic
runtime systems. There's so much performance variability in the query planner,
network, disk caching, etc, to make garbage collection a rounding error. Zero-
cost abstractions are just not a particularly useful thing in this context
from what I can tell.

Even if you do have genuinely CPU-bound tasks, why wouldn't you use Rust for
those modules + call it via FFI from whatever is doing most of the boring
business logic? If it's just about static-type-check-all-the-things! I just
can't get excited about it vs something like Go.

~~~
threatofrain
In my limited experience using both Go and Rust for hobbyist web scraping, I
haven't found either language to have a significant edge in development
difficulty. Go's most salient edge over Rust would be in the more numerous and
mature libraries.

So if Rust is not substantially easier or harder than Go, when Rust catches up
to Go in the library ecosystem, I can see why people might want to use Rust.
Also, it's nice to have better language support for functional strategies.

~~~
brandonbloom
I think that Rust is substantially harder than Go. At least extrapolating from
my personal experience and the 6+ highly-skilled people I've watched learn
both Go and Scala simultaneously.

~~~
pcwalton
I think you're right--Rust is harder to learn than Go--but I also think that
doesn't really matter, since the important thing is whether people _fluent in
both languages_ can write code faster in Rust or Go. I have not seen many if
any instances in which people who actually know Rust write code more slowly in
Rust than they would in other languages (modulo compiler performance). The
borrow check is, like, less than 5% of the errors I see, and the errors are at
this point not significantly harder to fix than a misspelled variable name.

~~~
dikaiosune
Re: modulo compiler performance

My productivity has been greatly improved by restricting myself to `cargo
check` when writing code. It only does type and borrow checking which is much
faster than doing all of the LLVM passes (or whatever rustc does after those
passes, on that subject I am ignorant).

~~~
Manishearth
Note that even for large codebase, the Go compile is faster than Rust up to
typecheck. Not much faster, but when I use Go I get errors immediately;
whereas in Rust I often context-switch to wait for errors (both for large
codebases). Some planned improvements to Rust may fix this.

~~~
dikaiosune
Fair enough, and being honest I haven't tried Go yet so I have no basis for
comparison between the two compilers.

I've found that when using Atom with linter/linter-rust installed and setup
with cargo check I rarely context switch while waiting for errors/warnings to
be highlighted (and this is on a 1.6Ghz Xeon machine at work). But maybe I'm
just OK staring blankly for slightly longer than others :).

~~~
Manishearth
Well, I work with really large codebases in Rust (Servo or Rust itself), so
the twenty seconds to a compile error are a lot. Whereas when hacking on Go's
compiler it's much, much less.

------
schwap
I know the ".rs" TLD is common to rust projects, but maybe add "for Rust" to
the title?

~~~
baddox
I had no idea that .rs was conventionally for Rust projects, but I'm also not
too bothered by having to figure it out.

~~~
schwap
I'm not sure it's convention but it's common just due to matching the file
extension.

~~~
rabidferret
It's technically for the Republic of Serbia. I only chose it because .io was
taken by a python library (which I wish I'd checked before deciding on the
name)

It's also a pain in the ass to register because you have to have a Serbian tax
ID

------
frik
Naming something "Diesel" is problematic. While the name is well known because
of Rudolf Diesel, a German inventor of the Diesel engine and the Diesel fuel.

There is an Italian clothing company "Diesel S.p.A." that tries to protect
their brand by suing almost anyone using "Diesel".

For example a German video game called "DieselStormers": due to a successful
trademark lawsuit by Diesel S.p.A., who owns the trademark "Diesel" in
relation to video games, Black Forest Games had been required to drop the
title DieselStormers.
[http://www.eurogamer.net/articles/2015-09-30-dieselstormers-...](http://www.eurogamer.net/articles/2015-09-30-dieselstormers-
forced-to-change-name-after-diesel-wins-trademark-dispute)

Such greedy companies act very similar to patent trolls. Well, there are sadly
several other common words that are tainted by such "brands".

Edit: In 2015/2016 the Volkswagen (VW) Diesel scandal tainted the "Diesel"
name too: [http://autoweek.com/car-news/vw-diesel-
scandal](http://autoweek.com/car-news/vw-diesel-scandal)

~~~
walterstucco
Chosing Diesel as a name in 2016 is not exactly smart. greedy companies or not

------
usaphp
Really nice home page, clearly explains what the product is for and how it
works, I really wish more tools would have such landing pages, most of the
times I am spending a lot of time just figuring out what the hell is that new
tool about. Great job!

------
ixtli
I am a big fan of typography but I have to draw the line at ligatures in code
samples. It's just not appropriate for type that actually has a compelling
reason to be monospaced =)

~~~
rabidferret
Mind posting a screenshot of what you're seeing? All our code samples are
`font-family: monospace`

~~~
bschwindHN
I think they're referring to the "fi" in filter:

[http://i.imgur.com/tJBeeNt.png](http://i.imgur.com/tJBeeNt.png)

~~~
ixtli
^ This is what I'm seeing. Sorry it took so long :(

------
AYBABTME
This is pretty cool, makes me want to take a look at Rust.

------
tracker1
I don't quite get why nobody simply creates function bindings that have a
query template string on one side, that is represented as a function that
creates a parameterized query under the covers, and a solid type that
represents a record that is returned? I mean instead of jumping through all
the binding chains.. just have a string and type... call the generated
function based on the string, return an array/stream/whatever of <type>

I'd rather use template string generated promises in node, tbh, than any ORM.

~~~
rabidferret
That is our `sql` function

------
lobster_johnson
Very cool. I like that it (as far as I can see) is actually built on top of an
SQL AST.

This is one of the shortcomings of ActiveRecord. The SQL building used to be
extremely ad-hoc, and though ARel was eventually written to fix this problem,
the integration between ARel and ActiveRecord is not very fluid (and ARel
itself has some design issues; if I remember correctly from the last time I
worked with it, it's not immutable, which can be very confusing when chaining
expressions).

~~~
rabidferret
Yeah, I'm planning on killing Arel at some point. I actually want to extract
`Relation` to a gem entirely, as it's really not an adequate query builder for
what we need in AR.

Also random fun story about Arel not being immutable. We had a bug in 4.1
where basically PG bind params are ordered, but Relation can't actually know
the order ahead of time (e.g. because a Relation can be used in a from
clause). Turned out actually fixing it was somewhat hard (the "real" fix was
eventually
[https://github.com/rails/arel/commit/590c784a30b13153667f8db...](https://github.com/rails/arel/commit/590c784a30b13153667f8db7915998d7731e24e5)).
But as a hack, basically someone took advantage of the fact that
`Arel::Nodes::BindParam < SqlLiteral` and `SqlLiteral < String`, and Strings
are mutable. So they just ignored the lack of a proper map function, and
fiddled with the strings. >_>

Anyway, yeah our AST is fun because Rust's compiler lets us set it up so that
we can walk it as many times as we need basically for free (because it gets
monomorphised and the walking ultimately gets inlined and optimized away)

------
15155
How does this reconcile the async IO problem?

I couldn't figure out from cursory reading how "PgConnection" works or how it
would be compatible with other IO-related code.

~~~
emergentcypher
This is still one of Rust's major shortcomings. Async IO, particularly async
database and network IO, are still pretty much nonexistent in any practical
sense. Some libraries are in the works, but for the time being, I'm still
passing over Rust for any real work.

~~~
steveklabnik
It's very close to landing in hyper, which should then percolate up through
the rest of the ecosystem, at least, HTTP-wise.

~~~
bsaul
Do you know of any resource that would explain the current status of running
rust as a webservice server endpoint ?

I'm thinking of question such as :

-embedded http server vs using apache or nginx ( eg : node and go seem to favor not using third party servers at least for simple cases)

\- prefered concurrency model ? ( async vs thread based vs coroutines vs...)

\- deployment issues ? ( single binary with everything inside, vs parts that
need to be predeployed on the server beforehand)

\- third-party connectivity ? ( diesel seems like a really fantastic way to
query a db, but what about other storage such as s3, mongo, redis, queue
systems, etc)

\- current frameworks status ?

I know it sounds like a lot, but as ORMs is usually the major pain point for
me when looking at a new techno ( it's one of the main reason i'm not using
go, for example) and now that diesel seems to answer all my needs, i'm REALLY
looking forward to start using Rust. I just need those questions to be
answered, and i think i'm not the only one.

~~~
steveklabnik
I don't know of one, so let me write up something short:

    
    
      > embedded http server vs using apache or nginx 
    

For now, put nginx in front of things, at least if you're using one of the
non-rotor frameworks. That's because they are synchronous for now. This will
change soonish, more below.

    
    
      > prefered concurrency model ?
    

If you're using a framework, this is handled for you already. If you're
building something custom, you probably want to use something like mio to make
it asynchronous. Current frameworks are all using synchronous IO, but work is
undergoing to switch that, which will then percolate upwards through the
ecosystem.

    
    
      > deployment issues ?
    

By default, everything is statically linked except glibc, so you make sure
that those are compatible and then copy the binary.

    
    
      > third-party connectivity ?
    

Wonderful or nonexistent depending. We're still youngish, so there's gaps.
Support ranges from "provided by the vendor itself" (mongo) to "nonexistent"
(some random noSQL store that might be popular amongst a certain group but
someone hasn't written a binding yet)

    
    
      > current frameworks status ?
    

The two most well-known are Iron and Nickel. Crates.io uses its own homegrown
framework due to age. The author of Diesel may or may not be working on a new
one to go with it...

~~~
dikaiosune
> The author of Diesel may or may not be working on a new one to go with it...

IIRC, he hints at working on one (I think with Yehuda Katz?) in the podcast
where he talked about Diesel:

[http://bikeshed.fm/49](http://bikeshed.fm/49)

------
kbenson
An FYI to anyone responsible for this site, or in contact with them, the
visited and unvisited link colors in the getting started guide[1] are _very_
close to the background color, which makes them hard to read. Specifically,
the link to dotenv and the CLI.

1: [http://diesel.rs/guides/getting-started/](http://diesel.rs/guides/getting-
started/)

~~~
rabidferret
Thanks. The designer is taking another pass at it soon, I didn't expect to
have so much traffic. I'll bump up the color on the links until then.

------
kevindeasis
This would be really awesome if someone used this with a rust framework being
used in production

It would be really cool if this had an easy integration with postgis that uses
simple syntax to query location

When I'm using a node framework and using postgres I have to use a nosql db
for playing with coordinates as postgis syntax is not something i am very fond
of

~~~
rabidferret
FWIW making sure it's easy to add extensions like this is part of the core
design of Diesel. For example, this adds full text search support (it's
slightly out of date)
[https://github.com/sgrif/diesel_full_text_search/blob/master...](https://github.com/sgrif/diesel_full_text_search/blob/master/src/lib.rs)

If you're interested in adding postgis support, ping me in our Gitter room.
I'd be happy to help.

------
gleb
How does this compare to Elixir's Ecto?

~~~
rabidferret
I'm having Jose on The Bikeshed (bikeshed.fm) next month to compare notes!

------
dikaiosune
In the getting started guide, it looks like they use an as-yet-non-existent
Post struct to create the migration SQL. Where is the migration CLI getting
the struct's fields from if it hasn't yet been written in that part of the
tutorial? Am I missing something obvious?

~~~
rabidferret
The CLI just generates the file structure. We fill in the SQL manually as part
of the guide, and then create the structs immediately after that.

~~~
dikaiosune
What I get for reading too quickly! Thanks for clarifying.

~~~
rabidferret
It wasn't super clear, anyway. Fixed in [https://github.com/sgrif/diesel.rs-
website/pull/5](https://github.com/sgrif/diesel.rs-website/pull/5) :)

------
dutchrapley
I like how the logo uses a red gas can. Diesel fuel is actually goes in a
yellow gas can.

~~~
rabidferret
I think it's green, at least in the US. XD

~~~
jdpedrie
nope, diesel is yellow. never seen a green fuel can.

~~~
andrewaylett
For plastic 'cans' in the UK, we use green for unleaded petrol. Red was for
leaded petrol and diesel is black.

------
DannoHung
Why did a NewPost struct have to be created to actually add a new Post? Is it
generally not possible to have the result type of a query be used to update
the same records?

~~~
rabidferret
Update yes. It's possible to use the same struct for create, too if you really
want, but you'd need to wrap `id` in an `Option`.

Generally we're designed for the idea that you'll have a separate struct for
read, create, and update. We also try and design for the latter two mapping
roughly to web forms, as this is how most large applications end up wanting to
be structured from my experience.

The main thing I'm trying to avoid is lumping all your logic in a single
place, and ending up in many of the same rabbit holes that Active Record has
led us to (which is at least partially my fault)

~~~
DannoHung
I guess that makes sense. The types should be able to ensure you don't have
any errors even with so many "redundant" types, but my first thought is of how
much boiler plate will result. Like, I would guess _most_ record types will be
very similar between create, update, and retrieve.

Also, what about upsert?

~~~
rabidferret
> The types should be able to ensure you don't have any errors even with so
> many "redundant" types

Correct. Sorry, I should have been clearer in my last comment. The reason
you'd want a separate type for insert is because wrapping your id in an
`Option` would be painful for anything _except_ insert.

> but my first thought is of how much boiler plate will result.

Yeah, if you're coming from Active Record (which is the other project I
maintain), we'll probably seem a little bit verbose. My goal here is to be 80%
as nice to get started with, but nudge users towards significantly more
maintainable code as their product grows.

> I would guess most record types will be very similar between create, update,
> and retrieve.

Early on, yes. 2 years in, not so much.

> Also, what about upsert?

This is something I'd like to add support for in the future. This comment made
me realize I didn't have a tracking issue, so here it is:
[https://github.com/sgrif/diesel/issues/196](https://github.com/sgrif/diesel/issues/196)

~~~
twic
We went through a somewhat similar process at an old job. Entities don't have
an ID before they've been saved, but do after. Other things can change over
their lifecycle too. At one point, we had separate types for each stage in the
lifecycle, a bit like your different types for insert and read, but that makes
it impossible to write any kind of code which is stage-agnostic - so for
example you need a separate to-string function for each stage.

We were working in Scala, and I had a go at solving this using a higher-kinded
type for the ID and other changing fields:

[https://github.com/tim-group/higher-kinded-
lifecycle/blob/ma...](https://github.com/tim-group/higher-kinded-
lifecycle/blob/master/src/main/scala/Idea.scala)

The idea is that if you say that the ID is of type 'T of String, where T is
some subtype of Option', then you can bind T to None for entities that haven't
been saved, Some for ones which have, and Option where you have a mixture. You
then have one type definition, with which you can write stage-agnostic code,
but static guarantees about whether there's an ID.

You can go further and use type constraints to enforce the order of lifecycle
stages: if your object can only be saved to the database once it's been given
a timestamp, say, you can make sure that the ID type can only be Some if the
timestamp type is also Some.

So maybe when Rust gets HKTs it'll be time for a new release!

~~~
Manishearth
This type of bound can be expressed with a trait and associated type already
in Rust, fwiw.

------
cetra3
Can you just use stock-standard parametrised SQL queries or do you have to use
the ORM to build the SQL statement?

~~~
rabidferret
We have a SQL literal ast node but you lose most of the benefits of the
library that way

------
dschiptsov
Safe from what?

~~~
steveklabnik
Type safe, memory safe.

~~~
dschiptsov
How is a strong-typed language, such as CL, is less safe?

Edit: let's paraphrase - how excessive safety nets for construction workers
makes buildings more safe?

A strong-typed language without implicit coersions is not more "unsafe".

Baning heterogeneous containers and conditionals, maybe-wrapping of everything
to be able to check some constraints at compile time and catch _simple_ errors
(which also would be caught in a strong-typed language at run time) does not
quality for this "safe" meme.

