

Three months of Rust - abecedarius
http://scattered-thoughts.net/blog/2015/06/04/three-months-of-rust/

======
Manishearth
> The Rust community seems to be populated entirely by human beings.

:D <3

Regarding your borrow checker example, note that your code is now prone to
blowing up if `step` is modified too much. You have created the necessity of
an invariant (step should not pop out of the vector) which may be broken by
later cleverness.

See [http://manishearth.github.io/blog/2015/05/17/the-problem-
wit...](http://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-
mutability/) for more details.

Note that in this specific case you could just use `&str` over `&String`
everywhere and push "some new thing" directly; `&String` is a double-pointer,
whereas &str is a fat pointer.

> Nor am I totally sure what the tradeoffs are between having a self argument
> and not.

It's not a tradeoff thing; it's a "do I want a static method, or a member
method" thing.

> Some kinds of constraints cannot be used in where clauses, so I believe the
> former is strictly more powerful.

Actually where clauses are much more powerful. With the type constraints stuff
like `A: Foo` works but not `Vec<A>: Foo`, but the latter is allowed in where
clauses.

Overall, loved reading this post! It identified some areas of diagnostics that
we can try to improve (I'm very interested in fixing diagnostics), and is a
pretty accurate picture of the language :)

~~~
jamii
> Note that in this specific case you could just use `&str` over `&String`
> everywhere

It's an awkwardly construed example. Here is the actual code -
[https://gist.github.com/jamii/ae46e8e0c9757330e9ea](https://gist.github.com/jamii/ae46e8e0c9757330e9ea)
. There the borrow makes more sense since the value is being created by
calling a function on the current solver state.

I've edited the post to include a solution that was suggested in the reddit
discussion - replace &'a Value with Cow<'a, Value>.

> It's not a tradeoff thing; it's a "do I want a static method, or a member
> method" thing.

> Actually where clauses are much more powerful.

I really don't understand this part of the language yet, but...

I linked to an active rfc about constraints that cannot currently be expressed
in where clauses ([https://github.com/rust-
lang/rfcs/blob/master/text/0135-wher...](https://github.com/rust-
lang/rfcs/blob/master/text/0135-where.md)), including the example you gave.

In cases where a function takes two arguments it isn't always obvious which
argument should be self. If that affects whether constraints end up being A:
Foo<B> or B: Foo<A> or `where Foo<A,B>` it seems like one could run up against
those limitations?

~~~
niconii
I think you may have misread that RFC. With my emphasis:

> Here is a list of limitations with the current bounds syntax that are
> _overcome_ with the where syntax:

The list is of the limitations of normal bounds, not the limitations of where
clauses. This was actually the RFC that _added_ where clauses, and that list
was the rationale for doing so.

~~~
jamii
Oh... well that's embarrassing.

So, I half-remembered the actual problem I ran into and found something that
half-looked like it mentioned it. Not my finest hour :S

I dug up the IRC exchange for the problem I actually ran into:

    
    
        jamii
        How do I write the type of a byte iterator:
        fn next_path<N: Iterator>(nibbles: &mut N) -> u32 where <N as Iterator>::Item = u8 {
        That gives me 'equality constraints are not yet supported'
        FreeFull
        jamii: <N: Iterator<Item = u8>>
    

Equality constraints seem to still be unimplemented (
[https://github.com/rust-lang/rust/pull/22074](https://github.com/rust-
lang/rust/pull/22074)) but I can write this instead

    
    
        where N : Iterator<Item=u8>
    

So that whole section of the post is incorrect. I've removed it and linked to
this discussion instead.

~~~
jroesch
I'm the author of the above pull request (as well as other pieces of where
clauses), unfortunately there are some internals that need refactoring before
we can complete equality constraints, and they weren't the highest priority
before 1.0 especially since you can encode a trait that acts in the same way.
I'm starting at Mozilla for the summer next week and hope to fix a couple of
the outstanding issues around them and associated types.

------
glandium
The take-away for me is this:

 _" Despite the restrictions of the type system, I am more productive in Rust
than I am in either Javascript or Haskell. It manages somehow to hit a sweet
spot between safety and ease of use."_

When I toyed with Rust last year (so, I'll admit my knowledge is outdated, I
need to refresh it), I had a pleasant experience on the productivity side. The
big reward for me, coming from C/C++, is that my programs simply worked as
expected once I was past fixing all the errors the compiler reported. That
usually doesn't happen this way in C/C++, where you spend additional time
fixing whatever null deref and whatnot that break your program in subtle ways
at runtime.

~~~
shmerl
Doesn't using C++ RAII eliminate some of that? Or you use raw pointers often?

~~~
the_why_of_y
Anything in C++ generally comes with its very own footgun.

The one for RAII looks like this:

    
    
      {
        mutex_guard(some_mutex);
        foo();
      }
    

What does this do? It locks some_mutex, then immediately unlocks it, then
calls foo().

~~~
kluge
I don't understand the example. Why would it do that and not unlock after
foo()?

~~~
the_why_of_y
The following code would do the right thing; see if you can spot the
difference, and think about whether you would catch that in code review
(g++/clang++/visualc++ won't warn about it).

    
    
      {
        mutex_guard guard(some_mutex);
        foo();
      }

~~~
kluge
Ah, now I see it and it makes sense.

The answer to the code review question is obvious. I didn't see the error even
though I was told what happens.

------
andrewchambers
"For our 2400 loc it takes 20s for a dev build and 70s for a release build. "

I have played with rust, but not written any large amounts of code. This makes
me a bit sad though, I have 7000 lines of go which takes less than a second. I
think there is a bunch of bloat in software compilation which the plan9/Go
people were wise to stamp out.

Compare gcc/clang/rustc build times from source with building go 1.5 from
source which bootstraps itself. It comes down to something like 20 minutes vs
20 seconds.

~~~
legutierr
If I remember properly, when Go was first developed, compilation time was one
of the primary metric that Rob Pike et al were optimizing for, and drove major
aspects of its design. It shouldn't be surprising that Go blows other systems
out of the water in this regard.

Here he is talking about it:

[https://www.youtube.com/watch?v=rKnDgT73v8s#t=8m53](https://www.youtube.com/watch?v=rKnDgT73v8s#t=8m53)

~~~
pcwalton
My point is that you pay a price for this: LLVM's optimization passes are
much, much more sophisticated than those of the Plan 9 toolchain. In optimized
builds of Rust, the LLVM optimization and compilation time tends to dominate,
so having a simpler type system wouldn't really help.

You could have a more C-like language that isn't so dependent on expensive
optimization passes like multi-level inlining and SROA, granted, but I think
most any high-level language—i.e. one that isn't a bare-metal language like C
and Pascal—is going to have lots of opportunity for expensive optimizations.

~~~
tormeh
If optimization is the problem, then compilation at Go speed should be
possible with -O0.

~~~
DasIch
Just having the ability to perform such optimizations requires an architecture
that is sure to have some overhead no matter which optimizations, if any, are
actually executed.

~~~
Jweb_Guru
Rust could still have a toolchain like DMD devoted to fast compilation with
minimal optimization. It just doesn't, yet (and likely won't for quite some
time, since the present advantages of having a single toolchain are fairly
significant and Rust doesn't have a formal specification yet).

~~~
pcwalton
There are C compilers out there that work like this (like, say, the Plan 9 C
compiler) and they rarely ever get used in practice, because GCC and clang -O0
are fast enough. I think making a second compiler backend just to eliminate
the IR generation step isn't going to be the solution we want in the long run.

~~~
andrewchambers
I think they are only fast enough because developers have not been exposed to
better. In general, I would like to see more aggressive build modes for CI
servers, and less aggressive modes for dev.

~~~
tarblog
Also, I'm confused by the word "aggressive" here. Could you elaborate please?

~~~
andrewchambers
Aggressive optimization. I just mean on a build server time isn't as much of a
factor as local development.

------
jwmerrill
_" Modern machines are a huge pile of opaque and unreliable heuristics and the
current trend is to add more and more layers on top. The vast majority of
systems are built this way and it is by all accounts a successful strategy.
That doesn’t mean I have to like it."_

This is a really valuable observation.

"Smart" compilers seem great for letting you write code without thinking too
hard when performance requirements are loose, but they make it difficult to
achieve peak performance in two ways:

1\. The fact that details of the machine are abstracted away from you means
that you may never learn them well.

2\. It ends up being insufficient just to know the details of the machine,
because you also need to know how to coax the compiler into producing the low
level result that you want, and then you need to be vigilant that later
changes to the compiler don't break your assumptions.

~~~
pron
I think the opposite is true. Modern hardware is so complex[1], made even more
so by its constant interaction with a complex OS, that any sense of
familiarity with the actual performance model is illusory, unless you're doing
something _very_ controlled and _very_ specific (like, say, DSP). Modern
hardware itself is an abstraction, hiding its operation away from you. We can
no longer hope to tame hardware with meticulous control over instructions as
we were able to up until the nineties.

Forget about clever compilers; forget even about smart JITs; even if you look
at such a big abstraction as GCs and only consider large pauses (say anything
over a few tens of milliseconds), it is now the case that in a well-tuned
application using good a GC, most large pauses aren't even due to GC, but to
the OS stopping your program to perform some bookkeeping. Careful control over
the instruction stream doesn't even let you avoid 100ms pauses, let alone
trying to control nanosecond level effects.

[1]: [http://www.infoq.com/presentations/click-crash-course-
modern...](http://www.infoq.com/presentations/click-crash-course-modern-
hardware)

~~~
jamii
Most of us don't have the time for meticulous control over instructions but
those who do can certainly use it to good effect eg
[http://www.reddit.com/r/programming/comments/hkzg8/author_of...](http://www.reddit.com/r/programming/comments/hkzg8/author_of_luajit_explains_why_compilers_cant_beat)

My aversion to piles of opaque heuristics is not because I'm against smart
compilers, just that for certain projects I want to be form a mental model of
what code I should write to get a certain effect. The trend of modern
languages with heavy heuristic optimisations or complex JITs is towards less
certainty and less stable optimisations, so that a program that runs fine
today might be unusably slow tomorrow.

Staging and compiler-as-a-library is a promising compromise for projects that
really care about stable performance eg
[http://data.epfl.ch/legobase](http://data.epfl.ch/legobase) . You can still
have an LLVM-smart compiler underneath but you get to make the first pass.

Rust is actually very predictable in some respects eg generic functions _will_
be monomorphised. I prefer it to wrangling GHC or the V8 JIT.

~~~
pron
> I want to form a mental model of what code I should write to get a certain
> effect

And how do you do that with hyperthreading, virtual memory, power management
that may decide to power down your core because what you're doing doesn't seem
important enough (and that differs greatly from one processor to another) and
cache effects on code, data and TLB (all are strongly affected by other
threads and processes running on your machine[1])?

While those effects didn't exist much before the 90s, and they don't exist
today in GPUs and small embedded devices, on desktops and servers those
effects may be much greater in magnitude than any difference you're able to
get by better control over generated code. Not running a hypervisor, turning
off virtual memory, pinning threads and isolating cores have a much more
profound effect on predictability than which language or compiler you're
using. Focusing on compilation before taking care of those much more powerful
sources of unpredictability is like trying to get a faster car by reducing the
weight of the upholstery fabric.

> so that a program that runs fine today might be unusably slow tomorrow.

I think that slowdown actually applies to assembly programs much more than to,
say, Java. As CPU architecture changes, it's actually easier to keep higher-
level code performant. I mean, why do you assume that compiler changes will
hurt your code performance more than CPU changes?

> You can still have an LLVM-smart compiler underneath but you get to make the
> first pass.

There are many ways to produce good machine code (my favorite is Graal,
HotSpot's next-gen JIT), but none of them really give you a good mental model
of what's going on. You may like one approach over another for personal
aesthetic reasons, one approach may actually produce better results for some
workloads than others, and some approaches really are _more_ predictable --
but no approach produces categorically predictable results, and more
predictability doesn't buy you better performance (though it still requires
more effort).

It used to be that if you knew what instructions your compiler would emit, you
knew how your program would perform. That is just no longer the case (well, it
is to some degree, but other effects are stronger). A single instruction may
perform anywhere within 7 orders of magnitude (L1 cache hit to virtual memory
miss) depending on effects outside the program's control! (of course, those
high-volatility costs are usually amortized, but so is a less unpredictable
compiler output).

[1]: That is the key to cryptographic attacks that let a process sense what a
cryptography algorithm running in another process is doing by the way the
cryptographic computation affects the performance of the first process.

~~~
jamii
I think you are taking a very black and white point of view. Yes, hardware is
complex and unpredictable. That doesn't mean that we can't reason at all about
performance.

I take a program, measure it's performance on a wide range of real-world
workloads across multiple different machines. Then I change some numeric
routine to use unboxed integers instead of boxed integers. I measure it again
on a wide range of real-world workloads across multiple different machines and
find that it is significantly faster in all cases. My approximate mental model
of how the machine works allowed me to make a change that empirically improved
performance. My model is not perfect so I do have to measure carefully, but it
is what allows me to make sensible decisions about which changes to measure
rather than just changing things at random.

In a language where the compiler controls unboxing, my mental model is much
more approximate. I have to figure out how to influence the heuristics to lead
them into making the correct choice, and the solutions tend to be hacks that
are highly sensitive to small changes to the heuristics, leading to
conversations like
[https://groups.google.com/forum/#!topic/clojure/GvNLOrN3lGA](https://groups.google.com/forum/#!topic/clojure/GvNLOrN3lGA)
.

Performance for non-tuned code may be better on average but my ability to tune
important areas is reduced. If the compiler was more predictable, or had a
interface that allowed me to add information, or if I could make my own passes
then that trade-off would go away. I'm not against smart compilers, I'm
against smart compilers that don't talk to me.

~~~
pron
> I'm not against smart compilers, I'm against smart compilers that don't talk
> to me.

There are some extremely interesting advances in that area in OpenJDK. Java 9
will contain two relevant changes. The first, JEP 165[1] (fine-grained and
method-context dependent control of the JVM compilers), lets you control
compilation with metadata depending on context (e.g. inline method foo when
called from bar); a much more interesting and powerful enhancement targeted
for Java 9 is JEP 243[2] (Java-Level JVM Compiler Interface). It will do the
following:

* Allow the JVM to load Java plug-in code to examine and intercept JVM JIT activity.

* Record events related to compilation, including counter overflow, compilation requests, speculation failure, and deoptimization.

* Allow queries to relevant metadata, including loaded classes, method definitions, profile data, dependencies (speculative assertions), and compiled code cache.

* Allow an external module to capture compilation requests and produce code to be used for compiled methods.

This opens the door to what I think is the most impressive compiler of the
last decade, and a true breakthrough in (JIT) compiler design: Graal[3]. Graal
supports languages of any level (it already has frontends for Java, C, Ruby,
Python, R and JavaScript), and then allows complete control over code
generation and optimization decisions at runtime. E.g. you tell it what kind
of speculations to make, and it tells you which speculations failed. Unlike
LLVM, you compile your language into a semantic AST (that may or may not match
the language's AST) and feed it to Graal, but each node may contain not just
semantics but instructions on speculation and code-gen control at any level
you wish. During compilation, Graal interacts with the node and the node gives
further instructions. As I understand it, JEP 243 will allow to plug Graal
into the standard OpenJDK HotSpot (though at reduced speed), until Graal
matures enough to become HotSpot's default compiler.

So what Graal will do is let the developer (if the language designer allows),
write simple, high-level code, but tell the compiler, "listen, compile however
you like, but when you get to this function, talk to me because I have some
ideas on how to compile it just right".

[1]: [http://openjdk.java.net/jeps/165](http://openjdk.java.net/jeps/165)

[2]: [http://openjdk.java.net/jeps/243](http://openjdk.java.net/jeps/243)

[3]:
[https://wiki.openjdk.java.net/display/Graal/Publications+and...](https://wiki.openjdk.java.net/display/Graal/Publications+and+Presentations)

~~~
jamii
Thanks, that is really interesting. I'll have to look into it.

------
halosghost
+1, at the very least, because of the nod to Terra. Terra, imho, feels a lot
like the perfect middle-ground between Lua and Rust. It has a clean syntax
with some handy/fancy features but keeps a simple static typing system that
makes me feel comfortable.

Also, nice to read a review of Rust that didn't reduce to “C is the worst
evar!” or “Haskell makes no sense!”. Reasoned and clearcut. I certainly
disagree with various parts of the review, but I don't do any dev for
webservices so it is unsurprising that the author and I have a differing of
opinion.

~~~
jamii
I would be interested to hear which parts you disagree with.

> I don't do any dev for webservices so it is unsurprising that the author and
> I have a differing of opinion

I don't either, I'm working on a database / language runtime. It happens to
have a html interface rather than a console interface, but the majority of the
work is very far away from normal web-dev :)

> Terra, imho, feels a lot like the perfect middle-ground between Lua and Rust

In particular, I really like the idea that the Terra type-system is just Lua
code, so you can have different kinds of static analysis in different places
instead of one-type-system-to-rule-them-all. Of course, it's a totally
unproven idea at this point so it's hard to say how that would turn out in a
real project.

------
d4rkph1b3r
Great to see some feedback on Rust, I've only played a bit but was quite
impressed, dsepite being a mostly high level programmer.

However, it sounds like Eve is a simple, dynamically typed programming
language/environment. So it's super weird to me to see him rave about the
safety and type system of Rust...

~~~
jamii
Right tool for the job. Static typing works well for systems software. Eve is
aimed at scripting / knowledge work, where you are mostly manipulating
collections of messy data and it's useful to be able to start with a loosely
specified program and only nail it down with types once it settles down.
Imagine a relational database where you can vary between setting every column
to Any and not caring about relationships, or strictly typing everything and
adding integrity constraints all over the place.

------
voltagex_
Coming from C#, that syntax looks completely alien to me. I need to write a
very small monitoring app to run on a tiny armel box so I may try Go. I will
still miss Visual Studio's debugger, though.

~~~
d4rkph1b3r
Go is shit. Trust me, Rust will suit you better. Coming from C# it will be a
lot closer to what you're used to than the bizzaro-world of Go.

~~~
arthursilva
Nonsense, go is fine, so is rust.

~~~
andrewchambers
Both are really great. I think Go is better for servers and rust can be great
on embedded and high performance applications.

~~~
simcop2387
I've been tempted to try getting rust going on some of the small arm micros
I've got lying around. The memory safety would be tremendous boon for a lot of
development there, same with the ownership/lifetime management. I think I'd
probably have to strike most of the standard library but it'd still be really
useful.

~~~
andrewchambers
For me I keep seeing buffer overflows in router management stacks. It seems
like rust would help protect routers from attack, while allowing low
footprints.

~~~
simcop2387
Yea that's a common one I've done in a bunch of microcontroller things I've
done. And there it's even more dangerous because you'll overflow the buffer,
and change something and never have ANY idea you've done it until the
consequences have happened. I ran into quite of few of those while setting up
an allocator to get Lua running in <30k of ram without any floating point
math.

------
phren0logy
The Eve language is an offshoot of what started as LightTable, which was
heavily focused on Clojure/Clojurescript.

Interesting that Clojurescript was not the language of choice here, both given
the roots of that project and the reputation of lisps for being languages to
write other languages.

I wonder if the team would be willing to comment on why they are moving away
from Clojurescript?

~~~
jamii
Writing a language runtime in javascript is hard enough, clojurescript adds
yet more layers of runtime overhead that we have to work around.

Clojurescript is a fine applications language, but it's not a good systems
language. Most languages aren't.

------
tatterdemalion
This is a super interesting read! Having come into Rust from an experience
with mostly object-oriented languages (Python, Java, C++), what you seem to
have taken for granted, I found surprising and new, and what you are surprised
by (such as self parameters), I found quite normal. It's great to see the
other side of this.

~~~
jamii
I've actually written a ton of python too. The self parameter didn't confuse
me because of OO, but because traits can actually dispatch on the types of all
of their arguments so it wasn't clear to me what the meaning of the self
argument was and how much it should affect my design.

Manishearth cleared up most of my confusion - it affects namespacing and auto-
borrow but doesn't interact with constraints. Traits are very similar to
typeclasses and I was just thrown by the surface level syntax.

------
vph
A Rust programmer can't seem to be Rusty. The more he's Rusty, the more he
knows about it, which seems like a contradiction.

------
cpp098
New languages always are trap. Programmers always waste too much time on new
languages or some tricky language syntax. They should focus on the business.

I will never try Rust.

~~~
jamii
Well, we first spent six months on 'cannot call function undefined of
undefined'. We could have spent the last three on 'segfault' instead. Rust let
us spend that time on actually experimenting instead of just fighting the
computer all the time.

~~~
aninteger
Spending 3 months on a segfault? In any language even a mediocre programmer
you can usually resolve a null reference exception / segfault in a couple of
hours at the most. With a complex program and a debugger it could be a minute
or so. Maybe if it only happens in production and you don't capture stack
traces or core dump files... but this happens once and then you learn your
lesson.

~~~
nightpool
He obviously meant the general class of errors, not one specific bug.

