Having done a (simple, but non-trivial) memcached implementation in Rust, the pointer semantic is terrible right now.
The compiler tracks all the use of the values and assign them lifetimes. These lifetimes are tagged from a special form of use (value, mutable value, reference, mutable reference, and some more).
The idea of Rust memory-safety is to totally forbid two simultaneous mutable lifetime whatever the code path could be.
Whatever the code path could be. Do you begin to realize how strong this assumption is ?
Yes, the rust compiler is smart. But sometimes, it's not enough and you have to do the lifetime tracking job by yourself. It's awfully hard (even without recursion).
But you know what's even harder ? When all the common code patterns you are aware of for a given task doesn't provide enough warranty for the rust compiler. You know what you want to do. It's indeed very simple (can be an inner swap in some sort of algorithm) but you can't find a way to warranty the compiler that all should be good. Because in its own sort of bizarre logic, it makes sense. Then you're trapped. And you are waiting on #rust@irc.mozilla.org an hint from the bunch of truly marvelous guys who pass their time here.
TLDR: Rust can't express 100% what others languages do (ML or C based). And in the portion where it can, you have to do a lot of gymnastic to make it compile.
And our Memcached, you'll ask ? Well. After 6 days of work, we finally make it compile, and it worked quite fine. But then we got some mysterious tasking errors when reaching 20 clients and gave up at this point.
FWIW, the code displays several signs of misunderstanding and generally fighting the system, rather than working with it (e.g. drop(&foo) always does nothing, and the single required lifetime doesn't seem too crazy: the returned reference comes from inside the memory that `self` points to, so the compiler needed to be told that they were connected to guarantee safety).
Of course, this is presumably a pedagogical problem (e.g. the language grows and changes so tutorials go out of date, including the official one, and the language is new, so there's not a huge number of examples anyway), but I think it's a little unfair to say that Rust can't express a lot of things, when it's mostly just a question of practice. That is, Rust's semantics are fairly different to a managed language like an ML or a "do whatever you like" one like C, and so it takes effort to really 'get' it, just like, for many people, it takes effort to 'get' Haskell.
> But then we got some mysterious tasking errors when reaching 20 clients and gave up at this point.
Your code has a lot of `unwrap` and `fail!`, it's generally recommended to use a more composable error handling technique (like returning Option and Result) or otherwise any error (either in the input, or a bug in your application) will lead to these "mysterious" errors, and especially avoiding `unwrap`.
I haven't done a serious Rust project, but it appeared to me that this kind of thing was exactly the "why" of Rust. The expectations I had gathered was that many obvious pointer algorithms would not (and should not?) translate to Rust.
I definitely asked for a Rust review from the C POV, but I feel like they do a good job marketing the idea that it's not supposed to express 100% of what other languages do.
Or perhaps it's just further from reasonably expressive than I had believed.
> I definitely asked for a Rust review from the C POV, but I feel like they do a good job marketing the idea that it's not supposed to express 100% of what other languages do.
You can express everything that other languages can do—we've written a compiler and a Web browser in it, remember. Of course there is a learning curve, but once you've learned the rules I've found that, by and large, people don't have too many problems, and the time saved by not having to track down segfaults in a debugger is a huge productivity booster.
To be clear, I meant "low level 100%" in the sense of outlawed bad programs. I'm quite confident that it'd still be sufficiently expressive for application level concerns—perhaps with a change of style!
You can only take mutability lifetimes if you are alone. That mean for example that you can't insert in a tree without going mad. Yeah, you have to search (non-mut operation), insert (mut operation), and balance your tree (mut operation). Not possible.
How do you do it in the official B-Tree implementation ? You clone all the branch at each insertion[1] so that the insertion don't compete with the search. Oh well.
Hopefully it's a B-Tree so it won't be too deep. But for a performance oriented language, it's make me cry a little.
Yes, I can see that. And there is also a bunch of interesting patterns to learn from.
Please understand that by no mean I want to discredit your work. Rust is a great thing, and will be amazing. However, a random programmer alone in the jungle with only the rustc and the doc will had a hard time to express whatever he likes. It will get better, I trust you. Just don't let the hype grow too fast as Rust is not ready yet, not only in features but also in accessibility.
Do you have the source released somewhere? I'm interested in Rust but haven't really written anything large - I would find it interesting to see what a large project looks like, especially the kinds of problems that come up (like you mentioned).
Here are two Rust projects I did back at the 0.8 time frame. I tried to make it more read-friendly than clever hacks. Some of the syntax might be outdated since Rust has changed quickly and I didn't want to keep updating it until 1.0 is frozen.
A memcached client lib implementing both the binary protocol and text protocol, plus sharding to multiple servers. This is simpler in scope.
https://github.com/williamw520/rustymem
> A memcached client lib implementing both the binary protocol and text protocol, plus sharding to multiple servers. This is simpler in scope. https://github.com/williamw520/rustymem
Oh, nice, I hadn't seen this before. That looks really elegant and idiomatic :)
But even if you are familiar with the patterns we used and can guess the architecture, there will be a lot of unexplained code. I don't think you'll be able to extract a lot of knowledge of it.
Anyway, you should look for io_thread.rs (I think it's the most interesting file). Look how the scope are defined (especially the use and abuse of "{}"), the alternation of &self and &mut self and random methods put out of the implementation cause rustc would accept anything when they where inside.
You can also look at the mixed use of String and str which introduce a lot of chaos. And, last but not least, bonus points if you succeed to understand the why and the how of the explicits lifetimes we had to put.
The idea of Rust memory-safety is to totally forbid two simultaneous mutable lifetime whatever the code path could be.
Whatever the code path could be. Do you begin to realize how strong this assumption is ?
Yes, the rust compiler is smart. But sometimes, it's not enough and you have to do the lifetime tracking job by yourself. It's awfully hard (even without recursion). But you know what's even harder ? When all the common code patterns you are aware of for a given task doesn't provide enough warranty for the rust compiler. You know what you want to do. It's indeed very simple (can be an inner swap in some sort of algorithm) but you can't find a way to warranty the compiler that all should be good. Because in its own sort of bizarre logic, it makes sense. Then you're trapped. And you are waiting on #rust@irc.mozilla.org an hint from the bunch of truly marvelous guys who pass their time here.
TLDR: Rust can't express 100% what others languages do (ML or C based). And in the portion where it can, you have to do a lot of gymnastic to make it compile.
And our Memcached, you'll ask ? Well. After 6 days of work, we finally make it compile, and it worked quite fine. But then we got some mysterious tasking errors when reaching 20 clients and gave up at this point.