Hacker News new | past | comments | ask | show | jobs | submit login
A Rust FFI adventure in unsafety (travisf.net)
190 points by fanf2 on Oct 10, 2018 | hide | past | favorite | 52 comments



Excellent and detailed post!

Up until now I've been focusing on Julia and Swift for projects that can use a new language, as the Rust pain/productivity ratio has looked too high.

Perhaps things are changing...correctness is certainly a worthy goal!


I've been using Rust in production for a little over half a year, and my team and I have run into very few issues. We're also calling through the FFI to an SGX SDK and a large C++ codebase, and that's been delightfully painless.

We're all extremely glad we chucked C++ and Go and switched to Rust. Rust isn't just a safer C++, it's a language that stands on it's own as extremely productive, safe, and fun to write. We combat the learning curve by giving people the O'Reilly book and the main Rust book, and we find new team members becoming productive in about a week. The Rust community is also amazing, and Cargo + RLS are the best ecosystem tools I've seen in any language.

I highly recommend Rust. If you have any questions, feel free to ask here. We're extremely bullish on Rust continuing to accelerate.


> Cargo + RLS are the best ecosystem tools I've seen in any language.

In my experience the java profilers and debuggers are still better than anything that's available for rust.

RLS is great on a fundamental level (decoupling IDE from compiler), but it still has to catch up in functionality with the total compiler integration that for example eclipse has with ecj.


We don't use Java. Java is old, of course it has plenty of tooling, but it also has a mountain of negatives in it's own right, besides the fact that it's not suitable for systems level work.


I guess it is a matter of what kind of systems one is developing, given its use in embedded platforms.


You switched from C++ to Rust and don't use Java because Java is old.

You've made at least one person's mood a little lighter today. Thank you.


You misread. Java being old is counted as a positive here; Java is old, so it has plenty of tooling, is the message. Despite this, they are not using Java :)


Yep, Rust still needs to do a lot to catch up with Java and .NET eco-systems.


Not really, like I said, java isn't really suitable for systems level work, or a good choice for FFI to C/C++. There are also a lot of disadvantages to using Java. Java lacks some of the compile time safety features of Rust. It has high level features like Sum types that Java does not have. The owners of Java have also sued reimplementors as well, which I think should give anyone pause about using it to do business with.


As I mention in another thread, it is a matter of what kind of systems level work you mean, given its successful use in embedded platforms, namely military, office devices, manufacturing automation and IoT.

Kotlin and Scala have Sum types, and Rust still needs to catch up with Scala's type system.

The owners of Java only sue reimplementors that don't respect licenses, like Microsoft and Google.

None of other commercial third parties has had any issue with Java owners.

https://en.wikipedia.org/wiki/List_of_Java_virtual_machines#...

Naturally I am not advocating for you to use Java instead of Rust, rather that it also has its uses in system work, and it is a valid option in scenarios where having a GC enabled language isn't a problem.


Rust is amazing (but also frustrating), but RLS is, frankly, terrible. It is unreliable, slow and unstable.


As of now, the Rust IntelliJ plugin combined with CLion provides the absolute best Rust IDE experience (it even gives you a visual debugger that works as it’s supposed to...)


When you switched was it a unanimous decision to write rust? And is this a small team where you all have the same opinion? Or did you have to drag any one along? From what I have seen people who aren't motivated to learn rust struggle. I'm curious how that worked out


There was some convincing, we had to sell why we felt Rust was worth rewriting some of our tools in, and that it was ready for primetime. An initial small group of us spiked and wrote integrations for our logging, tracing, and wrote a Rust compiler for our rpc IDL. After that, we kind of got some shocked reactions as to how we were able to move so quickly and that things actually seemed to 'just work'. We also kept evangelizing, giving internal talks, and just generally pushed the snowball until it gained enough momentum that it became clear that Rust was a win.


Did you lose any developers during this period?


No, in fact it's been a selling point in our hiring. Rust seems to jive with a wider audience, it has aspects of FP, C++, and the tooling solves a lot of pain points for more seasoned developers and has energized some of our employees who felt burnt out on pushing our older services forward.


What domain are you working in?


I am learning Rust now, in an attempt to redo some Python scripts that run too slow (using threading). I was going to go with C (worked with it for years back in the 90's), until I stumbled upon Rust.

Rust is so easy to install and get up and running on any platform. But I am getting increasingly frustrated with documentation of many crates I try to use which were so easy in Python. For example serde, log4rs, clap (docs are pretty good).

For example, I can not find any way to change the log level in the log4rs implementation at runtime, or how to configure formatters with the yaml file.


Would you have any sort of stats on your productivity with C++ vs. Go vs. Rust?


It's hard to measure productivity, but one thing I'd like to point out about Go is that is has relatively few productivity feature. It's main advantage is that's simple, but with automatic derivation, traits, closures, real enums (sum types), Rust actually provides more high level niceties than Go. We can also encode rules using the richer type system of Rust over Go or C++, things like state changes can be verified at compile time via phantom type for example. Expression based programming coupled with the borrow checker allows us to write expressive code that is also efficient.

I'm not going to claim that we have proof the Rust is more productive than Go or C++. I see some concrete advantages, however, Rust may simply better suited to the work we're doing. We might just enjoy writing it and therefore work faster.

The way I see it is that Rust guides you to better architectures, it offers a lot of very powerful features from FP and PL theory, and it's all engineered to make sense as a language. In that way, I think it's similar to Go in that they're both engineered while C++ has a lot of fossilized layers and has been organically grown for decades now. We have C++ code from half a decade ago that looks nothing like what we are writing now. It's this organic beast that has a lot of baggage. Rust and Go are engineered from the ground up, and with Rust editions I think they will avoid becoming the organic beast that C++ is now.


It probably depends greatly on what you are doing. There is a lot of work where I would be much more productive in Go because I have yet to really master Rust, as it is a big and complex language while Go is so simple. But I have also made a library where the various meta-programming features in Rust saved me thousands of hours of work and ongoing maintenance compared to what it would have taken to recreate it in Go.

Similarly, the productivity difference will depend on how much run-time performance you need. It might be take longer to make Go code very fast if you need very fast. Having to work around the garbage collector, and avoiding interface {} etc.


> We have C++ code from half a decade ago that looks nothing like what we are writing now.

well, Rust code from half a decade ago is also somewhat distinct from today's rust.. even basic cases don't compile anymore : http://progopedia.com/language/rust/


Half a decade ago, Rust 1.0 didn't exist. The language was explicitly unstable.


I'm very skeptical that one can transition a team to another language and have everything go perfectly.

To summarize:

* learning curve successfully flattened to a week by... reading two books.

* painlessly interacting with a large C++ code base

* no team member dislikes the change, instead they're feeling energized and this has even brought back some from near-burnout.

* Rust is extremely productive

I can accept that the tooling's good, community's nice and type system's a pleasure to use, sure. But people don't change so easily and such drastic changes in large code bases don't go right by default.

If all of what you said above were true, every C++ codebase would have been switched yesterday.


Out of curiosity, what are you building and with what company?


What do you use as your development environment?


vim + w0rp/ale, ale has LSP support and I now use it as a one stop shop for go to dec, find refs, and autocomplete. It has support for linting with cargo, rustc, rls, and autoformatting with rustfmt. rustup makes installing rls extremely painless.


mind sticking up the relevant sections of your vimrc somewhere so I could take a look?

the rust stuff in my vim dates to v. early post 1.0 and would love to link ale + rls up.


Sure, install w0rp/ale using your favorite plugin manager, install rustup and run:

  rustup component add rls-preview rust-analysis rust-src --toolchain nightly
then add this to .vim/ftplugin/rust.vim:

  let b:ale_linters = {'rust': ['rls','cargo','rustc']}
  let g:ale_fixers = {'rust': ['rustfmt']}
  let g:ale_completion_enabled = 1


perfect, thanks!


IntelliJ IDEA CE is way better than VS Code, IMHO. Much more reliable in the type inference.


In my experience, the majority of the community uses VSCode with the RLS plugin or IntelliJ with the rust plugin (which is, ironically, written in Kotlin)


Can you link to the aforementioned books?



Yes, those are the sacred texts. I do believe the O'reilly book is worth paying money for. It's extremely well written, and I think it's painstakingly crafted pictures of memory layouts and thorough coverage of lifetimes and traits are tremendously helpful.


Thank you. Well, now I have to choose whether I prefer the physical copy or the Safari subscription... :D


For me, the ebook version on my laptop in Calibre has been great; no regrets buying that, instead of the dead tree version.

It's a nice complement to the open-source Rust book too, which may as well be read in digital form.


The correctness I'd argue is worth it.

My company has been shipping a service in rust for >1 year and its really nice knowing when the tests pass, and the app gets deployed it'll do exactly what it needs to in perpetuity. Once tests pass locally pushing to master doesn't require a second thought, if it ever does it means local tests should be updated.

While there is a non-trivial spinup learning to work in the language, and unlearning bad habits from other languages can be frustrating (especially coming from C). The performance, without losing correctness is wonderful.


I'm studying rust. In the past, I've used C when I wanted or needed to get close to hardware; even short snatches of assembler. I hope to never again have to write a non-trivial C program and make it thread-safe. Rust looks promising. The pain/productivity of rust should be high; it's trying to let you work as low-level as C, while helping you achieve a "single mutator at a time" safety model. I think of Julia and swift as higher-level languages, more domain-specific. Different tools for different problem domains.


This was exactly the case that got me into Rust; I had something that needed to be fast and needed to be callable from a variety of languages & environments (from python, C and Java immediately, and expecting to add swift later) but it also had to do a lot of unicode string handling, and I just _really_ didn't want to do it in C. After making sure that everything I needed was possible, I did it in Rust and it was an absolute pleasure; I haven't looked back.


> I think of Julia and swift as higher-level languages, more domain-specific. Different tools for different problem domains.

Well...my particular interest in these types of languages is primarily in the realm of soft real-time simulation, specifically various kinematic simulations.

Of the three, only Julia is garbage collected, and (unlike some other GC languages) it's fairly easy to not exercise the collector. I'm encouraged that will continue to be the case, since there's an organization using it for robotics, which is implicitly a hard real time use case.

http://www.juliarobotics.org/

Julia, Swift and Rust are all clearly general purpose languages. Swift is unabashedly general purpose, while Julia and Rust each have a primary niche - math/science and systems, respectively. All three use the excellent LLVM infrastructure.

Aside from determinism (which mainly requires pre-allocating nearly everything), my primary requirements are expressiveness/productivity, readability, and efficient runtime performance.

All three languages produce highly optimized code, and Rust probably has the edge as far as efficiency goes - but it clearly loses on the first two criteria, at least to Julia. If one needs access to machine level functionality in Julia, there's an extremely efficient C FFI, so mixing Rust and Julia (for instance) would be painless if needed.

https://github.com/dyu/ffi-overhead

It's a great time to be a software developer, and things will only get better as languages and tooling continue to improve!


From my dabbling experiences with Rust, I would say that it should mostly be considered for scenarios where a GC is absolutely no go.

If the application deployment scenario can handle typical GC workloads, then using a programming language with such support is more productive.

One example would be modern GUI toolkits.


GC doesn't factor into my use of rust, really. It's definitely a nice benefit, but not the main motivator.

I use it because it's easy to write code that I can come back to later and understand, and have confidence in.


> We wanted a primitive that would guarantee that initialization of the global variables would happen exactly once.

IIRC, At one time, there was a "lazy_static!" that fit this use case? But maybe you can't do that here if what you need is the side-effect of the library call.


Also, there is `once_cell` that seems to be a more general form

https://crates.io/crates/once_cell


lazy_static is a wrapper around std::sync::Once, mentioned in the post. I'm not sure if it would be a better fit for the OP, as I haven't read their code.


See, this is why you need the lines above and below the letter I. I thought this was a Rust remake of Final Fantasy 1. So disappointing...


> the lines above and below the letter

Serifs is the word you are looking for.

Hmm, from examining the font style in the stylesheet, and playing around it looks like Verdana is preferred, and then it falls back to Geneva, and then some default sans-serif.[1] Verdana has serifs though, and Geneva either doesn't or I don't have Geneva, so that seems an odd choice of preferences and fall backs.

1: The CSS is "font-family:Verdana, Geneva, sans-serif;"


Looks like you've outnerded me. Well done.


Hmm, what was lost in revising that comment was the sentence noting that I have swrifs on every device I view HN on, so I was wondering what you were viewing it on. It seems odd that it would change that much between devices when explicitly defined with fallbacks, which is why it's an odd config.


Don't the Final Fantasy games use Roman numerals for numbering anyway?


I guess you're right. Maybe I just have FF1 on the brain. Either that or 3 remakes wasn't enough.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: