
Rust in Large Organizations - lalaithion
https://gist.github.com/rylev/0e3c3895dcb40b6a1c1cf8c427c01b5e
======
dtolnay
Here is some context for the event at which these notes were taken, and a
summary writeup:

[https://users.rust-lang.org/t/rust-in-large-organizations-
me...](https://users.rust-lang.org/t/rust-in-large-organizations-
meeting/32059?u=dtolnay)

This was a meeting between a few employees of large organizations using Rust
(Facebook, Google, Microsoft, Mozilla) at RustConf two weeks ago to discuss
the common needs of enterprise users and how to collaborate effectively on the
language and ecosystem going forward.

------
est31
It's interesting how many organizations had problems with build.rs. Other
build systems like Make, scons, or autotools have the ability to run arbitrary
things in any step in the process compared to Rust which has a clear
separation.

There is a proposed RFC to reduce the need for build.rs scripts in many cases
[1], will likely be merged within the next 2 weeks.

[1]: [https://github.com/rust-lang/rfcs/pull/2523](https://github.com/rust-
lang/rfcs/pull/2523)

~~~
saghm
> It's interesting how many organizations had problems with build.rs. Other
> build systems like Make, scons, or autotools have the ability to run
> arbitrary things in any step in the process compared to Rust which has a
> clear separation.

I think the history behind this is that Cargo was built with only Rust in
mind; when your world is 100% Rust, you generally don't need much
extensibility in your build process. Make, scons, and autotools all seem like
they were designed to be more language agnostic from the start. It'll be
interesting to see how cargo needs to evolve to be able to support interior
with other ecosystems at the level that larger companies need.

~~~
ChrisSD
The way I heard it was that Cargo was intentionally avoiding being a complete
build system. The idea being that Make et al would still be used when
necessary so build.rs (and cargo in general) could be free to concentrate on
managing the Rust parts exclusively.

------
mmastrac
Glad to see some thinking going on in this area. We're using Rust at FullStory
and it's about 1/3 source-wise for one of our projects. A few of these
concerns echo conversations that teammates have had on Slack.

FullStory itself is somewhat polyglot (Go on the server, TypeScript/react in
the browser, Java/ObjJ/Rust on native mobile) and we've felt some of the pain
around having a consistent and reliable build environment. We've also had
internal discussions along these lines - ie: which deps can we trust from
various package managers (npm has interesting arbitrary code execution as
well!).

Building tooling to manage all these languages in a common system is a _huge_
commitment. We're actively hiring for the role of productivity engineers which
is something that I see becoming more important in orgs as they scale up, the
same way that devops/SRE split out in the 2000s.

------
awestroke
Super interesting read. The geometric growth of deps problem seems like a
natural consequence of having a good package manager. In languages like C/C++,
libraries often include all their code - there are no transitive dependencies.

~~~
vlovich123
My reading of that isn't that the problem is the large set of transitive deps
but that it's opaque to the native build system at these companies
(Buck/blaze) which means rebuilding one requires rebuilding all which is time-
consuming. I infer this from the discussion about being able to query
inputs/outputs of the build so that integration can be done more meaningfully
into these systems to rebuild just changes.

------
lacker
I was entertained by this exchange:

 _ms: would like to know how to control use of unsafe in codebase

google: grep_

~~~
mijoharas
As I got to that point I started laughing and then explained it to a non-
programmer that was with me.

My fumbled metaphor of "Find in page across a lot of pages" didn't really
land...

~~~
heavenlyblue
So what do you do if “unsafe” is in a string or a comment? How do you automate
the linting of it?

I believe that’s what MS is speaking about. Google probably simply puts this
on the shoulders of the developer.

------
kccqzy
The build system issue isn't inherent to Rust or to large organizations. Even
in a startup, we have a polyglot development with a separate build tool in
addition to the language-specific build tool. We ended up doing much of the
same things: use the language-specific build tool to generate artifacts which
are then consumed by the normal build tool.

~~~
monocasa
A slight difference is that Rust really wants to be the top level compiler.
More than a lot of languages, even C++, it really relies on link time
optimization to get the perf you'd expect.

~~~
hope-striker
How this is more true of Rust than modern C++?

~~~
SamReidHughes
Most "modern" C++ involves template metaprogramming of a sort that is in
header file, so the modern aspect doesn't stand to increase the need for LTO
that much.

I would imagine that Rust programs benefit from LTO by removing static
assertions necessary for memory safety that C++ programs simply omit, and by
unraveling functional style code that Rust programs often get shoehorned into.
Or maybe not, but that's one non-LTO that's needed.

------
nunchuckninja
Can any rust experts tell me that unsafe rust is powerful and expressive
enough to let me do anything?

I have a use case where I have two iterators on a linked list in nested loops
and the second iterators deletes some elements as it goes. Is this something I
can express in unsafe rust?

~~~
unrealhoang
I’m no expert but yes, you can do anything in unsafe as in C can do anything.
Unsafe Rust provides you access to raw pointers.

~~~
nunchuckninja
I'll add more details to see if it helps-

C:

``` node _prime= list.head;

while () { node _probe = prime.next;

    
    
      while()
      {
    
        if () //condition on prime.data and probe.data {
          probe.next = probe.next.next; //& drop stuff
      }

} ```

~~~
pitaj
You need to use four space indent to create code blocks, triple-ticks don't
work here.

------
ausjke
I tried to learn those languages, my IQ level can handle java/golang/python/c
just fine, but c++ and rust is beyond my comfort zone for sure, for c++,
getting familiar with it is OK, but getting good at it is hard, really hard,
mastering it is impossible for me. similar can be said to rust as far as I can
tell.

even python has some "tricks", try "x=256; y=256; x is y" and "x=257, y=257, x
is y" under python, or 'x=[], y=[], x is y; x=y=[], x is y' you will know what
I mean, you actually need know some internal designs to use python properly.

At the moment I feel golang has the right trade off for programmers, be it
beginners or senior developers.

~~~
kbp
I don't think that's a great example for Python; that's just knowing what "is"
is for. The fact that it works the same as == for small integers in CPython is
an optimisation showing through, but only in a place where it doesn't really
matter.

You still need to have a grasp on the difference between reference equality
and value equality without getting into anything anyone would call tricks or
implementation details (eg, after `x = []; y = []; x.append(1)`, how many
elements does y have?).

~~~
coldtea
> _I don 't think that's a great example for Python; that's just knowing what
> "is" is for. The fact that it works the same as == for small integers in
> CPython is an optimisation showing through, but only in a place where it
> doesn't really matter._

Python should have prevented checking primitives with is though -- only let
object pointers...

~~~
kbp
I believe they're still object pointers in CPython, they're just ordinary
singletons; in other implementations, they aren't even singletons. Python's
object model doesn't have primitives. Adding special rules around them would
significantly complicate the language as well as limit the implementors'
ability to fiddle with the ranges of pre-allocated instances, for minimal
benefit.

~~~
coldtea
> _in other implementations, they aren 't even singletons_

That would have been less surprising (since for < 256 the singleton-ness means
all same valued instances have the same "address" (id), so behave in equality
check as if they were plain numbers).

------
vardump
Some sort of "how-to" for Rust in large organizations would be very nice.

------
amrx431
I have to learn Rust for my new job starting next month. Any recommendations?

~~~
mlindner
How did you get hired into a place that uses Rust when you don't know any?
Also where do you work because I want that.

------
epx
I'd really like to engage in a job or project that uses Rust.

~~~
gaogao
Try asking in the r/rust weekly status update thread, in the Rust discord, or
going to a local meeting! (Here is pretty off-topic)

------
jeffrallen
Oh good lord. Reading that made me happy that (1) I am no longer in a large
organisation, and (2) I program in Go. Good luck guys, you'll need it. You've
got interesting problems.

