
Clear explanation of Rust’s module system - rkwz
http://www.sheshbabu.com/posts/rust-module-system/
======
Fiahil
There is a mistake in the screenshots: filesystem view doesn't show `mod.rs`.

To all others struggling with Rust's module system, it may seem unnatural if
you're coming from another language (like python), but once you finally
understand how it's wired it will be as simple as python's.

Consider `mod.rs` like `__init__.py`, with the added benefit of being capable
of renaming and choosing when to expose internal module components.

~~~
rkwz
> There is a mistake in the screenshots: filesystem view doesn't show
> `mod.rs`.

Yes, didn't realize it would be confusing. I've fixed the diagrams in examples
2 & 3 to include mod.rs in filesystem tree :)

------
jeffparsons
The confusion around Rust's modules reminds me of the different ways that
people learn how to use car indicators.

Some people learn that you push the stalk up to indicate your intent to turn
left, or down to turn right. And then they might eventually learn that in a
European car, you push the stalk down to turn left, and up to turn right. And
then of course some of these people get confused when switching between
different cars. Yes, I've seen plenty of people on the road who indicate one
direction and then turn another. It's... kind of frightening.

The other way of learning to use car indicators is much simpler: push the
stalk in the direction that you would turn the wheel. Of course this still
leaves the possibility that some people will push the wrong stalk, and briefly
activate their windshield wipers. But it's a much easier mistake to _notice_,
and the consequences are minor.

Back to Rust's modules. I keep seeing articles trying to offer a simple or
clear explanation of how they work, that end up unnecessarily complicated in a
way that feels a lot like the "up/down" model of car indicators. The
explanation that makes the basics of Rust modules clear to me is this:

\- Child modules must always be _declared_ in the parent module, or they don't
exist.

\- The content of child modules may be _defined_ either inline in the parent
as `mod child { ... }`, or in a file with a relative path of './child.rs' or
'./child/mod.rs'.

Did I miss anything important?

Without this basic explanation up-front, I have no idea what to do with the
stream of information I'm reading in a lengthy article on the topic -- I've
been given no scaffolding onto which to bolt all the details and examples. So
this is the "bottom line" that I would like to see "up front" in descriptions
of Rust's modules.

Other misunderstandings, e.g., around item visibility, are explained really
well by the compiler if you get them mixed up, so I'm not sure how much value
there is in mixing them in to articles about how modules are _structured_
before those two basic facts are presented.

~~~
Twisol
> Some people learn that you push the stalk up to indicate your intent to turn
> left, or down to turn right.

I was _very_ confused for a good moment, because the turn signal in my car
(which I drive in California) is on the left side of the steering wheel
(itself on the left side of the car). I didn't realize the stalk might be on
the right side of the wheel in other driver's-side-left cars until you said
"some people will push the wrong stalk".

The "push the stalk in the direction that you would turn the wheel" method is
intuitive to me.

------
nercury
My suggestion for beginners would be to write modules inline and then use the
IDE refactor command (IntelliJ plugin has that) to move them to another file.
This is the best way to "get it".

------
dbrgn
That's a great explanation! However, it misses the part where modules don't
need to be in a separate file, but can also live in a "mod { ... }" block
within the same file.

~~~
pornel
I actually like that it omitted the inline syntax. It was the source of my
confusion when I read the official Rust Book. Examples with inline syntax
don't make it clear that `mod` goes in the parent file (since there's no child
file, so you don't have the reference). "But it's the same as if you just put
the {} in another file" was like "the proof is trivial, left as an exercise
for the reader".

------
Someone
From comments, this does fill in some important blanks for some, but IMHO
(H=humble and possibly a bit harsh), it’s not a clear explanation because it
doesn’t tell all for absolute beginners.

I think you’ll lose beginners at _“These are the different ways we should be
able to consume our modules”_.

There is no description of what a module is, why they’re needed (or are they
optional?), and what that “Cargo.toml” file is about.

I happen to know (rightfully or wrongfully) Cargo is Rust’s package manager,
but does Cargo require modules (I would guess so, but reading
[https://doc.rust-lang.org/book/ch07-01-packages-and-
crates.h...](https://doc.rust-lang.org/book/ch07-01-packages-and-crates.html),
it may need packages, but not necessarily modules), do modules require the use
of Cargo (I would guess not, but then, why make a discussion of modules harder
by adding Cargo-related stuff?)?

------
flying_sheep
I read the official document several times and still not feel understanding
the module system. But I feel like an expert after reading this :) Thanks
Shesh

------
burakemir
This got me thinking about how "module" \- a language concept - is about
bringing in definitions (values, types...) into scope, but then there is
deployment, something being "installed at a version".

Modules, understood as such, are inevitably connected to the stuff that is
happening "one level above" where these things _providing the modules_ are
installed (Rust crates, npm packages).

Reaching for a language-independent word, I'd propose to call these things
"components" (slightly longer-form thoughts here:
[https://blog.burakemir.ch/2020/07/modules-components-and-
the...](https://blog.burakemir.ch/2020/07/modules-components-and-their-
limits.html))

... but those things happen to be called (or heavily associated with) "module"
in Java, C++20 and go.

------
speedgoose
Thanks for the explanation. As a beginner in Rust with experience in a few
other programming languages, I think Rust modules are indeed very confusing. I
hope they could have something better in a few years.

------
sitkack
This is the single worst aspect of Rust. Can this be fixed in say in a 2021 or
later edition?

~~~
burfog
The worst is that they keep messing with things outside the language.

It is normal to have a set of tools that you like or wish to retain. There may
be a package installer (rpm, apt, msi, etc.), a build system (plain Makefile,
autoconf, CMake, the Visual Studio project/solution files, etc.), a linker
(link.exe, the old binutils ld, gold linker, etc.) and so on.

Most of us aren't looking to replace any of that. Even if we are, the stuff
coming along with Rust is probably not what we are looking for.

Cargo is thus, to many of us, an unwelcome surprise.

~~~
StavrosK
Your comment is very odd to me, it sounds like Rust is a version of C++ and
you dislike them changing the tooling. Where did that assumption come from?

I come from Python, and it would be ridiculous if I said "well we already have
Poetry so why doesn't Rust just use that". I don't think saying the same thing
about the C++ tooling is different.

~~~
burfog
Python is not comparable.

Rust is a natively-compiled systems language that supports the C language ABI.

Python, Perl, Java, Lua, Ruby, Scheme, and other such things are expected to
bring along some extra gunk. I'm not saying that this is good, but at least it
is unsurprising.

Rust needs to fit in with C, C++, Ada, Fortran, Pascal, and all the other
natively-compiled systems languages. Using generic system-provided tools isn't
just for C. You'd likely use a Makefile to compile Fortran.

The idea is that the languages are interchangable parts with compatibility.
You can create a project that uses a different systems language for each
source file, run "make", and it all builds. There is no glue code, no
interface definition language, no wrapper, no thunk, or anything of the sort.

The expectation is just different. Nobody expects Python to cooperate with Lua
or Ruby or Perl, with source files in different languages freely calling into
each other. That is the expectation for natively-compiled systems languages.

------
Tobu
Sweet and to the point, thanks!

Also of note, aside from finding the crate root and dependent crates (these
are passed explicitly to the compiler, from Cargo most of the time), all of
this is strictly part of the Rust language, and handled by the compiler. I
find the language model really clear and explicit. I would not touch it, aside
from discouraging the use of mod.rs.

I think there's room for an article that expands into Cargo's convention-based
source model, which is a bit trickier. A package directory can contain
multiple crates: at most one library crate rooted in `src/lib.rs`, but also
the various executables that are discovered from the filesystem: src/bin/*.rs,
src/main.rs, examples, the test binary, bench binaries… These are crates of
their own, with examples and benches needing a library crate to be enabled to
access files in the src/ folder (without ugly hacks).

------
monokh
Great post. Could this get merged to the rust book? Because the guide there
was super confusing.

~~~
steveklabnik
We have re-written the book chapter many times, and we aren’t making major
changes. We also can’t accept big things like this wholesale.

I am glad it exists though!

------
danellis
> The module system is explicit - there’s no 1:1 mapping with file system

From the examples, it seems like there is exactly a 1:1 mapping with the file
system, and it's only the visibility of that mapping that is changed by the
`mod` keyword. Am I missing something?

> We declare a file as module in its parent, not in itself

So I need to declare the `models` module in `main.rs`, even though it's only
used by the `routes` module and not `main.rs` itself? To me, that seems like
it's breaking modularity.

~~~
zaarn
If it's only needed by the routes modules, you should create a folder named
"routes" and put the "models" module in there.

~~~
danellis
And then where would I declare it?

~~~
zaarn
In the module it's under?

~~~
danellis
I've no idea.

------
wrs
This seems like a counterproductive example. The author points out that the
module system ignores the directory structure for good reasons, then
demonstrates a complicated way to set up a module tree that mirrors the
directory structure! It’s just reinforcing the complaint the author is trying
to argue against. A better example would show why you _wouldn’t_ want to just
automatically derive the module tree from the filesystem.

~~~
Rusky
But the module system doesn't ignore the directory structure. It is designed
to mirror it!

Instead, the author points out that the module system ignores _undeclared
source files._

There are a couple of practical reasons that prevented us from automatically
deriving the module tree, as part of the 2018 edition changes to the module
system:

* Interaction with partial, in-progress programs. People wanted to be able to comment out `mod` declarations to temporarily exclude files from the build.

* Interaction with version control, e.g. `git stash`. Similar to the above, people want to exclude untracked files from the build when their corresponding `mod` declarations are stashed.

Notably, this is basically the same set of problems people face when using
globs in their build system. Maintaining an explicit list of source files can
be slightly tedious, but it avoids a lot of surprising behavior in the edge
cases around removing files. Rust's list just happens to be distributed
throughout the source files, rather than consolidated in a build system file.

~~~
nicoburns
I don't understand the desire to exclude files using mod. If you are including
code from that file via use, then your build will still be broken. And if
you're not, then the files would be excluded from the build anyway.

~~~
Rusky
Think about the `git stash` use case. The goal is to go from a messy, WIP
state that may or may not compile, back to a more stable, probably-working
state represented by a commit.

That messy state may include a new, partially-implemented module file. By
default `git stash` will leave it alone, because it's untracked. But `git
stash` will also remove its corresponding `mod` declaration, _and_ any uses of
it, all at once.

Now say you want to build and run. If that WIP module file gets picked up
automatically, it will probably break the build! It may simply be incomplete,
or it may depend on some of those stashed changes, or whatever else. Even if
it happens to typecheck, now you're assuming that it's all dead code- but it
may have some public API (including trait impls!), and still be included in
the build!

In fact, we already have a taste of this kind of problem, ever since Cargo
started running build.rs scripts automatically, even without a `package.build`
key in `Cargo.toml`. If you add a build script with some build-dependencies,
run `git stash`, and then build, the build script will fail to compile!

Perhaps you could argue that this is a deficiency in `git stash` that Rust
shouldn't have to be concerned with. But if `git stash` removed untracked
files as well, that would simply break other use cases instead! For example, I
often have a few extra files laying around, which I don't ever intend to
commit _or_ .gitignore- sample inputs or outputs, short-term todo lists, or
other scratch-pad-like data. Ideally `git stash` would leave those alone- but
it's hard for git to tell those apart from the problematic cases above.

~~~
nicoburns
If you run git stash with the -u option (`git stash -u`), then it will also
stash untracked files. IMO leaving uncommitted/unignored files around is
asking for trouble. It's far too easy to accidentally delete or commit them.
Why not have a prefix that you use for these files, and use a wildcard
gitignore?

~~~
Rusky
I'm perfectly aware of `git stash -u` ("by default") as well as `git add -N`,
etc, but as I also very intentionally pointed out, that has its own downsides.

That is, my own personal workflow poses essentially zero risk of deleting or
committing those files, so "quit using the file system the way I've always
used it" is no less of a stumbling block for me than "maintain `mod`
declarations" is for you.

We simply haven't found an approach that doesn't step on _someone 's_ toes, so
the one that Rust was already using from the 2015 edition stuck.

~~~
nicoburns
Hmm... would you object to `use` automatically linking in modules without a
`mod` declaration? It seems that that would work in the situations with regard
to stashing (a git stash that stashed `mod` statements would also stash `use`
statements).

------
rkangel
This is a very clear 'from first principles' explanation. The module tree and
filesystem tree picture next to each other are great.

The only step in the logic that isn't explicitly explained is why you need to
do 'pub mod' as supposed to just mod.

------
Animats
Then there's the "cargo.toml" file, which this doesn't cover.

 _" There's more than one place you have to write it."_

