
The Rust module system is too confusing - AndrewDucker
https://withoutboats.github.io/blog/rust/2017/01/04/the-rust-module-system-is-too-confusing.html
======
awinter-py
In keeping w/ rust's philosophy of being 'c++ with marginally better tooling',
I'm happy with the current setup. It's light years ahead of the perf overhead
of #include + the pain of setting up linker args for libraries.

Are the keywords idiosyncratic? Maybe. Can a new user get it working in an
hour and learn it in 2 days? Yes.

It would be cool if there were a polyglot build system that you could learn
once and have the same expectations in every language. But there are downsides
to this proposal (not least of which is the fact that when someone names a
polyglot build tool, it's often NPM). I doubt the module system is the #1 or
even the #5 thing people struggle with when learning rust.

~~~
pjmlp
> It's light years ahead of the perf overhead of #include + the pain of
> setting up linker args for libraries.

Sadly not yet.

My VC++ builds are still faster, because they can enjoy using binary libraries
across projects, something that cargo doesn't yet support.

Also VS 2015 and the new VS 2017 do have quite a few improvements regarding
the linker, incremental builder and C++ modules support.

~~~
pcwalton
> My VC++ builds are still faster, because they can enjoy using binary
> libraries across projects, something that cargo doesn't yet support.

This isn't anywhere near the top issue that I (or anyone else I know of)
encounter with build performance in Rust. Dependent library builds are very
parallelizable, so they tend to build quickly. Moreover, the build performance
that matters in the day-to-day cycle is mostly _re_ build performance, and the
binary artifacts are cached in this case.

We could spend time adding a global cache of build artifacts, I guess, but all
the time we'd spend working on this wouldn't impact my development velocity
much at all. Incremental compilation, on the other hand, would.

~~~
pjmlp
Every time a new Rust release comes out, I try to update all the VSCode
relevant plugins.

It takes me more than one hour on a core duo with 8 GB + HDD, while I
occasionally see the same crates being compiled and get to do something else.

I could probably buy a new computer, as this one isn't something that one can
easily parallelize, but why should I when my C++ builds are actually fast
enough?

I am just an opinated Rust dabbler, please take my complaint as contributing
feedback about what I see as one possible show stopper for Windows developers
to adopt Rust, vs our current .NET + C++ workflow, in terms of productivity.

Thanks for Rust.

------
jonathanstrange
IMHO there are many obscure things about Rust for newbies, and its syntax is
also less readable than that of Ada. This gives me high confidence that it has
a bright future.

I'm not being sarcastic here, Rust seems to be a great language (definitely on
my list to learn), I'm really just observing. Looking at the success and
history of programming languages it seems to me that many if not most
professional programmers _want_ some obscurity, a desire that is often
disguised as an apparent need for brevity which in reality plays no role with
modern editors.

Maybe this makes sense, after all it's better to have a language that an
incompetent boss or a marketing department cannot just pick up easily to 'roll
their own solution'.

~~~
JulianWasTaken
Which evidence do you see that points to obscurity or partial obscurity
leading to success (or adoption, I can't tell which you're claiming)?

~~~
jonathanstrange
Not evidence, only subjective gut feelings, and I've had adoption in mind. But
I'm hesitant to answer this question, because it will invariably lead to
language flame-wars. Anyway, it seems to me that Algol68, Pascal, and Ada are
less obscure than C and C++, but the latter are clearly more popular and have
proven their worth over a long time.

At the same time, implicit-block type procedural languages like Visualbasic
and Realbasic/Xojo _are_ popular, but they do not get much love from
professional programmers, even though they are maybe the least obscure among
all imperative mainstream languages. LISPs are also an interesting case,
S-expressions are among the least complicated and easiest to grasp syntax and
CommonLisp offers practically everything a programmer could ever wish for
(including multiple dispatch) but it never became very popular.

But maybe I'm wrong. For example, advanced Lisp macrology is definitely
obscure, so that should count in favor of Lisp being more popular according to
me 'hypothesis'. Haskell syntax is fairly easy, but monads are certainly
obscure, yet Haskell does not seem very popular. Likewise, Python does not
seem to be very obscure, it certainly isn't syntax-wise, and it's still very
popular.

So I don't know, perhaps my assessment was premature. It almost certainly was.
;-) I've just always had this impression that many programmers prefer a
certain amount of special trickery and shortcuts over clarity. But perhaps
it's the availability of libraries what counts most in the end.

~~~
fpig
C is FAR less obscure than most of the languages you mention, ignoring things
newbies generally don't need to worry about like standards, compiler
implementations, memory alignment, virtual memory, etc.

I was able to learn C reasonably well as a child precisely because of how
straightforward it is. You can reserve a block of memory. You can free it. You
can get the address of a value. You can get the value from an address. It
works in an extremely straightforward way, unlike most languages. To a child,
programming in C feels like the computer is doing exactly what it was told to
do and nothing else. It _feels_ like you understand what is going on, even
though your understanding is a very simplified model of what is actually going
on.

Modern languages like C#, Javascript, Ruby... They are much harder to
understand than C. The first time I saw a closure, I was completely
dumbfounded. This doesn't make sense, a function is accessing a local variable
of another function that has since been popped off the stack! What sorcery is
this?? Even today things often confuse me, when I saw C# async/await for the
first time I just couldn't understand how it can possibly work until I spent
some time reading about it and I had been programming for decades by then
including a decade in C#.

~~~
Manishearth
> This doesn't make sense, a function is accessing a local variable of another
> function that has since been popped off the stack! What sorcery is this??

This is an issue with learning a _new_ language when you've already got your
mind molded to a different one.

JS programmers have the exact opposite issue when learning C. Why can't we do
this? Why must we worry about scopes for the stack? Wtf is the stack? I wonder
what happens if I try to use this variabSEGMENTATION FAULT.

These languages generally are dealt with differently in one's mind, and trying
to apply the mental model from language 1 to language 2 usually makes it feel
harder than it really is.

I really could say the same about "it feels like the computer is doing exactly
what you told it to do" about the other languages you listed. A crucial part
is that you don't actually need to _understand_ what the computer is doing
under the hood to use it, especially in a higher level language. C is more
like "you were forced to talk to the computer in its own language", whereas
with JS/C#/etc you don't have to know what addresses and memory are. You just
tell the computer what tasks you want done, and how to model them, and it does
them.

So I'm very skeptical that the other modern languages you listed are harder to
understand than C. They're just _different_.

~~~
fpig
The only problem is, I've used C# and JS way more in my life than C, and C
wasn't even my first language (QBASIC was).

So unless learning C somehow causes permanent disability I will have to
respectfully disagree.

~~~
Manishearth
Right, my point is that it's highly subjective and depends on how you approach
things. I know programmers who have learned C later and they've always found
the whole having-to-deal-with-memory thing an unnecessary distraction that
they spend way too much time wrangling with, even after spending lots of time
with C. I'm one of these; I have done lots of C programming but find languages
like Python to be easier to learn.

C sort of does cause a disability (not really a disability, this is a useful
skill!) in the sense that now you're actually thinking about how a language
feature works in the hardware, which is not something you necessarily _need_
to do for higher level languages. The moment you _ignore_ all this it's likely
that the language will make sense again. Of course, like I said, this is all
subjective. The solution to "I can't use async/await because I don't
understand how they work" is basically that you pretend that they're magic,
and try to figure out the internals on the side. You need not understand how
things work under the hood to _use_ them, though it is an extremely valuable
skill to know what C#/JS/etc end up doing at the lower level.

It's a sort of "ignorance is bliss" situation.

It varies from person-to-person though. It's not specific to programming
either. When I was a physics student I personally would be easily able to
define layers of abstraction in my mind and work within one layer, even if I
didn't understand another -- you ignore how the other layer works and assume
that it is magic (while simultaneously working to understand the other layer).
One of my friends could not do this -- he _had_ to understand rigorously how
the theorems worked all the way down. As you can imagine, this required a lot
of effort. It meant that he always had a thorough understanding of the topic
at hand, but often meant that he spent a lot more time to understand something
and had no middle ground to choose.

For me, this has carried over to programming. If I'm dealing with a language
like C or C++ I _will_ try to understand how everything works on the metal.
When I first learned about C99 VLAs I spent an hour mucking with a debugger
and inspecting the stack. When I realized that C++ had function-local
initialized statics I spent many hours digging through C++
spec/implementations to understand how it worked
([http://manishearth.github.io/blog/2015/06/26/adventures-
in-s...](http://manishearth.github.io/blog/2015/06/26/adventures-in-systems-
programming-c-plus-plus-local-statics/)). These are things at the same
abstraction level as C/++ which I must understand to be able to use them.

But when I'm dealing with languages like JS, I can operate perfectly fine
without knowing how they work. Indeed, when I first learned about generators,
I was able to use them for quite a while before I learned how they work. I
still _want_ to know how things work (and generally dig into it), but it
doesn't block me from proceeding if I don't.

This is not how everyone approaches languages. Like I said, it is subjective.

------
ekidd
The Rust module system is a little bit weird. As far as I can tell, this is
the result of two things:

\- 'use' statements are relative to the root of the crate, not the current
module. To work around this, you can write 'use self::name' or 'use
super::name' for relative imports. Annoying but no big deal.

\- If you import a symbol into module 'a', it doesn't also get added to
'a::b'. I don't know why I keep expecting this.

Once I learned these two things, Rust imports were easy.

I'm not sure I agree with withoutboats' proposals. His chnages would make
several obvious things work the first time, but at the expense of taking a
simple, easy to explain rule and replacing it with something implicit and
mysterious.

The bigger Rust learning curve issue is making friends with the borrow
checker. I like the borrow checker. It has my back, even when I'm trying to
write fast code that would be recklessly abusive of pointers in C. But I can't
deny that it took me a week or two to make friends with the borrow checker.

~~~
lucaotta
_' use' statements are relative to the root of the crate, not the current
module. To work around this, you can write 'use self::name' or 'use
super::name' for relative imports. Annoying but no big deal._

I'm writing an application. Inside main.rs I can write

    
    
      extern crate foo;
      use foo::Bar;
    

If I create a module and use the same crate, I need to write:

    
    
      extern crate foo;
      use self::foo::Bar;
    

This always surprises me and I never found anything explicit written on the
Book. To me, it looks like a nice papercut to fix.

~~~
gliptic
What if I really do want the 'foo' crate to be inside the sub-module? If you
want it in the crate root, you should put it in the crate root.

~~~
lucaotta
Sorry, I don't really grasp your comment. Maybe I can clarify what I'm trying
to do.

My typical use case is importing definitions, for example an error type or
enum from the crate, let's say something from tokio_core.

I'd like to use this enum both in main and in my module; from my (limited)
experience, you need to _extern crate_ tokio_core both in main and the module,
then _use_ the definition. In this case the syntax is different for somewhat
obscure reasons.

~~~
carols10cents
Hm, I'm still confused about what you're trying to do :-/ If you're trying to
use something from tokio_core in your own crate, in neither place will `use
self` do that, like you had in your first example... it would be `use
tokio_core::foo::Bar`... and the `self` was the only difference between the
binary and library, so I'm not sure what the paper cut is exactly.

~~~
lucaotta
Basically I start hacking on a project, then when finally something is
working, I want to split it into a separate "something". Coming from the C++
world, I'd create a new cpp/h couple, maybe in its own subdir. I understand
that the equivalent in Rust is creating a module.

I made a simple example with just two files: (In the meanwhile I realized that
my first example was wrong :)
[https://github.com/lucaotta/rust_modules](https://github.com/lucaotta/rust_modules)

Let's say that after my refactor, I've "moved" the crate import into the
module since "logically" it belongs there. In this case the compilation fails
if I don't use _self::_. But previously it was working in main.rs! And I have
no idea why...

~~~
steveklabnik
This is the "use starts at the crate root" thing. If you move the "extern
crate", then the place it ends up in the module hierarchy is different, and so
it breaks.

I always leave "extern crate" in the crate root; then it all Just Works.

------
marcoms
For me what's quite unnerving about `extern crate` is that the `extern`
keyword has another function, that is to declare foreign functions [0].
`extern crate` on the other hand, is not an extension of `extern` but serves a
completely different purpose.

0: [https://doc.rust-lang.org/book/ffi.html](https://doc.rust-
lang.org/book/ffi.html)

~~~
neikos
I fail to see how the usage is different? They both reference things that are
'external' eg. already compiled etc...

~~~
marcoms
I mean if anything the syntax should me more specialised as you go into sub-
keywords, which makes sense in this industry (e.g. ISO 8601). The problem is
that `extern` seems to have been taken from C++ and extern crate is almost
tacked on (or that's the impression I get). Comparing with other languages,
Rust looks to be the only instance where two keywords are required to simply
require a module/library, which sticks out to me.

------
the_mitsuhiko
The main issue with Rust's module system is that your own modules and crates
live in the same namespace. So if you have a `mylib/image.rs` file and you
want to use the image crate then you can't without either renaming your
`image.rs` or you alias the crate on import.

I really wish your own modules would require some relative import and be
scoped within `mylib`. Eg you would do `use mylib::image::MyImage` and `use
image::OtherImage` instead of `extern crate image as image_crate; use
image::MyImage; use image_crate::OtherImage`.

~~~
throwawayish
So Rust essentially has the same design issue that Python had before PEP 328?

~~~
Manishearth
Not exactly; Python had the problem where `import foo` had two different
meanings and there was no way of working around it. Rust still understands
that crates are different, and will let you do `extern crate foo as bar` to
avoid conflicts.

------
lhnz
This was my experience, too. I had to keep checking other crates to see what
was expected from me, and managed to finally understand through trial-and-
error.

The problem with this is that I'm not likely to remember how modules work a
few months after finishing a Rust project. I reckon core aspects of a language
should be so simple that they don't need any work memorising them.

~~~
nickez
Mine too, or rather, for my first rust project I initially had all the code in
main.rs and then I found it very difficult to split up the project into
several source files. I think with some better documentation it wouldn't have
been that hard.

~~~
mathw
I think it's the documentation. It took me a few tries to get everything I
needed out of the Book to use my own modules, but then it was easy.

Although I still don't quite get how macro importing works. It feels like when
you #[macro_use] on an extern crate you get all the macros implicitly
everywhere without having to issue any use statements. This seems weird and I
have yet to try to find an explanation of it. But I've got modules all over my
Advent of Code solutions which use macros from nom without having any nom-
related use statements.

~~~
steveklabnik
Yeah, I hear you. I've always thought this, and honestly, I think the issue is
that I've always found the module system _very_ intuitive, so it's been harder
for me to connect with how people struggle to learn it.

[http://rust-lang.github.io/book/ch07-00-modules.html](http://rust-
lang.github.io/book/ch07-00-modules.html) is the second edition of the book's
chapter on it, I think it's much better.

> This seems weird

Yes, that is how it works. Macros aren't in a namespace. Yes, it's weird. It's
an artifact of history; eventually, it will be fixed.

~~~
mathw
> [http://rust-lang.github.io/book/ch07-00-modules.html](http://rust-
> lang.github.io/book/ch07-00-modules.html) is the second edition of the
> book's chapter on it, I think it's much better.

It is better, yes, although I think it takes a while getting to the point in a
few places.

Docs are something I'm interested in, so you might hear more from me through
appropriate channels about this and that.

> Yes, that is how it works. Macros aren't in a namespace. Yes, it's weird.
> It's an artifact of history; eventually, it will be fixed.

I'm glad that I both got it right and that it's not necessarily going to stay
that way. I don't know how much macros are going to proliferate in the future,
but I don't want to find myself with macro naming collisions. I can certainly
imagine some of nom's causing issues with other crates, and having to define a
new crate within my tree just to contain them would be irritating. I assume
that would work as a workaround, anyway.

~~~
steveklabnik
Great. Please don't hesitate to reach out and or file issues or whatever, I
really want to make this stuff good.

Basically, today you define macros with macro_rules!. This shipped in Rust 1.0
because we did not have the ability to invent and implement a more proper
macro system, and didn't want to hold the whole language up on one thing. In
the future, you'll write macros with a "macro" keyword, and macro_rules! will
be deprecated, and "macro" macros will support namespacing, etc.

~~~
mathw
Is that "Macros 2.0", or something beyond that?

------
tempodox
> extern crate is like mod, but it declares an external crate dependency
> instead of a submodule.

Naive question: What the hell's the difference? Is a crate not a module? IMO,
there shouldn't be a difference.

(EDIT for clarity)

~~~
awinter-py
crate is what gets released / distributed (i.e. a whole 'library', 'package',
'dependency', 'bundle'). Modules are single source files.

~~~
tempodox
A binary is of course something different from a source file. But that's just
a detail. The “user interface” should be the same for both, except maybe in
the Makefile / Cargofile.

~~~
awinter-py
crates aren't binaries -- they're source bundles.

~~~
tempodox
Oops, my bad. That makes the distinction even more baffling to me.

~~~
ben0x539
To be clear, `extern crate foo` refers to the compilation artifact, not the
source of foo.

------
nercury
Rust module system is similar to C#: mod is namespace, extern is assembly
reference. The difference here is that there is no separate "project" file, so
all references to inner module files need to be described in the parent file.

------
charlieflowers
FYI, /s/bare/bear

~~~
awinter-py
Well, maybe. Or you're viewing the article through your western cultural bias
and missed the author's primary intent.

~~~
charlieflowers
"bare with me" can only make sense if the author is asking us to remove our
clothing along with him. I don't think that was his intent.

Telling him about the typo was meant as a courtesy to him, because I
appreciate his post and want to help strengthen it in this small way.

~~~
awinter-py
you're assuming he wants to bare his body. maybe he wants to bare his soul.

------
cousin_it
I think it's possible to design a more intuitive system with less busywork:

1) For imports, the namespace hierarchy should have a one-to-one
correspondence with the file and directory hierarchy.

2) For publishing code, instead of modules and Cargo, there would be one huge
worldwide repository. You'd be able to freely import anything marked public,
and the build system would cooperate with version control to make things work
seamlessly.

Unfortunately it's probably too late for Rust to adopt such ideas. I wish some
new language used them, though.

~~~
steveklabnik
Given that crates.io has no namespaces, this is kinda sorta how this already
works, I think. Or at least, I don't understand the distinction.

~~~
cousin_it
Well, you have two tools that can fetch stuff - git and cargo. In my preferred
workflow there'd be only one tool. It would also work as a cloud filesystem,
like p4fuse.

~~~
steveklabnik
Cargo can use git, but it doesn't primarily. Crates are stored as tarballs on
S3.

~~~
cousin_it
Right, I don't see why that's necessary. The whole world could be one big
source control repository where each client keeps only their local changes. So
e.g. if you want to make a local tweak to a third party library you're using,
you 1) edit their file in your filesystem which is a transparent view into the
world repo, 2) recompile your project.

~~~
steveklabnik
That forces everyone to use git, which is not something we're interested in
doing.

~~~
cousin_it
Agreed, it would force everyone to use the same VCS. I've lived in this kind
of setup for years and I'm very happy with it, but point taken.

------
tveita
This matters less once we get good IDE support. E.g. "mod" should be inserted
automatically when you create a new submodule, "use" will be inserted as
necessary if you try to use something that is not in scope.

The other part is to have a set of conventions. Do I put "extern crate" in the
root module or in the submodule that uses it? How do I order my use
statements? Do I import just the module or each item in it? When the community
settles on a standard way, IDEs can follow it.

~~~
sidlls
Unless an IDE is part of the standard somehow I strongly disagree that "IDEs
make it better" is a good argument for preserving a language feature that is
confusing. Not everyone uses an IDE, and for certain low level systems
programming kinds of things (you know, the things Rust wants to target)
sometimes an IDE isn't available or is cumbersome to use.

That said I am a bit confused at the confusion. Rust is better than C++ in
this regard, in my opinion, even if the system is a tad slower to build still.

~~~
Manishearth
Agreed. This should be fixed in the docs, diagnostics, and maybe by changing
the language in a way withoutboats recommends, but we should not rely on folks
having an IDE.

------
zyxzevn
They should have looked at the good module system of OBJECT PASCAL (derived
from MODULA). It is easy and allows all kinds of protections.

The main difference is that in this language a module always is one file,
which consists of 2 parts. 1\. INTERFACE (public) 2\. IMPLEMENTATION (private)
The way it is organized makes compilations also very fast.

USES only uses the public parts of the module. A LIBRARY is like crate in
rust.

Sadly, this module system does not use private classes/types in public
functions. But that would not be hard to add.

I have no idea what Rust tried to accomplish. Being used to Object pascal many
other module systems seem primitive.

~~~
pjmlp
Actually that model came originally from USCD Pascal and was adopted by Apple
for Object Pascal, but Borland did improved it quite a bit.

You forgot to mention, cyclic dependencies were allowed on implementation
part.

Also later on, Delphi introduced packages, which are a set of units, with just
some of them being public. Similar to the upcoming Java 9 package model.

------
Ericson2314
All you need to do is make `use` paths look absolute with a leading `::`,
followed be either name a crate or `create` for the current crate.

