Ok, I don't know rust at all, but often when I'm trying to understand a change, I look at how it affects the test cases.
Seeing the diff in the test cases should show how the API has changed, but I'm having a hard time reconciling the description of the changes with how the tests have changed.
There are several occurrences of changes like this:
In ancient Rust, the standard library attempted to provide an abstraction over the underlying threading model. The purpose of this abstraction, called a "task", was to allow libraries to make use of the standard concurrency constructs while allowing the consumers of those libraries to determine whether or not to use 1:1 threading (the typical platform thread model) or M:N threading (green threads), and it would all Just Work with no effort required.
This was quite an exciting idea at the time. However, in practice, it just didn't pan out: the compromises required to paper over the differences between 1:1 threading and M:N threading managed to almost entirely obliterate the respective advantages of both models.
As a result, Rust is in the process of removing the "tasks" abstraction and replacing it with a bog-standard 1:1 threading model (though it still benefits from all the typical memory-safe and concurrency-safe abstractions that Rust otherwise provides). All users of concurrency in the stdlib will now be using native threads, and it will be left to outside implementors to experiment with green threading libraries (see https://github.com/carllerche/mio for one example of such).
One of the consequences of this change is that the `std::task` module has been replaced with `std::thread`. Formerly, every Rust program imported the function `std::task::spawn` to use for spawning tasks. With the removal of `std::task`, the newly analogous function from `std::thread` has not been added to the prelude, and is therefore not imported by default in every Rust program. It could be added if such a thing were desired (you can see remarks to this effect in the comments on the PR itself (https://github.com/rust-lang/rust/pull/19654#issuecomment-66... and https://github.com/rust-lang/rust/pull/19654#issuecomment-66... )).
> This was quite an exciting idea at the time. However, in practice, it just didn't pan out: the compromises required to paper over the differences between 1:1 threading and M:N threading managed to almost entirely obliterate the respective advantages of both models.
I would love to read a longer article about this. M:N has quite an allure, but it never seems to reach mainstream. Erlang and Go provide it and they seem to be happy. Both languages do not focus on performance, though.
M:N is great for maximising throughput, at the expense of fair scheduling. If you want any kind of fair scheduling, make sure you 'yield' often. Hope you're using a language that helps you with this.
If you want to avoid deadlock, don't get stuck in any infinite loops and avoid OS thread synchronisation, e.g. in third-party libraries. Oh, and if you're running on a NUMA architecture, make sure you resume tasks on the same thread that suspended it, unless you like needlessly consuming your processor interconnect's bandwidth.
On the plus side, it's a lot easier to write concurrent algorithms with green threads than when you have to farm work out to thread pools. It's even easier if your language was designed for it and can help you avoid the common pitfalls above. But if you can use a thread pool easily, you probably shouldn't be looking at M:N threading.
Java tried green thread since day one in 1995, and it never worked out well and was dropped long time ago (around 2000). Go did a good job with its gorountine at certain cost, for example embed Go library is much harder than C library, so Go application is typically a standalone static binary.
It is really great to see Rust drops the mixture of green and native threads, and reduce the runtime to minimum. It will make Rust stand out in many environment that no other languages would work well (besides C).
So now, will an "unhandled" panic on a thread abort the process? Or just get silently ignored until another thread stumbles upon a call that's poisoned?
Agreed. Defaulting to native threading gives developers so much more freedom. After all, M:N threading is just implemented on native threads anyway. You can still do that if you want to. Rust isn't going to be a systems language used only for networking, so forcing a M:N threading model on it doesn't make sense.
This change was necessary to keep Rust as a general use systems programming language.
> Rust isn't going to be a systems language used only for networking, so forcing a M:N threading model on it doesn't make sense.
M:N threading decouples degree of concurrency from degree of parallelism, which is not desirable only for networking. And the old way wasn't about forcing M:N threading, it was about providing a common interface for concurrency independent of the relation of concurrency to parallelism in the backend implementation.
I can understand the idea that that abstraction may have been too expensive, but I can't get behind the idea that 1:1 threading as the only thing supported in the standard library somehow provides more freedom for developers.
Short-term it may look like there's less freedom since it's harder to use M:N threading (and 1:1 threading and M:N threading don't necessarily work perfectly together), but remember that the choice there is 1:1 threading with overhead and M:N threading with overhead.
Long term, we can have libraries that provide M:N threading with minimal overhead; and also libraries that provide 1:1 & M:N together like the old std (with the associated overhead).
That is, the programmer is getting more flexibility, just not right now (especially with Cargo, which makes it particularly easy to use external libraries). The team is trying to take a longer term view focusing std on safe interfaces to the OS primitives, which can then be used to build fancier/more abstract libraries more easily.
Having done some things with libgreen threads myself, they really didn't give anything over proper threads (especially without segmented stacks). So I am curious how this will evolve in the future.
The two different threading models have different costs/benifits. Native threads are much faster at context switching then green threads, however, native threads have a significant startup cost where as spawning green threads is nearly cost-free.
Actually, green threads are much faster at context switching than native threads. They also consume less heap space for (if it's growable) stack.
Native threads are natively supported by the OS, so managing them requires calls to the OS, the main reason why it's slow. However, it comes with a few advantages. The main advantage of native threads is that they are preemptive - they can be scheduled based on ticks (i.e. number of microseconds spent running the thread), even if they are in the middle of a loop. Also, they can be run on different CPUs, the OS takes care of all CPU flags/registers/etc., they are actual OS structures, ...
Do other OSes have an equivalent to User Mode Scheduling? It seems to allow you to get the best of both worlds, if your tasks are amenable to the model.
Google supposedly has implemented something similar called user-level threads, which they talked about at the 2013 Linux Plumbers Conference. If I recall the video correctly, they talk about plans to open source it, but as far as I know there hasn't been much movement in that space yet:
>Native threads are much faster at context switching then green threads
No, native threads have a much higher context switch overhead. The downside of green threads is the lack of parallelism, you can only use one CPU core. This is why decent languages multiplex green threads over OS threads (M:N). The mistake rust made was trying to make that distinction transparent.
Green threads also suck for other reasons. Async file I/O doesn't really work properly on Linux or OS X, so you have to use a thread pool for that, and you can't casually use any other library you'd otherwise want to use, either.
That's why a decent language multiplexes green threads over native threads so you can have your cake and eat it too. You still need the ability to choose to create a green thread or a native thread though, it is the idea of trying to make the two interchangeable that was a problem.
Quote from Wikipedia article on Green Threads: "Linux native threads have slightly better performance on I/O and context switching operations."
Some of the actual context switch in native threads is performed by the CPU, where as with green threads it is not. On top of this, the whole stack is copied upon a context switch.
Wikipedia is wrong and generally speaking you shouldn't trust it on CS matters. Or any other subject, really. Green thread context switching is much faster than native thread switching, as long as you don't do it badly. For example, on Linux, don't use setcontext/swapcontext/etc, because they do a system call to change the signal mask.
The paper[1] cited for that quote is specifically about PersonalJava 1.1.3 on Linux 2.2 on PowerPC. It may not be relevant to implementations from this century.
Yeah, I'm just trying to describe the above poster as reasonable before other readers jump on the opportunity to white-out the comment for no good reason.
There is not much to say which has not already been said. I think it is sad that so much effort is going into projects which do not advance the state of the art. First Go, now Rust — these high-profile projects draw people’s attention away from actually innovative work.
As a core contributor, you will retort that Rust has different priorities than, say, Erlang, which is built around the concept of providing N:M processes (green threads) as a language primitive. My answer to that is, why not improve on Erlang, instead of settling for improving on C?
It's understandable that you think that I'm a core contributor, but I'm really just an informed community member. I have no vested stake in Rust, and I'm not afraid to criticize the core team when they make baffling decisions (example: the tone-deaf new syntax for fixed-length arrays).
And just because Rust has a focus on pragmatism doesn't mean that it's not blazing new territory. Neither Ada nor Cyclone can boast such extensively useful statically-safe references. And figuring out how to do unboxed closures memory-safely in a language without a garbage collector is, as far as I can tell, entirely novel (though you'd have to ask a PL researcher to be certain). I expect aspects of Rust to show up in all future systems programming languages, including future revisions of C++ (as well as brand-new efforts such as M#).
It is unfortunate that you think Rust is not improving the state of the art: no other language is low-level and memory safe, with the same flexibility as Rust. Rust is certainly pushing the research envelope further than Go, and, anyway, it seems to me that improving on C is something very desirable given the recent spate of horrible bugs in C libraries, caused by memory safety violations (heartbleed etc).
Because we actually have to release something. And you can't be all things to all people. The best we can do is try to create a powerful, safe, extensible language, low overhead language with unsafe escape hatches. The hope is that this will allow for future advances in lightweight, task based concurrency in the form of libraries, as opposed to as baked-in language features.
Seeing the diff in the test cases should show how the API has changed, but I'm having a hard time reconciling the description of the changes with how the tests have changed.
There are several occurrences of changes like this:
Thread methods have been namespaced under "Thread::" and now a "detach" method is appended to each call to spawn.Can anyone shed some light?