Asynchronous programming is hard, even when hidden behind abstractions like futures or promises.
It was cool and trendy when libevent and Nodejs came out, but now we should all have the experience and knowledge that tell us to stop doing asynchronous I/O.
Go, Hakell, or Erlang do a great job at concurrency, without the async i/o craziness.
No, this is not true and you got couple of things wrong. Erlang's model is more like an asynchronous event driven programming on steroids, while idiomatic Go is synchronous and is merely a traditional multithreading, which is known to be unusable for non-trivial concurrent problems. Here's the thing though, people don't really understand these things, they always want something easy that solves a simple problem they have in mind and always fail to grasp how much flexibility they sacrifice and how much harder or even impossible it will get for more complex problems. And believe me, if you get into concurrency you're gonna have a lot of problems that are very hard or even impossible to solve synchronously. The world is an asynchronous place.
I personally find the nodejs async much more intuitive than goroutine + synchronous code in it, but this is a highly subjective subject.
nodejs use to be pretty annoying when dealing with complex workflow in term of code legibility when it was fully callback-based, but with async/await you get the best of both world IMHO.
async/await is a huge improvement over callbacks, but you still have to make sure that everything in your code is async, else you block everything. Also, it doesn't magically works with all functions, the functions have to support async/wait.
And it's still less natural than writing sync code
Go certainly does not do traditional threading. It has Go routines, which is are lightweight "threads" which gets multiplexed on top of real OS threads. It's basically implementing async IO at the runtime level.
In term of programming interface, you deal with them as if it was a traditional thread. (with its pro and cons:
pro: all your code is sequential, it's intuitive to understand what the programmer wanted to do in a specific scenario.
cons: data-races are hiding in every corners :/)
Being a green thread with a growing stack is an implementation detail for the developer writing Go code.
Explicit async IO is the building block which allows the things you mention to be efficient, and Rust's aim is being great at low-level code, e.g. implementing the runtime systems for those languages. Another use of explicit async IO for Rust is building nice abstractions for other Rust code to use, and these abstractions may or may not be so explicit about the asynchronicity.
Like with a lot of Rust development at the moment, these libraries are building blocks, they're not the end of the story.
When the ecosystem is synchronous like the Rust ecosystem, you are basically going to rewrite everything. All your network code, all your client libraries.
Async code can not use synchronous code because this would block it, and prevent it from returning to the event loop.
This is a tedious task, and you end up with less tested, less complete code (at least during the first few years) compared to the sync libraries provided by vendors and std libs.
Event Nodejs still doesn't have ported the world to async yet, and still uses a thread pool under the hood for a number of things (e.g. name resolving), which defeats the promises of async I/O.
Synchronous code can not use asyn code either because, well, in order to get anything from async code you have to be async yourself.
Async code is also more difficult to reason about compared to classical blocking code.
The idea behind async code is to avoid the cost of context switches and the memory usage of OS threads. But they are not the only way to avoid these costs. Go, Erlang, Haskell do a great job at this, without forcing the world into async.
> The idea behind async code is to avoid the cost of context switches and the memory usage of OS threads. But they are not the only way to avoid these costs. Go, Erlang, Haskell do a great job at this, without forcing the world into async.
You're not avoiding the cost, you're just moving the runtime and language/code complexity costs around. Each of the techniques used by Go, Erlang, and Haskell to implement coroutines have trade-offs and there is two that Rust simply cannot make and still fulfill its goals: lose low overhead bidirectional C interop and add a language runtime.
Performant coroutine implementations (AFAIK) all require moving the stack pointer around which makes it very expensive to have code call FFI functions. C makes certain assumptions about the stack and invariants need to be upheld, especially when the foreign library takes a function pointer from the host language. These features require a runtime which is out of the question for a low level language.
We're definitely aware of all of this. Tokio is made by two Rust core team members and the person who wrote the most widely used async io tool; it's virtually all but provided by Rust itself. And the ecosystem is aware of this too; the other people who were working on AIO have backed tokio as well, and the Rust community in general is interested in not having this split.
> Synchronous code can not use asyn code either because, well, in order to get anything from async code you have to be async yourself.
This is solvable through a threadpool, which tokio provides. In other words, it lets you make a blue function red.
> But they are not the only way to avoid these costs
These do not avoid all costs. For example, they pay the overhead of green threads, which means that you can't interoperate with C code at zero cost. That's a price a language like Rust cannot pay.
> How will you avoid having two variants of each and every lib ? E.g. redis-rs, sync, and redis-tokio, async ?
By having one, the async one. If someone doesn't care about asynchronicity, there's always some sort of "wait until completion" functionality. (This is practically what the "normal" synchronous IO functions are doing anyway, just internally.)
This is actually what Go got very wrong. They implemented net completely synchronously, instead of doing it in an event loop and providing synchronous wrappers that communicate with that event loop for those who need them.
Having a sync interface would make it easy to use in sync code, yes. I feel that we are far from zero cost abstractions now, though.
> switch stacks
Is it because the stacks in green threads is smaller than what C would expect ?
An interesting approach taken by Go here is to avoid calling C as much as possible. They don't call the libc for system calls, for instance. This is also what allows them to switch the execution to an other goroutine just before the syscall.
> An interesting approach taken by Go here is to avoid calling C as much as possible. They don't call the libc for system calls, for instance. This is also what allows them to switch the execution to an other goroutine just before the syscall.
Go and Rust don't have the same goals. Rust has been designed as a replacement fro C and C++, that can be progressively integrated in a existing code base (like Firefox's one, or librsvg's). Go is Google's replacement for Java and Python to build independent micro-services.
Go does a great job in its niche, but won't work at all where Rust shines. They are different languages, meant for different use-cases and if people could stop comparing them every time one is mentioned, I think we've made a great step forward …
> They are different languages, meant for different use-cases and if people could stop comparing them every time one is mentioned
Are they really that different use cases? Only rust is aimed at systems programming, but it seems like it could fill the application programming role quite well, where it is competing with go.
People are welcome to use Rust for applications programming, but systems programming is where Rust brings the most to the table (memory safety without a GC was barely thought possible), and where development focus is: trade-offs are made with systems problems in mind, not application problems. This is reflected in many APIs through-out the ecosystem, which give fine control but require a lot of manual explicitness. IO is no different.
> Having a sync interface would make it easy to use in sync code, yes. I feel that we are far from zero cost abstractions now, though.
Why do you say that? Taking an async zero-cost-abstraction API and calling wait() on it doesn't magically make it more expensive. It just blocks the current thread until the async operation is done. Said operation is just as fast and zero-cost as it was before.
You had a call to read(). Now you have a thread, an event loop, a pooling mechanism, and a synchronization with the thread. That's much more system calls and cpu cycles.
If you accept that using the async call in an asynchronous nature doesn't have overhead, then you can turn it into a synchronous call by saying something like `.wait()`.
steveklabnik was not saying that it costs Rust to avoid libc, he was saying that "calling C has high overhead" (i.e. the main underlying reason for avoiding libc) is not something Rust can do, given its goals. This also means there's not nearly as much reason to put the effort into reimplementing the libc abstractions on every platform.
About the cost of calling an async function synchronously:
You had a call to read(). Now you have a thread, an event loop, a pooling mechanism, and a synchronization with the thread. That's much more system calls and cpu cycles, for an synchronous async read()
Can you explain how you run an async function synchronously, and how it has no overhead compared to calling a synchronous implementation of the same function ?
As far as the kernel-side implementation goes, IO is always asynchronous. The CPU is not involved in the actual movement of data between memory and the network interface.
When you make a synchronous syscall, the kernel initiates the operation, saves the state of your thread, and starts another one. When the network interface is done, it signals the kernel, which then marks your thread as runnable and schedules it for execution.
When you make an asynchronous syscall, the kernel initiates the operation but does not block your thread. This is usually done in the context of an event loop, which makes a synchronous syscall (like epoll_wait) when it runs out of tasks to run.
Thus, converting a single async syscall to a sync one means two syscalls: initiate the operation, then wait for its result. The extra round trip between user and kernel mode is basically free in this case because you're blocking on IO, and any logic it implements has to happen in the synchronous case anyway.
> How will you avoid having two variants of each and every lib ?
The python community is also struggling with this problem. The emerging approach, which seems to me to be the right approach, is to write the libraries such that they do not do any IO; then you can use them anywhere, with some integration. So basically it's just the principle of separation of concerns.
Incidentally, this is sort of one of the design goals of tokio/finagle: that you can write your code in a transport-agnostic way. A timeout future works no matter what protocol you want to implement a timeout for, etc.
Go has different memory layout and calling conventions in part because of its green threads implementation. It has to switch stacks to call C code because the Go stacks are small and relocatable, to make growing more efficient, which is only possible because of the GC.
> This is solvable through a threadpool, which tokio provides. In other words, it lets you make a blue function red.
I'll add that once you have this, the async/sync distinction (functions that return Future and those which don't) in Rust becomes exactly the same as fallible/infallible (functions that return Result or don't) and gets handled pretty much the same way.
Futures aren't enums. I meant that you have the ability to handle it there and then (block on it via threadpool), or defer handling (chain to the next async calls in your async function). You have this same pair of abilities with Option, which bridges the basically nonexistant gap between fallible and infallible functions.
I really dislike the "what color is your function" article, because it pushes the idea that Go's userspace M:N threading is somehow different from everything just being synchronous and using threads. It isn't. Go just doesn't have async I/O, with a particular idiosyncratic implementation of threads.
It IS different. In async code every single line of code must be async (or be very fast and i/o free), else you block everything.
In Go, you can decide that a call frame and all its descendants will live their own life in a separate thread of execution. But inside of that call frame, the code is just usual, synchronous code. It doesn't event need to know that it's a goroutine. It is just executing concurrently to other code, at a very low cost to the computer and to the programmer.
Go doesn't have async i/o by choice, it just doesn't need to. Though I'm pretty just there must be a libevent or libuv binding somewhere.
pcwalton said Golang's userspace threading model with Golang's I/O is equivalent to using "normal" threads with synchronous I/O, not that it's different from using threads and "async" I/O.
>> somehow different from everything just being synchronous and using threads
> It IS different. In async code every single line of code must be async
I think you misread the gp. He was not saying that Go code is not different from async code (what you understood), he said it's not different from synchronous code using threads.
I agree that there is no visible difference in the code, and that's exactly what I like. What's different, however, is how Go routines have much less overhead than OS threads.
It's not as much as you think, and goroutines also have significant overheads that OS threads don't. Most of the time, when people talk about goroutine overhead, they're referring to the small stacks, which are actually a property of GC--there are language implementations that are 1:1 that also have small stacks, such as SML/NJ.
> Async code can not use synchronous code because this would block it, and prevent it from returning to the event loop.
This sounds like a JS-specific issue, where you're not allowed to spawn new threads for historical/implementation reasons? Even in Python and Ruby, where the GILs prevent threads from really running in parallel most of the time, you can still use them to unblock an event loop around a long-running function.
Using threads to make sync code async defeats the advantages of doing async code in the first place.
You are writing async code to avoid threads. If you bring threads in your async code, you get the worse of both worlds: Convoluted code, and thread issues (pool exhaustion and/or thread overhead).
Of course. Switching back and forth between async and threads can be a bad sign. I didn't mean to say that you should do it all the time, rather just to put some context around this:
> You can only call a red function from within another red function.
That's really really true in JS. There's no way to call an async function from a sync function, if you need to return its result. You are Capital-S-Screwed if you need to do that.
But it's going to far to apply that absolute rule to other languages. When you have threads, it's pretty easy to mix sync and async code. It can be a bad idea, just like having a codebase that's half exceptions and half error returns is usually a bad idea, but you're certainly allowed to do it when it makes sense.
Threads and async are orthogonal. Threads allow you to do work in parallel. Async allows you to increase CPU utilization in a single process/thread.
writing async code to avoid threads
Async can be used in single-process systems.
It can make sense to use async techniques to maximize work being done on each process/thread. Working with threads and using async methods can both be complex if you don't have good abstractions for doing so. They can be used together to great effect if you have the right tools.
I tend to agree that high-level languages that are already paying the cost of a pervasive runtime (e.g. for GC) should abstract away async vs. sync operations. But Rust is a low-level language, and with that comes the expectation of explicitness and direct programmer control.
You can't really do IO in coroutines without async IO in the implementation, and (depending on your baseline) you can't make them zero-cost without something like async/await.
> They just let an other coroutines execute during the syscall.
They do this by using async IO under the hood (for network sockets at least- convincing disk-based read() calls to be async is much tricker and Go doesn't do it).
Because synchronous IO will block the OS thread. If you're multiplexing a bunch of green threads on that OS thread, they'll all stall.
That's why Go, for example, does use async IO. But it makes the use of it look like sync IO for simplicity. In the runtime, however, it is doing all the async event management so that you don't have to.
From a programmer POV this is a good thing, but it comes at the price of a runtime that has to exist. Something that Rust eschews.
Sibling comment by lucozade is good. To be more specific, it's because OS-level threads are the only way the kernel gives applications to run any code at all. When an OS-level thread makes a syscall, it ceases to run any application code until that syscall completes- it's just an ordinary function call that also switches to kernel mode.
The OS kernel itself provides both blocking and asynchronous syscalls. So if your OS-level thread is switching between several coroutines, and one of them makes a synchronous read() syscall, the whole OS-level thread blocks and can't run any other coroutines until the read completes. If it instead makes an asynchronous syscall, the kernel returns immediately and the OS-level thread can switch to another coroutine while the first one waits.
You don't have to rewrite the (vast) majority of crates to be asynchronous because almost all of them can run from a thread pool with zero problems. Worst case scenario, you spawn a new OS thread and return a future which integrates very well with Tokio. Since it's built on top of futures-rs it will play nicely with other asynchronous crates and wrapping a synchronous library is much easier than rewriting it.
It would be great to have every I/O library use OS specific polling mechanisms but that places a significant burden on library developers to not only know their domain, but have experience with each platforms quirks and limitations.
> all of them can run from a thread pool with zero problems
Until you exhaust the thread pool, in which case everything relying on the thread pool will be much longer than usual to complete.
Nodejs does this for name resolving, and it's a nightmare. Basically, with the default pool of 4 threads, it suffices of 4 slow resolves to DoS everything that needs to do a name resolve in the same process.
I hope that Rust will not have the same problem when mixing async and thread pools.
If the thread pool is not limited in size, you avoid this problem, but you lose all the benefits of async.
> If the thread pool is not limited in size, you avoid this problem, but you lose all the benefits of async.
You can't have both: either you have a limited pool and can block it on long running tasks or you don't and can wind up with a large number of threads. Go currently doesn't give you this choice and you're limited by whatever you set GOMAXPROCS to before you start your program. If you have GOMAXPROCS set to 4 and you have 4 goroutines that take a long time (not waiting for IO) you've blocked the ability to do any other work. This isn't entirely true of course because they have a runtime in-process scheduler (which adds more overhead) but you could easily avoid this particular problem with tokio by using an async DNS resolution solution, which is what Go is doing in their stdlib for you.
It was cool and trendy when libevent and Nodejs came out, but now we should all have the experience and knowledge that tell us to stop doing asynchronous I/O.
Go, Hakell, or Erlang do a great job at concurrency, without the async i/o craziness.