
Await in Rust - luu
https://docs.rs/dtolnay/0.0.3/dtolnay/macro._01__await_a_minute.html
======
divan
Reading through comments on async/await-related articles, I wonder if I'm the
only person who find the whole concept of async/await utterly weird.
Specifically I have troubles embracing the need to mark function as "async" –
it just doesn't make sense to me. Synchronous or asynchronous is something up
to the caller to the decide – not to the function itself.

Like in real life I can do something synchronously (meaning paying full
attention), or I can do the same task asynchronously (watching youtube videos
in the background while cooking, for example), but it's still up to me. Being
"async" is not a property of the "watching youtube" action, it's the property
of the caller running this action.

That's the reason why CSP-based concurrency models really works well for me –
it's just so easy to map mental models of system's behaviour from the head
directly to the code. You have function/process and it's up to you how do you
run it and/or how synchronize with it later.

Async/await concept so popular in modern languages is totally nuts in this
aspect. Maybe it's just me, but I find myself adding more accidental
complexity to the code just to make async/await functions work nicely,
especially for simple cases where I don't need concurrency at all, but one
"async" function creeps in.

~~~
PJDK
Thing of it as a different analogy. There are various chores to do around the
house - some of them must be done synchronously (do the ironing - we don't
want to leave the iron on). Some of them can be done asynchronously (run the
washing machine). If you want to write out your tasks it might look something
like this.

    
    
      LoadWasher() (sync) 
      washerRunning = RunWasher() (async)
      var cleaning = CleanKitchen()  (async)
      wetClothes = await washerRunning;
      LoadDryer(wetClothes)
      clothesToIron = await RunDryer()  // while we're awaiting we walk back up the stack and can continue cleaning the kitchen until the dryer is finished
      IronClothes(clothesToIron) // this is just happening synchronously
      await cleaning //last job to do so we need to make sure we finish!
    

Don't know if that helps, but I think it's an interesting analogy

~~~
dleslie
Wouldn't it be desirable to replace async with sync, then?

I'd rather dirty the signatures of blocking/synchronous methods then of the
rest.

~~~
Cyph0n
Unlike JS, most languages, including Rust, are not async by default, so this
would not make sense.

~~~
dtech
How is JS async by default?

~~~
steveklabnik
When people say this, they usually mean "a significant amount of things
JavaScript is used for are asynchronous, and IO is virtually always
asynchronous," rather than something more literal. The language has
asynchronicity deep in its bones, even if it's not entirely made out of it.

------
jkoudys
I used to be a huge fan of async/await in the JS world, until I realized that
we were tethering a very specific implementation to new syntax, without adding
any clarity beyond what a coroutine and generator could do. eg `async function
foo() { const a = await bar();` vs `co(function* foo() { const a = yield
bar();` isn't really with adding extra syntax for.

I came in not wanting to see rs go down the same path. The idea of adding
something that looks like it should be a simple value (`.await`) seemed odd to
me, and I was already finding I liked raw Futures in rust better than
async/await in JS already, especially because you can into a `Result` into a
`Future`, which made async code already surprisingly similar to sync code.

I will say, the big thing in rust that has me liking async/await now is how it
works with `?`. Rust has done a great job avoiding adding so much sugar it
becomes sickly-sweet, but I've felt `?` is one of those things that has such
clean semantics and simplifies an extremely common case that it's practically
a necessity in my code today. `.await?` is downright beautiful.

~~~
kolektiv
I spent quite a lot of time writing Typescript using Fluture and a few other
libraries a while ago. Coming from a functional background, combinators +
futures actually feels quite natural to me, so I looked at the examples of the
"bad" code here and thought I actually quite liked it. But I certainly don't
mind the async syntax too much either. Will see how it works out some time!

~~~
leshow
It's not just about the combinators vs specialized syntax. I've had to write a
library with 0.1 futures and you have to implement a lot of Futures manually.
i.e. you need to implement the Future trait for your types pretty often. In
those cases you cant use the combinators, you need to directly drive the state
machine of your future with poll() and try_ready macro etc. With async/await I
don't have to implement any futures manually anymore, the diff of my library
updating to async/await is basically deleting 75% of my code.

~~~
jkoudys
Yeah and this is huge. Not something I realised I needed coming from my
experience in js, but it's a big deal in rust. It's not that different from
needing to manually branch match arms and `into` your errors back for
`Result`s, when those could all be communicated just as clearly with the far
more concise `?`. The `.await` removes so much re-implementing the Future
trait. It's been especially bothersome for me, because often I'm trying to
figure out how this particular library implemented their Future. Now I feel
like if I get the gist of how a Result is returned, I can apply identical
reasoning to how `.await` is returning my Future, except that it's suspending
the function and saving the state. Obviously that's what a generator does, but
the `.await` syntax is far more expressive than `await` in js, because passing
back that `Result` is a much more difficult thing to manage by hand.

------
autarch
As a fairly new rustacean myself, I can confirm that the pre-syntax examples
are incomprehensible gibberish to me. The bits with `await` are much simpler
to make sense of.

~~~
Arnavion
You'll still need to write enum state machines if you want to implement custom
Streams (since libstd is only adding Future at the moment).

Even with Futures there can be times when you need the fine-grained control
over execution that you don't get with async-await. Eg if you want to
implement a `struct TimeOut<F: Future> { inner: F, timeout: std::time::Instant
}` from scratch, you can't do that with async-await.

~~~
pimeys
Now being able to just dig into the lower level of Futures at work, I'd say
even that is not that hard. You just write the `poll` function and then
pattern match inside. There are even nice macros, such as the `ready!` that
helps you with some of the boilerplate.

The trickiest thing for me to grasp first was the `Pin` type. Like how does
this prevent the value to be moved... Until I realized it's `Pin<&mut T>` and
then I had my a-ha moment.

~~~
Arnavion
It's slightly more complicated than just pattern-matching.

\- In general you have to be careful that if you're returning Poll::Pending,
it's after something has registered a wakeup, otherwise your future is going
to just stop.

\- As a specific case, your match has to be inside a loop so that a state
transition results in the new state being polled immediately. You can't just
return and expect the caller to poll() you since there will be no wakeup
registered that triggers the caller to do so.

\- Because futures receive `&mut Self` rather than `Self`, it's difficult to
do a state transition where the new state needs to own some properties of the
current state. Eg `State::Foo { id: String } -> State::Bar { id: String }`
Instead you have to do Option::take / std::mem::replace shenanigans on the
state fields, or have an ugly State::Invalid that you can std::mem::replace
the whole state with.

------
hansjorg
A bit confusing at first, but quite the novelty to use something like rustdoc
and docs.rs as a blogging platform.

~~~
Narishma
I was confused as well. I thought it was the official Rust documentation until
I read your comment.

~~~
saghm
Slightly OT, but in case anyone else is curious, official Rust documentation
lives at doc.rust-lang.org. docs.rs is auto-generated docs for packages
published on crates.io.

~~~
steveklabnik
docs.rs is run by the Rust team these days, so it is also "official", but
documents un-official things.

~~~
saghm
Fair enough! "documents un-official things" is what I intended to imply;
apologies if I wasn't clear

~~~
steveklabnik
It's chill! It used to be its own thing, and I'm not sure if people know that
it's not or not anymore :)

~~~
saghm
Now that I think of it, I'm not actually sure whether I knew before that it
was official or not. I've definitely been using it for a long time now though,
so I'm happy to hear it's in good hands!

------
iddan
I'm building an async server application
([https://github.com/iddan/minerva](https://github.com/iddan/minerva)) with
async Rust and I feel the pain of not having await. Every time I'm using a
Future I need to care about borrowing and Error types that if it was sync code
I won't.

------
_bxg1
Am I the only one who finds it incredibly hard to grok how async/await and
"regular" sequential code interplay? In JavaScript I strongly prefer the
regular Promise way of doing things because async feels like it's mixing
metaphors. Being relatively new to Rust it's not totally clear to me whether
the Futures example in the article could be made more readable - it's
definitely pretty gross - but I have to wonder.

~~~
macromaniac
It took me a while to get comfortable with it too. The intuition I use is that
in the past it used to be DrawWindow(), DrawWindow(),
DisplayResultsofSlowFunction() and then the window would freeze for a few
seconds, but with async/await theres a queue basically and your
DisplayResultsOfSlowFunction() gets put to the side while awaiting so
DrawWindow() can keep doin what its doin and your app doesnt appear to freeze,
for a gui example (idk if that helps or if ye already knew it but just in
case, for me all the internet explanations never talked about the scheduler
for some reason so it took me a while).

~~~
chrismorgan
Note that futures, promises, _& c._ are all about deliberately asynchronous
operations, most commonly I/O. Expressed in older terms, it’s _cooperative_
multitasking.

Thus, when you have a CPU-bound function, they are of no assistance in UI
responsiveness, unless you deliberately insert yield points; in JavaScript,
for example, you might do a unit of work, or process work until, say, ten
milliseconds have elapsed, and then call setTimeout(keepGoing, 0). (Critically
in JavaScript, you mustn’t use `Promise.resolve().then(keepGoing)`, because
that’ll end up in the microtask queue which must be flushed empty before the
browser will render, so you won’t actually yield to the UI with this
approach.)

So be careful about your DisplayResultsOfSlowFunction(). If its slowness is
that it takes a long time to calculate, and it doesn’t actually have any
asynchronous stuff in it, async will not help you.

------
MuffinFlavored
> Although the more ergonomic syntax is not ready yet, asynchronous I/O is
> already possible in Rust with ecosystem around futures, mio, and tokio.

Does this mean all 3 of those crates are required to achieve async I/O?

tokio is a large project. Which part of tokio specifically is supposed to be
used in conjunction with futures + mio to achieve asynchronous I/O?

Edit: it seems futures + mio are dependencies of tokio.

~~~
steveklabnik
Futures are the interface for asynchronous code. You need _some_ sort of
executor to actually drive the chain of futures to completion. Tokio is the
most popular option, but is not required, strictly speaking. You can whip up a
simple executor with nothing but the standard library. It won’t perform as
well as Tokio, of course. There’s a good reason it’s a big project.

(Tokio is built on top of mio so if you’re using Tokio, you’re using mio)

~~~
majewsky
Doesn't futures 0.3-alpha bring its own executor? I haven't followed Tokio too
closely recently, but my understanding was that, going forward, futures
provides the executor and Tokio provides the reactor. At least, that would be
a division that would make sense to me.

~~~
steveklabnik
futures-preview 0.3 does have futures::executor::ThreadPool; a simple thread-
pool based executor. However, I thought that tokio did its own thing here. I
could be wrong.

~~~
Matthias247
That is right. And while it theory having a separate executor (eg in the
futures lib) and IO providers (timers, sockets) in another lib works it will
in practice never be as efficient as having an integrated system like Tokio.
The reason for that is that decoupled solutions can’t integrate the reactors
in the same eventloop and there is always extra synchronization and thread
hopping involved. The difference can be huge, eg 50% less throughout on a
server.

------
j1elo
As a pure C & C++ programmer, I was amazed of the (to my eyes) incredibly
clever little trick that is the Duff's Device [0], and started using it right
away to provide embedded programs with easy to write "concurrency" [1], [2].

Some time later I had to dip my toes into the world of Javascript, learning
first about Promises, and finally reading about async/await. I just realized
that it is basically the same trick I had been using all along. And now it's
coming to Rust, neat!

[0]:
[https://web.archive.org/web/20190217162607/http://www.drdobb...](https://web.archive.org/web/20190217162607/http://www.drdobbs.com/a-reusable-
duff-device/184406208)

[1]: [http://dunkels.com/adam/pt/](http://dunkels.com/adam/pt/)

[2]:
[http://dunkels.com/adam/pt/expansion.html](http://dunkels.com/adam/pt/expansion.html)

~~~
Tyr42
I don't see how Duff's Device provides "concurrency"? It's a way to unrolls
loops. Now surplanted by SIMD instructions.

Or do you mean state machines?

~~~
iherbig
Look upon the horrors and weep:
[https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html](https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html)

------
bfrog
There's been a long history of refinements leading to this.

Fantastic work! Look forward to the async/await simplicity

------
giovannibajo1
Can somebody explain the performance characteristics? For instance, let’s say
I have a deep stack of 15 async calls. If the deepest call yields, what code
is executed? Does it “jump back” to the top ala “longjmp” or it needs to
return back up the stack frame by frame? And what happens when it resumes
execution? IOW is the overhead linear with the stack depth or not?

~~~
Rusky
You are correct, the overhead is linear with the stack depth. I suggested a
more longjmp-esque approach [here]([https://internals.rust-lang.org/t/pre-rfc-
cps-transform-for-...](https://internals.rust-lang.org/t/pre-rfc-cps-
transform-for-generators/7120)) early on, but it didn't seem to be worth the
extra implementation effort. Benchmarking never showed this linear overhead to
be an issue.

The current thinking from the team seems to be that, if this does become an
issue, a) inlining before state machine transformation can flatten this out
quite a bit and b) convincing LLVM to do what would be required is a _lot_ of
work.

~~~
Arnavion
Also, you as the user always have the option of manually reducing the depth of
the stack - spawn() a separate future with a one-shot channel sender and
.await its receiver.

------
infinity0
Yet another 1000-word rederivation of a monad, yawn. Wake me up when you've
realised you might as well just write Haskell.

~~~
steveklabnik
You're missing the forest for the trees. While in some abstract sense,
async/await is monadic, sure, Rust does not have the ability to express the
monad abstraction, and it's unclear if it ever will be able to. Those details
are what makes this topic interesting.

~~~
carterschonwald
from my previous interactions with the rust years ago and related rust design
discussions on their issue tracker,the main barrier seemed to be rooted in the
unfortunate "self" trick for how "the type im instantiating a trait with"
syntax making higher kinded trait support expressible as a simple extension,

This is of course ignoring the related need to also write code parametrized
over lifetime variable structure ..

