
Update on await syntax in Rust - steveklabnik
https://boats.gitlab.io/blog/post/await-decision-ii/
======
rileywatkins
What a milestone! This was the most impressive work in open source decision-
making that I have ever followed (in my limited experience). People have
contributed _thousands_ of comments discussing this for over a year. It has
even caused the team to rethink how discussions like this should be guided in
the future beyond comments on GitHub issues. I cannot imagine how difficult it
has been to manage it all, and I applaud and thank the Rust team for their
hard work in fielding comments, weighing trade-offs, and ultimately making a
decision in order to move the project forward.

~~~
hu3
> This was the most impressive work in open source decision-making that I have
> ever followed...

You'd be blown away by Linux Kernel development then. Multiple magnitudes
above any measurement of an await syntax bikeshedding. /u/Perceptes puts it
best than I ever could in this top voted comment on /r/rust [1]:

"...I personally don't care at all about the await syntax, and have been very
unhappy with that bikeshed being such a focus of time and discussion. The real
problems I have with the unstable async system are how difficult it is to
reason about. Compiler errors are still largely indecipherable. impl Trait
didn't really help here. Documentation is still sparse, so I spend a lot of
time banging my head against the wall, trying many formulations of my programs
to get just the right incantation of futures wrapped in futures wrapped in
futures to get things to compile, only to get page-long type mismatch errors
that are near impossible to read, and multiple closures/functions deep of
nested futures being returned where any of them could be the culprit. I'm also
very frustrated by the lack of progress on "abstract types" that are required
for using async functions (and returning impl Trait) from trait methods.
Traits are the cornerstone of Rust's type system and yet I haven't seen any
activity on this for months. And now we're talking stabilization with this
glaring hole. Async streams are another thing I've seen almost no progress on,
aside from a blog post from withoutboats on how to deal with the ? operator
within async for loops..."

This was 2 months ago, mind you. [1]
[https://www.reddit.com/r/rust/comments/awa3t1/baby_steps_blo...](https://www.reddit.com/r/rust/comments/awa3t1/baby_steps_blog_asyncawait_status_report/)

I'm just glad that they finally settled on something.

------
CJefferson
I'm nit going to stop using rust, but I find this very disappointing.

One thing I strongly disagree with - - this article said other notations would
require Rust users having to learn more notation. This still requires learning
more notation, just it's easy to not even be aware it exists and think a
struct had a member called await instead...

~~~
kbenson
My understanding of the issue (which is shallow at best) is that it's a battle
between a prefixing keyword or sigil which has the problem of causing the
reader to scan back and forth in the statement when chaining multiple items,
which sucks), or a postfixing keyword or sigil, with dot notation for a
keyword to help parsing (and everyone seems to dislike the overloading of the
dot notation for this). Ignoring sigils, I think the chosen solution is the
lesser of two evils.

That said, I think a postfixing sigil would have been much better. It seems an
important enough thing to denote that a bit of special syntax is called for,
and the more unique the better.

~~~
Nican
For example, on C#, I sometimes have to do:

    
    
      var value = (await (await startRequest()).GetBody()).Root;
    

While the Rust syntax would make this a little more clear, I do not think it
is a dealbreaker.

~~~
mdpopescu
Please don't do that. I had to look several times to rewrite it as

    
    
        var temp1 = await StartRequest();
        var temp2 = await temp1.GetBody();
        var value = temp2.Root;
    

which is instantly grokked.

~~~
steveklabnik
This is a great example of why syntax and semantics can't be separated. This
works great in GC'd languages, but in Rust, these two things may not be
equivalent. With the distinction between owned and temporary values, this may
change the lifetime of what stuff is in scope and when. This is reduced a bit
with non-lexical lifetimes, but it's not, strictly speaking, actually
equivalent.

~~~
nemothekid
I think this is a good example to show people who are on the fence about
postfix then. I haven't felt strongly about postfix, but this is an eye opener
for me.

~~~
steveklabnik
Yeah, I think the trick is that the theory and practice _might_ be the same.
And usually, it's the other way around; spreading it out makes it work,
whereas it may not otherwise. For example:

    
    
        fn main() {
            let world = gives_string().split(" ").next();
            
            println!("{:?}", world);
        }
        
        fn gives_string() -> String {
            String::from("hello world")
        }
    

This will fail because the String is temporary, and we're trying to get a
reference to it (via split), and so it would be deallocated at the end of the
line, being a use-after-free. This, however, compiles:

    
    
        fn main() {
            let world = gives_string();
            let world = world.split(" ").next();
            
            println!("{:?}", world);
        }
        
        fn gives_string() -> String {
            String::from("hello world")
        }
    

We're shadowing 'world', but the underlying String now lives to the end of
main, so everything works, no more use-after free.

I think before I'd want a good real-world example of where doing the multi-
line thing goes wrong before I'd want to make an argument that this is why
postfix is better.

~~~
iknowstuff
This has surprised me on occasion. Shouldn't the compiler be smart enough to
figure out how long to keep it around for, instead of failing to compile?

~~~
tylerhou
The compiler _is_ smart enough to deduce lifetimes — that’s why it can throw a
compile error — but fixing them (auto-keeping temporaries) might introduce
extra memory consumption/leaks that the programmer did not intend. Requiring
the programmer to explicitly assign a temporary to a variable makes the
programmer’s intent more clear.

A comparison: in C++, the behavior of auto-extending the lifetime of const
references to temporaries (but not non-const references) is considered a wart
in the language design. (Really, you should just assign a temporary to a value
because the compiler can elide the copy. :
[https://abseil.io/tips/101](https://abseil.io/tips/101) .)

------
childintime
All control flow mechanisms except await have their own statement: if, for,
while, match. Howcome await is different?

The dot await syntax might hide an await within an expression, as "just
another method". You'll need to hunt it down.

I feel await has costs and consequences that need to be explicit. The dot
syntax does the opposite, and therefore I feel Rust might be making a mistake
of a lifetime (pun intended) on this one.

~~~
kibwen
_> All control flow mechanisms except await have their own statement: if, for,
while, match. Howcome await is different? The dot await syntax might hide an
await within an expression, as "just another method". You'll need to hunt it
down._

Not quite; Rust is an expression-favoring language, so `if`, `match`, and even
the `loop` keyword can all appear deep within expressions (the first two do so
somewhat commonly; the latter is comparatively rare). In addition, the error-
handling operator `?` is also used within expressions. (And while we're on the
topic, let's remember that even C, a statement-heavy language, also has `?` as
a control-flow operator that is not in statement position.)

------
apo
The article doesn't actually show what the new syntax looks like. Nor does the
article linked in the first paragraph.

What's a simple example of the new syntax?

~~~
chucksmash
Postfixed and preceded by a dot is my understanding:

    
    
        pub async fn do_stuff() {
            // ...
        }
    
        // elsewhere, inside a fn
        // as svnpenn points out
        let result = do_stuff().await;

~~~
svnpenn
await goes inside the function, at least thats how its done with JavaScript -
do you have evidence otherwise?

[https://developer.mozilla.org/Web/JavaScript/Reference/Opera...](https://developer.mozilla.org/Web/JavaScript/Reference/Operators/await)

~~~
k__
They worte explicitly that Rust went a different way than C#/JS

~~~
steveklabnik
It's true, but that's around

    
    
      let result = do_stuff().await;
    

vs

    
    
      let result = await do_stuff();
    

_not_ what your parent is talking about.

~~~
k__
Ah, they meant like await goes into async functions etc.

------
ben509
A bit off-topic: Is there any theoretical reason you _need_ async / await
syntax at all?

(It's certainly desirable for performance and compatibility to avoid making
all subroutines into coroutines, so I understand why most languages have done
this.)

And restricting it to the case where coroutines have a single return...

Subroutines are naturally coroutines that don't yield. And it seems like the
question of whether it's a subroutine or a coroutine shouldn't be something
the programmer needs to worry about.

What I've seen in coroutine libraries is the main reason we _need_ await seems
to be for cases when we don't want to await a result.

If you need an unawaited invocation to pass to a routine like gather, you do
this:

    
    
        my_results = await gather(run_job(x) for x in work)
    

But, even if we can't infer from the type of gather that it must accept a
future, a function can have a method that asks it to return a future:

    
    
        my_results = gather(run_job.future(x) for x in work)
    

You'd often want to dispense with gather. If I have these statements:

    
    
        alpha = run_alpha()
        beta = run_beta()
        gamma = run_gamma()
        return alpha + beta + gamma
    

In most cases, e.g. if these were requests going over the network, we'd prefer
to schedule all three at once. If we want to sequence side-effects, then
explicit await makes sense:

    
    
        alpha = run_alpha.await()
        beta = run_beta.await()
        gamma = run_gamma.await()
        return alpha + beta + gamma
    

That makes the most concurrent option easiest, as opposed to the clunky idiom
of "gathering" many results.

~~~
deathanatos
I think (but I cannot find a reference to now) one of the proposed syntaxes
was that awaits were immediate and implicit in async functions, and that if
you _didn 't_ want that, you could opt-out and get futures normally. Loosely,

    
    
      async fn foo() {
        let result = blocking_req();  // awaiting immediately here.
        async {
          let future = blocking_req();  // not awaited.
          // can do more complex stuff w/ the future
        }
      }
    

IDK what happened to it, and it is a bit more complex IMO than what the Rust
folks went with. However, I don't think it helps your gather case.

For gather, I think the problem w/ your proposed

    
    
      return alpha + beta + gamma
    

is when does that happen? If I do

    
    
      let some = alpha + beta;
      // < ... code ... >
      let more = some + gamma;
    

what happens / when are things evaluated?

At any rate, with the syntax as it is, I _think_ your gather example is
essentially¹:

    
    
      alpha.join3(beta, gamma).await
    

or

    
    
      Future::join3(alpha, beta, gamma).await
    

(Perhaps minus the `.await`, depending on when you need/want the result.)

There's no operator overloading, which I'm fine w/, as there is a need to
choose between, e.g., select or join.

¹but I'm new w/ the async stuff, so if I'm wrong I'm assuming that
Cunningham's Law will kick in and someone will correct me.

~~~
Rusky
Here's the proposal: [https://internals.rust-lang.org/t/explicit-future-
constructi...](https://internals.rust-lang.org/t/explicit-future-construction-
implicit-await/7344)

One of the goals behind it was to align async control flow with sync/thread-
based control flow, to make it more intuitively obvious when things would run
concurrently and when they would not. That is, calling a function, even if
async, would immediately run it to completion, while wrapping it up in a
closure or async block that gets spawned would run it concurrently.

This is especially relevant to Rust, which departs from other implementations
of async/await in that merely constructing a future is insufficient to start
it running, which is why you need that `join3` call to explicitly introduce
concurrency.

Under the "implicit await" proposal, your first example would look like this:

    
    
        async fn foo() {
            let result = blocking_req(); // await immediately
            let future = async { blocking_req() }; // think of async{} like a lambda function
            // can do more complex stuff with the future, like:
            //  future.await()
            //  Future::join(future, other_future)
            //  tokio::spawn(future)
        }
    

The thread-based equivalent looks very similar:

    
    
        fn foo() {
            let result = blocking_req(); // block the current thread immediately
            let closure = || { blocking_req() }; // don't run this yet
            // can do more complex stuff with the closure, like:
            //  closure()
            //  rayon::join(closure, other_closure)
            //  thread::spawn(closure)
        }

------
seanalltogether
Question for people more familiar with async/await semantics, how do you
typically control what thread the async procedure runs on? Having spent a lot
of time now in rxJava and really getting into the power of stream processing
and composition, it feels like async/await is almost too simplistic.

~~~
steveklabnik
async/await produces a Future. In order for that Future to execute, you place
it on an executor. It won't start executing until you do so. Which thread it
runs on, and how, is all the executor's job. So the answer is basically "pick
the executor which has the semantics you want".

~~~
hinkley
I never liked this behavior in Java. Or rather, I wasn't sure it was right or
wrong and then I found code where people returned a future without starting
it, causing the caller to wait forever in some situations.

Also if you use multiple modules with their own executors, they don't compose
well. You can easily get oversubscription of CPUs.

Javascript's promises are self starting, which I would say is an improvement
but the total lack of back pressure is anywhere from galling to highly
problematic depending on the problem domain.

Goldilocks solution might be to have default executor behavior with an obvious
and straightforward way to override it.

------
yingw787
I'm not familiar around the syntax, but I'm interested to know how much of
this was a religious battle and how much of it results in meaningful
complications for user code down the line. I've heard really good things about
the Rust community (like nothing bad at all), until maybe two weeks ago when
somebody mentioned this stuff. I'm a really big fan of not having religious
battles.

Is this syntax for native coroutines? Can it be combined with existing user
and stdlib syntax? What pathways for syntax development does this decision cut
off entirely, and pathways does it leave open? What did the Rust community /
maintainers learn from this debate, and how can the Rust community avoid such
battles in the future?

~~~
steveklabnik
> I'm interested to know how much of this was a religious battle and how much
> of it results in meaningful complications for user code down the line.

Due to the connotations of "religious battle", it's hard to get a good answer
here. Nobody wants to be painted in this light.

> Is this syntax for native coroutines?

No.

> Can it be combined with existing user and stdlib syntax?

Yes.

> What pathways for syntax development does this decision cut off entirely,
> and pathways does it leave open?

Nothing outside of the async/await feature itself.

> What did the Rust community / maintainers learn from this debate, and how
> can the Rust community avoid such battles in the future?

This is too long for an HN comment :)

~~~
ben509
> Due to the connotations of "religious battle", it's hard to get a good
> answer here. Nobody wants to be painted in this light.

Haven't we all found ourselves lined up as one of a pair of camps over some
heated dispute over a bit of minutiae? The whole time you know it's a bit
silly, but not entirely and you feel compelled to continue to argue.

Using a term like "religious battles" with tongue firmly in cheek is
recognizing that we're prone to such things because we're human beings, by
putting a lampshade on it.

~~~
oconnor663
This gets trickier when you have a community where some people practice
religion in their daily life, and some people don't, and where the
interactions between the two aren't always fun. It's not that the subject
needs to be totally forbidden of course, but when it's just as easy to use
different metaphors, we might as well.

------
wst_
I am amazed how well documented and transparent was the whole process. They
new the decision would be difficult to make and there will always be people
who wouldn't like the final syntax so they made sure to state their reasoning
and trade offs clearly. Fantastic job. Can't wait for August. Thanks!

------
BozeWolf
Nice! That was quicker then I Expected. How far are they with the
implementation? Not using rust, but following progress because it is on
hackernews so much.

Why does it take so long for swift to have this? Swift + iOS developers really
really benefit from it. And there are loads of ios developers compared to rust
developers.

~~~
Arnavion
It has been usable on nightly since `nightly-2019-05-09` (with
[https://github.com/rust-lang/rust/pull/60586](https://github.com/rust-
lang/rust/pull/60586) )

------
tracker1
Time to update [https://areweasyncyet.rs/](https://areweasyncyet.rs/) again...

Really looking forward to this once it reaches down into some of the
network/web frameworks, database drivers etc.

------
kgraves
Fantastic! I'm extremely excited with this update on the Rust async/await
syntax! I have been following the blog posts and the issue tracker for a long
time and couldn't wait till this day arrived!

Eagerly waiting for the 1.37 release!

Great job Rust Team!

------
Thaxll
I'm not familiar with Rust, someone can explains why they went with .await? Is
await a field / method on a class / struct?

~~~
eximius
None of the above. It's compiler magic. It's a keyword that says 'await on
this awaitable object' (currently only a Future can be await-ed).

You can read the OP which contains links to more blog posts on the reasoning.
I think this is a bad decision precisely because of the questions you're
asking. At least making it a 'magic method' (i.e., Future::await(...) or
future.await()) with no implementation would have been better in basically
every way. A 'dot' operator already exists and this just makes it confusing.

~~~
asark
I've been planning to try out Rust for a side project, and this decision is so
principle-of-most-surprise that it's got me reconsidering, wondering what
other unpleasant weirdness lurks in the language. Looking down their other
options in one of the linked articles, they seem to reject a couple fine ones
outright and then went with one _I 'd_ have rejected outright as being too
hostile to anyone who's not quite familiar with the manual, having memorized
exceptions to ordinary behavior like this. I'm struggling to understand why
anyone liked this one at all, let alone enough to push it to the top of the
pile.

~~~
staticassertion
Yeah, this is really one of the only big warts, and hopefully not a trend.

But the rest of the language is really good so don't let this scare you off.
Most of rust feels very well thought out and reasonable.

~~~
asark
OK, if the exceptions are _quite_ exceptional and hoping, as you note, this
doesn't represent a trend, I'll keep it in consideration. Thanks for the
insight.

~~~
eximius
To reinforce what others are saying, this is the only Rust decision that gives
me even a moments pause. Everything else is pretty pleasant. Honestly, I hope
that this backfires and gets fixed in a future Rust Edition. It's a very silly
thing.

------
eximius
Excited for the feature.

Disappointed we didn't get something like a magic method (e.g., lang item)
(which isn't without precedence [1] [2]) so we could have had
`Future::await(fut)` or `fut.await()`.

[1] - [https://manishearth.github.io/blog/2017/01/10/rust-
tidbits-b...](https://manishearth.github.io/blog/2017/01/10/rust-tidbits-box-
is-special/) [2] - [https://manishearth.github.io/blog/2017/01/11/rust-
tidbits-w...](https://manishearth.github.io/blog/2017/01/11/rust-tidbits-what-
is-a-lang-item/)

~~~
eridius
`fut.await()` looks too magical. There's no hint in that expression that it's
anything other than a method call. You could argue the same about `fut.await`,
except in this case `await` is a keyword and thus the expression `x.await` is
known to be an await expression without knowing anything about `x`, whereas
`fut.await()` requires knowing the type of `fut` to understand that the method
call `.await()` resolves to the lang item. Also, since it's a lang item, you
could write your own nostd library that picks a different name for the same
lang item and cause even more confusion.

~~~
eximius
> There's no hint in that expression that it's anything other than a method
> call.

Compared with `fut.await`...

> There's no hint in that expression that it's anything other than a field
> access.

At least there is precedence for functions to have magical compiler-ness for
'doing stuff', instead of overloading field access which is one of the
simplest operations you can have.

> ... requires knowing the type of `fut` ...

tbh, you _don 't need to know_ how await works. It goes and does stuff and
returns a value. If you need to know what it does, you're looking up the types
and seeing, 'oh its a future' and looking at that.

Contrast with the field access one and you don't even know its something else
entirely.

I guess I just don't buy that it can't be a method. Like, formally, yea, it
isn't a well-defined, safe Rust method. But make the impl

    
    
        fn await(self: Self) -> Self::Target {
            intrinsic::await(self)
        }
    

Now its a method defined on the `Future` trait, it's discoverable, it's
clearly magic, it's ergonomic... In my opinion, it beats the field access one
in every conceivable metric.

It's just frustrating, I guess.

~~~
Thiez
If it's a method you should be able to create a vector of futures and
`.map(Future::await)` or even `.map(Future::await as FnMut(_) -> _)`. Clearly
this would never work, since `await` is implement by transforming the
containing function into a state machine, your suggestion would make this
impossible.

~~~
eximius
I mean, why couldn't it?

    
    
        let newvec = vec![fut1, fut2, fut3].map(Future::await).collect()
    

would be equivalent to

    
    
        let f1 = fut1.await();
        let f2 = fut2.await();
        let f3 = fut3.await();
        let newvec = vec![f1,f2,f3];
    

or

    
    
        vec![fut1,fut2,fut3].map(|fut| fut.await).collect()

~~~
Thiez
I think you fundamentally misunderstand async await. `vec[fut1, fut2,
fut3].map(|fut| fut.await).collect()` will not do what you might expect,
although I haven't bothered to read the RFC whether it will be allowed at all.
Await _must_ have compiler support and cannot just be some function you call.

~~~
eximius
I am certainly being sloppy, though I think fundamentally misunderstanding it
is a bit of an exaggeration. I know you can't await in a non-async function
so, at the very least, you'd need (IDR the syntax for async closures?)

    
    
        vec![fut1, fut2, fut3].map_async(async|fut| fut.await).collect()
    

I believe that could be built. Though, now I'm getting myself tripped up.
Because I claimed that this is similar to

    
    
        vec![fut1, fut2, fut3].map(Future::await).collect()
    

But I notice that I expect `await` to be a blocking, non-async function.
Which, feels strange in that the lambda in the first block needs to be async
but the function itself doesn't.

> Await must have compiler support and cannot just be some function you call.

I'm not saying it doesn't need compiler support. I explicitly state that it
does - I just want the magic to be a little more explicit. Moreover, I believe
that most programmers think of functions as 'encapsulated code', something
that does a thing. In this sense, it matches what await does better than
syntax that looks like it's accessing a field. Admittedly, in the purest sense
of encapsulation, this method 'escapes' that encapsulation, but I don't
consider that terribly important.

------
nathcd
Awesome, thanks Rust team! I'm super excited for this to be stabilized.

> That won’t be the end of the async/await feature - there will be a lot of
> extensions left out of the minimum feature

Is there a summary of these somewhere for perusal?

~~~
uryga
i saw [https://areweasyncyet.rs/](https://areweasyncyet.rs/) linked here, it
mentions future extensions

------
Dowwie
I think that the majority of people still don't understand what they're not
getting in August. It's an async-await mvp.

------
NyashMyash
This guy is so qt and nice.

------
Analemma_
The fact that discussions over syntax generate pages and pages of furious
bickering, while discussions over semantics (which is what actually matters)
get a shrug, is the ultimate example of bikeshedding in PL design. Honestly,
syntax just _doesn 't matter_. Yes, particular poor choices can impede
usability, but that's not applicable here, and ultimately after two minutes or
so of learning the new syntax there's no difference between this and "await
foo".

~~~
anchpop
Syntax may be irrelevant at the end of the day, but nice syntax can make a big
difference in usability imo. I'm not following the Rust example, but
discussions of it remind me about discussions about UFCS (Universal Function
Call Syntax). That's where `foo(a, b)` can be rewritten as `a.foo(b)`. That
may seem minor, but look at this code:

    
    
        half_square = divide(square(a), 2)
    

In comparison to:

    
    
        half_square = a.square().divide(2)
    

Many find the latter much more pleasant to read, and things like this can make
a big difference in how fun I find it to use a language.

~~~
byproxy
Interesting example...I find the first much easier to read and reason about.
¯\\_(ツ)_/¯

~~~
jcelerier
It would be nice for both of us to get a brain scan while reading that kind of
code, because I find the second immensely more intuitive to parse

~~~
chessturk
I find them completely equivocal. This is the geeky version of the
orange/white dress.

