
I'm not feeling the async pressure - pauloxnet
https://lucumr.pocoo.org/2020/1/1/async-pressure/
======
LennyWhiteJr
I wish he would have commented on .NET's async implementation. Microsoft
really got it right here and it's arguably the best implementation I've seen.

All .NET async APIs take an optional cancellation token parameter which solves
his flow-control problem by allowing the async request to be canceled at any
time. If the token is canceled, the async task will (or should) throw an
OperationCanceledException which can then be cleanly handled in a standard
try/catch block up the stack.

The best part about this is that it pervades the entire .NET runtime, the
APIs, code examples, and has excellent documentation on correct usage
patterns. Sure, a 3rd party library could choose not to support cancellation
tokens, but they would be going against the entire .NET ecosystem by doing so.
Every other async implementation I've seen has really seemed like a haphazard
bolt-on.

I honestly don't know how I would write robust async code without cancellation
tokens, so I guess he has a point when it comes to javascript and python
ecosystems.

~~~
fanf2
Cancellation and backpressure are different things. Backpressure is about not
accepting work when there is not capacity to handle it. It is incorrect to
accept too much work, hit overload, and respond to the emergency by throwing
away partially completed work. That makes the overload worse by wasting
effort.

~~~
dnautics
the default backpressure response in the BEAM is that when you make a call to
another actor, and your call times out because the other actor is too busy,
the network temporarily is too closed, etc, etc, YOU die, with no intervention
from the service you've called.

------
pantulis
IMHO all the rage on async comes from an era when interpreted languages where
the main trend as they demonstrably increased developer productivity --I'm
looking at you, Rails, Django. Those platforms were not designed for runtime
speed, so it made sense to push the bottleneck to the most inmediate backend
system in the chain (i.e.: your trusty database, which is much faster than
your agile framework of choice, remember the discussions of ORMs versus pure
SQL?)

Then there came async frameworks a la Node, Twisted et al. and changed
everything. Again in my opinion, async code is harder to reason about versus
synchronous code.

Things to keep in mind:

\- Are you really working at scale? Does the arguably added complexity of
async benefit your particular use case? Specially when SPA technologies allow
to build simpler backend for frontend systems (pure API, no HTML rendering).
And not only regarding pure operational performance, Rails is still impossibly
hard to beat when it comes to productivity.

\- New players like Go, Rust have async capabilities but you dont necessarily
need to use them to perform closer to native speed, hence becoming simpler
solutions than Node, Ruby, or Python. Guess that also applies to old dogs with
new tricks in the JVM (Micronaut, Quarkus...)

~~~
skohan
> Then there came async frameworks a la Node, Twisted et al. and changed
> everything. Again in my opinion, async code is harder to reason about versus
> synchronous code.

One change here which has also made this less important is the trend toward
serverless/microservice architecture. In a world where you were running your
MVP on a single VPS instance somewhere, and you were running a monolithic
webserver which handled everything from request parsing to business logic,
then moving from one-thread-per-request to a runloop implementation
represented a potentially huge win in terms of performance.

But nowadays we largely work with more atomized bits of logic: business logic
is implemented in terms of small, self-contained operations, and the problems
of scale are delegated to some other system or orchestration framework. In
that world it matters much less if your database access is blocking a thread
or not, because throughput issues can be solved at a different level.

~~~
pantulis
This is another plane that seems to make async less relevant, yes.

------
ncmncm
I wonder if the advent of async/await in all the popular and upcoming
programming languages will be seen, after some (buffering) interval, as a
disaster of major proportions. And, I wonder how the programming world will
respond to the disaster.

It seems advisable to begin that response now. The linked article might be the
beginning of such a response, but it seems too tentative. We may need an Iron
Law of Flow Control, visibly acknowledged and observed in each system that
uses an async/await facility, at the point of use, or a note explaining where
it is handled farther back up the chain.

TCP vs IP is an excellent example of such an alternative: IP does not bother
with buffering, except as a completely local performance optimization, and
happily drops packets at the first hint of trouble, assured that somebody
closer to the source has buffered copies of whatever they actually care about.

~~~
lazulicurio
> I wonder if the advent of async/await in all the popular and upcoming
> programming languages will be seen, after some (buffering) interval, as a
> disaster of major proportions. And, I wonder how the programming world will
> respond to the disaster.

I think that there is value in async/await as a concept, but that some
language's specific implementations will be eventually seen as a mistake. My
primary experience is in .Net, and I've grown disenchanted with async/await in
that realm. I feel like it makes the easy things easier and the hard things
harder[1], which, IMO, is the wrong trade-off for a concurrency primitive[2].
By comparison, what I've read about async/await in Rust seems very promising.

[1] Specifically, I think that the Task API is warty and over-complicated,
with abominations such as .ConfigureAwait (I know that there are situations
where it can be useful, but those don't make the method any less terrible).
While await doesn't strictly depend on Task[1a], in practice you're going to
be using Task.

[1a] [http://blog.i3arnon.com/2018/01/02/task-enumerable-
awaiter/](http://blog.i3arnon.com/2018/01/02/task-enumerable-awaiter/)

[2] Before anyone chimes in with "async/await has nothing to do with
concurrency/threads---it's just state machines": yes, that is technically true
in the narrow sense of how the compiler desugars the async/await keywords.
However, as soon as you start talking about actually scheduling and executing
your tasks, concurrency/threads become very important.

~~~
Animats
Threads _are_ state machines. Packaging up task state as a closure and
packaging up task state as a stack do roughly the same thing. In Go, the two
are closer together than in other languages where async was a retrofit.

~~~
lazulicurio
Sure, but if you're going to go that route, why not just say "the entire
computing system at a whole is just a state machine, so what's the
difference"?

Threads provide a very privileged form of state machine that are tightly
coupled with the underlying platform. This means you have to be thinking about
things like cache coherency and context switches. Without careful use of other
synchronization primitives (or stronger language guarantees[1]) it's easy to
find yourself writing incorrect or less-than-performant[2] code.

[1] Like in Rust

[2] e.g. creating bottlenecks by blocking on access to a shared resource
(Edit: perhaps a better way to word that would be "accidentally reintroducing
blocking behavior by using synchronization primitives to access a resource")

------
majke
> _In most async systems … you end up in a world where you chain a bunch of
> async functions together with no regard of back pressure._

yup. Back pressure doesn’t compose in the world of callbacks / async. It does
compose if designed well in coroutine world (see: erlang).

> _async /await is great but it encourages writing stuff that will behave
> catastrophically when overloaded._

yup. It’s very hard, in larger systems impossible, to do back pressure right
with callbacks / async programming model.

This is how I assess software projects I look at. How fast is database is one
thing. What does it do when I send it 2GiB of requests not reading responses?
What happens when I open a bazillion connections to it? Will a previously
established connections have priority over handling new connections?

------
fyp
I really wish we had more control over the scheduling of async tasks.

For a javascript example I ran into recently, say I am firing off a fetch for
each image that comes into view in a large gallery. If I suddenly scroll down
to the 1000th image, a naive implementation might fire off 1000 fetches for
all the images we scrolled past. Then you'll be waiting a long time before the
images in your current viewport is loaded.

Backpressure can save you a little bit here. Say you do the semaphore trick
mentioned in the article and only allow a max of say 10 fetches in flight at
once. Then if you quickly scroll through, all the subsequent fetches after the
initial should fail, including the ones at the viewport you stop at. But since
the queue is short, when the images in your current viewport retries it should
now succeed.

This works but it isn't ideal. Ideally I would be able to just reprioritize
the newer fetches to be LIFO instead of FIFO. Or maybe inspect what's
currently queued up (and how big the queue is) so I can cancel everything that
I don't need.

The backpressure solutions might just be a symptom of async tasks not being
controllable in any way once started which is why you're forced to commit to
it or not from the start even if that might not be the best point in time to
make that decision.

~~~
z3t4
Just load all images. No JavaScript! Let the browser handle it. I hate when
web sites only display 1-3 pages worth of content and unload/load more when I
scroll. The JS code for that uses more resources then pulling everything
would. It's a user nightmare, where I cannot use "find" and I can't zoom out,
or scroll fast. The browser can easily handle 10,000 DOM elements. There is
already optimizations in place in browsers which solve render issues. Beating
the browser at it's own job will require a tremendously amount of work. These
problem was already solved when computers only had 256MB of memory. And your
web site would be blazingly fast today if you did not complicate things.

~~~
fyp
Native lazy loading is an extremely recent feature! [https://web.dev/native-
lazy-loading/](https://web.dev/native-lazy-loading/)
[https://caniuse.com/#feat=loading-lazy-
attr](https://caniuse.com/#feat=loading-lazy-attr)

So no, browsers don't have these problems solved out of the box. And it seems
like what they are implementing will suffer from the exact problem I described
here too.

I think you might also be thinking of bad implementations of infinite scroll
which isn't what I am talking about. Have you never use something like
photos.google.com? Scrubbing/scrolling to an arbitrary point in time is
honestly a great user experience. Would you rather wait hours/days for a
webpage to load all the past photos images you've ever taken? Or have to deal
with pagination and click through hundreds of pages to find what you want?

------
j88439h84
As mentioned in the article, Python's Trio solves all of these issues much
better than asyncio does.

[https://trio.readthedocs.io](https://trio.readthedocs.io)

~~~
jsmeaton
Do you happen to know how it compares to curio?

~~~
j88439h84
Curio is more like an experimental sandbox for its author. Trio was created as
a production-oriented application of similar ideas. There are some places
where Trio diverges from Curio, and I think Trio's choices are improvements
there. For example, the handling of schedule points and cancel points
([https://trio.readthedocs.io/en/stable/design.html#cancel-
poi...](https://trio.readthedocs.io/en/stable/design.html#cancel-points-and-
schedule-points)).

------
kd5bjo
> So why is write not doing an implicit drain? Well it's a massive API
> oversight and I'm not exactly sure how it happened.

Separating these two operations allows code to use multiple write() calls to
build up a single record atomically before yielding control to the system,
where some other task might also write to the same stream. This reasoning is
only valid if the program is running on a single thread, but that’s a
reasonable architecture decision for many programs.

~~~
PudgePacket
Not a super great argument. It would be easy to just build up the data to be
written then write it in 1 call, while at the same time preventing people from
being able to use it incorrectly at all.

------
christiansakai
Why is Go mentioned here? AFAIK, Go's goroutine makes async not async like in
NodeJS async sense, but just lightweight user space threading, so just
blocking like regular threading.

------
jph
Ideally async and backpressure will advance to make better use of scheduling,
prioritizing, quality of service shaping, termination conditioning, and the
like.

As an example, there's a big difference between the async needs of one paying
customer who's loading one medical chart web page vs. some free third-party
web crawler trying a daily full text scan of an entire site.

Async with cost functions feels like a promising area for real-world use
cases.

------
Doxin
It seems to me that all that this boils down to is the fact that await/async
makes back pressure something you need to deal with explicitly. Having the
default be buffering isn't ideal but since each application will have their
own idea of what to do with backpressure it'd be hard to figure out a
different default that works better.

In any case all this can be solved without major rewrites by making sure every
awaitable is awaited _at some point in the future_. Instead of doing this:

    
    
        while connection.accept():
            handle_connection()
    

you might do something like this:

    
    
        connection_pool = []
        while connection.accept():
            connection_pool.append(handle_connection())
            if len(connection_pool)>=MAX_CONNECTIONS:
                await wait_any(connection_pool)
    

And there you go. Anytime there's more than MAX_CONNECTIONS the program stops
accepting new connections, providing back pressure. It's more code but it's
also defining exactly HOW to provide back pressure. Your specific use case
might warrant providing back pressure not based on connection count but cpu
usage or average response times. You might want to have a single global
maximum connection count instead of one per thread. All of these aren't much
more complex than what I've shown above as long as you keep the cardinal rule
in mind: any awaitable MUST be awaited.

And in fact in python -- and other programming languages too I bet -- you get
a warning on exit if there are any awaitables that never got awaited. In my
opinion that should be an error. I can't think of a single scenario where it'd
be proper form to never await an awaitable. You might await immediately,
sometime in the future, or at the end of the program. But you never don't
await at all.

Threading isn't any easer, or harder. If you spawn a thread for each
connection you run into the same issue as await unless you do something about
it. If you use a pool of threads you get backpressure for free, but the same
goes for a pool of awaitables!

tl;dr: async/await has the exact same problems as threads, the tooling around
async/await is just less mature. Rewriting async code to provide back pressure
is near trivial.

------
mayoff
Backpressure is the big thing that Reactive Streams adds to Rx/ReactiveX.

[https://www.reactive-streams.org/](https://www.reactive-streams.org/)

------
iforgotpassword
My main problem with async is that it's much harder to build a mental model of
what exactly is going on. Admittedly, I only had to deal with existing code so
far doing only minor work in it so you can definitely say it's due to lack of
experience. However it seems that while problems like concurrent access to
shared memory known from traditional threaded code don't exist, having to
think about what can block etc. is of equal burden.

------
grok2
How do actor model languages (like ponylang) handle this? It seems like not
having back-pressure support would be a fundamental issue with the language.

~~~
ramchip
There’s a detailed article on the topic at: [https://ferd.ca/handling-
overload.html](https://ferd.ca/handling-overload.html)

~~~
grok2
It seems like the gist is that the responsibility is on the application to be
architect-ed (perhaps using one of the various libraries/strategies) to handle
flow-control/back-pressure. The language itself doesn't directly help.

------
carapace
FWIW, having climbed Twisted's learning curve long ago the current async-all-
the-things fad looks so childish to me. I remember when Tornado came out and I
was like, why would you use a go-kart when there's a free _Maserati_ right
there?

[https://twistedmatrix.com/trac/](https://twistedmatrix.com/trac/)

------
davidjnelson
Doesn’t solve the issue with calling an api that doesn’t return a promise, but
the tslint rule “no-floating-promises” flags missing awaits which is quite
useful. Also useful is flagging awaits that don’t return a promise, which can
be done with the tslint rule “await-promise”.

------
panitaxx
In nodejs you can use streams or rxjs. Both handle backpressure quite nicely.
They work on bytes and on objects as well.

------
andrewstuart
The thing I like about async Python is being able to run subprocesses and
capture and processes their stdout/stderr

~~~
viraptor
You can do that without async in python. What did you find specific to it?

~~~
ComputerGuru
I imagine he means without having to understand/use the platform or framework
equivalent of select/poll or start dedicated threads or write a threadpool to
handle reading from streams simultaneously without blocking. Now he has
someone else wrote that code for him ;)

