
Sync vs. Async Python: What Is the Difference? - gilad
https://blog.miguelgrinberg.com/post/sync-vs-async-python-what-is-the-difference
======
calpaterson
> If these hundred tasks make heavy use of the CPU, then the sync and async
> solutions would have similar performance, since the speed at which the CPU
> runs is fixed, Python's speed of executing code is always the same and the
> work to be done by the application is also equal.

I think one of the things the author has not discussed is that if you have
high CPU then you are going to get starvation as async does not have a
fairness mechanism - timesharing is co-operative and based on FIFO in all the
implementations I've seen.

If you start to saturate CPU (even a bit) you get uneven distribution of cpu
time between your various async threads. This translates into chronic
timeouts. For async to be applicable you really need to have a) high io load
and b) lowish cpu load.

~~~
sltkr
> ... you are going to get starvation as async does not have a fairness
> mechanism - timesharing is co-operative and based on FIFO in all the
> implementations I've seen.

If it's FIFO based, why would you get starvation?

~~~
calpaterson
FIFO systems can have starvation when dealing with time bound things like web
requests - slow request handlers can cause other (waiting) requests to
timeout.

In a thread-per-request or process-per-request model where you run say 4 * cpu
workers the operating system scheduler is able to interrupt stuff that's
hogging the CPU but that isn't the case in a typical 1 * cpu worker async
model.

When there's little cpu load this is not a real threat but when there is
material load you can easily run into problems.

~~~
sltkr
I see; it sounds like latency would become very unpredictable under that
model. I can imagine that's a serious problem in practice, but it's not
typically what's meant by starvation.

[https://en.wikipedia.org/wiki/Starvation_(computer_science)](https://en.wikipedia.org/wiki/Starvation_\(computer_science\))

~~~
formerly_proven
"Task does not receive resources to advance within the required time interval"
is exactly starvation. A "task" here is responding to a request. The required
time interval is the timeout.

~~~
Joker_vD
"Starvation" is being denied the necessary resource indefinitely. What you
describe is "soft real-time failure".

~~~
formerly_proven
I think these are equivalent when applicable, because starving a task long
enough of a resource for either the resource to expire or the task's deadline
to expire has the same effect as starving the task of the resource
indefinitely: task fails to progress.

~~~
Joker_vD
That's called "firm real-time failure". Again, the standard definitions of
starvation/liveness only use "eventually" and "never" without mentioning
actual physical time IIRC, and the real-time systems use a whole another bunch
of concepts to deal with actual, physical time.

And it doesn't really fail to progress if it's cancelled on deadline
expiration, it progresses to a non-successful completion. If it instead hanged
indefinitely, yes, that'd be a problem.

------
jgehrcke
If you're curious about this topic then I would love to link you to
[https://techspot.zzzeek.org/2015/02/15/asynchronous-
python-a...](https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-
databases) written by Mike Bayer (author of SQLAlchemy) -- it's an article
that I keep referring to as a must-read, incredibly well written and all about
real-world implications (totally transferable to other non-Python ecosystems).

If you like to, maybe also have a read into
[https://gehrcke.de/gipc/background.html](https://gehrcke.de/gipc/background.html).
This is where I tried to put into words what I learned a couple of years ago
when I wrote a library for the gevent ecosystem. You'll find some paragraphs
about "Cooperative scheduling vs. preemptive scheduling", about "Asynchronous
vs. synchronous", and about a few more simple and yet important topics in the
world of event-driven architectures.

------
doubleunplussed
Not mentioned in TFA, but I'm utterly convinced that the only compelling
reason for async is to avoid the per-thread stack memory allocation of 8MB per
thread or whatever it is, in order to be able to scale to an extremely large
number of concurrent threads/coroutines. You can't do this with threads.

Async lets you do this whilst still storing the state of unfinished things in
a stack, i.e. not having to make a million callbacks. Making it look like
you're using threads even though you're not.

The better async code and interfaces become, the more it looks like regular
old multithreaded code, except there aren't actual threads underlying it. You
still need to make sure you serialise access to shared resources, don't share
data that shouldn't be shared, etc. All the same considerations as
multithreaded code.

Almost ends up looking like the underlying mechanism: threads with a GIL vs
async ought to be an implementation detail that doesn't require you to modify
your entire programming model.

Extension code not holding the GIL can still run in true parallel with real
threads, so that's a meaningful difference but is usually not relevant for IO
where async is usually used.

~~~
lmm
> avoid the per-thread stack memory allocation of 8MB per thread or whatever
> it is, in order to be able to scale to an extremely large number of
> concurrent threads/coroutines. You can't do this with threads.

It's 8 _K_ B per thread, so you can scale a thousand times further than you
thought. One dark secret of the async movement is that if your goal is C10K
(10,000 concurrent clients) then actually bog standard threading will handle
that fine these days.

> The better async code and interfaces become, the more it looks like regular
> old multithreaded code, except there aren't actual threads underlying it.
> You still need to make sure you serialise access to shared resources, don't
> share data that shouldn't be shared, etc. All the same considerations as
> multithreaded code.

Depends what approach you're using. I prefer making an explicit distinction
between sync and async functions (
[https://glyph.twistedmatrix.com/2014/02/unyielding.html](https://glyph.twistedmatrix.com/2014/02/unyielding.html)
), so you effectively invert the notion of a "critical section" \- instead of
marking which sections can't yield, you mark which sections can yield, so your
code is safe by default and you can introduce concurrency explicitly as and
when you need it for performance, rather than your code being fast-but-unsafe
by default and you're expected to fix a bunch of rare nondeterministic bugs
with minimal support from your tools, which is how it works in a
multithreading world.

~~~
mlyle
> It's 8 K B per thread, so you can scale a thousand times further than you
> thought. One dark secret of the async movement is that if your goal is C10K
> (10,000 concurrent clients) then actually bog standard threading will handle
> that fine these days.

Default virtual memory allocation for threads on Linux distributions tends to
be 8 megabytes. Actual memory used is the peak stack depth used, rounded up a
bit. It'd be pretty unusual to only use as little as 8 kilobytes per thread;
just the standard per-thread libc context information for concurrency is a few
kilobytes, plus at least one page of stack, plus the kernel's information
about the thread (which isn't counted against the process)...

Yes, you can spawn thousands of threads on relatively modest hardware; I was
spawning thousands of threads a decade ago.

Spawning 5000 bare-minimal python threads that do nil seems to use about 300
megs of ram on my system; real threads that do anything substantial will use a
whole lot more, even if their use of the stack depth is intermittent.

Not to mention allocators that cache part of freed heap per-thread, etc.

~~~
gpderetta
I believe 8k is the size of the per thread stack on the kernel side. This is
non-pageable memory, so it will consume physical memory whether it is needed
or not, while of course the 8 megabytes is paged in on demand.

~~~
mlyle
Yah, I'm ignoring the kernel stack and all kernel data structures. User space
memory used will be at least a page of stack (reaching up to the maximum
amount used in the thread), plus the libc reentrancy data structures, plus
per-thread heap caches, etc. It's can all get paged out, but we hardly want
that these days.

The key distinction is that the 8MB of stack VM doesn't have a backing until
the memory is used in the thread, but afterwards it does forever.

~~~
lmm
Well, it's forever if you assume that the threads live forever; for a web
server it's perfectly practical to have threads that only live for a single
request, or to reuse them for multiple requests but not allow a single thread
to live longer than say 10 minutes.

~~~
mlyle
Decent nits, but...

Short-lived threads (for one request) are a performance and scalability
disaster; tens of microseconds or worse to spawn and join, contention on
important locks, bad for caches, etc. There's not much concurrency when it
comes to _spawning_ threads, too.

If you have long-lived threads in a pool, yes, they may not live forever, but
you generally have to assume that each thread will end up with a resident
stack size equal to the largest stack use: each will get a turn to run the
stack-intensive functions.

------
jacquesm
All this async stuff should die as soon as possible. It exposes the
limitations of the underlying tech and forces the application developer to
deal with things that the systems level people couldn't bother with.

Look at the Erlang eco-system to see how this is done the right way.

Threads, asynchronicity, locks and so on should be dealt with at the OS or the
framework level, _not_ at the application level.

Javascript also suffers from this and it makes JS ugly and hard to read.
Callback hell will be yours or you end up with crutches such as promises.

At the application level things should be as deterministic as possible and the
default should be that statements executed in sequence will have their side
effects updated in sequence as well.

~~~
speedgoose
Thanks to the async/await keywords, modern javascript is not a callback hell
anymore.

~~~
jacquesm
No, now you get it all mixed up. One library will use callback functions you
supply, another will use callbacks but inline them and yet another will use
async/await. And you, the application developer are left somewhere in the
middle.

Oh, and you can't use 'await' in the main thread of your JS, you can only use
it from within another async function, so when you need it most - for instance
during application start-up - it isn't available.

~~~
speedgoose
You can use promisify for the first one. For the second one I think these
libraries doing that but still not supporting promises are not that common
anymore but nothing you can't do with promises yourself.

Javascript has a single thread, but I see what you mean and it has been fixed
recently. To support older versions you can simply declare an anonymous async
function and execute it immediately.

------
pansa2
> _Greenlets are similar to coroutines [...] the async ecosystem in Python is
> fractured in two big groups._

Async in software development in general is fractured in two big groups. On
the one hand languages like C#, C++ and Dart support (stackless) coroutines
like Python’s - and on the other hand, Go, Java and Lua support stackful
coroutines that are more like Python’s greenlets.

There are pros and cons to each approach. I wonder if one or the other will
eventually become dominant.

~~~
BorisTheBrave
What is Java's support for "stackful coroutines"?

~~~
pansa2
Project Loom:
[https://wiki.openjdk.java.net/display/loom/Main](https://wiki.openjdk.java.net/display/loom/Main)

~~~
fnord123
You used the present tense. Loom is not available yet.

------
boardwaalk
I just want to rep Trio, mentioned in the article. I'm using it to prototype a
system in a different language before doing another iteration of said system
and it's is quite nice to use, at least compared to what I remember asyncio
being like.

~~~
florimondmanca
> it's is quite nice to use, at least compared to what I remember asyncio
> being like.

Yes, I agree. Fairly recently there's been anyio [0], which brings ideas from
trio to other async libraries, in particular asyncio. E.g. it has equivalent
of trio nurseries (what anyio calls "task groups") for implementing structured
concurrency ideas in asyncio environments (saving you from headaches due to
having to deal with tasks manually). Very neat way to use trio ideas when
stuck on asyncio (e.g. for web apps). :)

anyio was also discussed in the latest Python Bytes episode [1].

[0]: [https://github.com/agronholm/anyio](https://github.com/agronholm/anyio)

[1]: [https://pythonbytes.fm/episodes/show/197/structured-
concurre...](https://pythonbytes.fm/episodes/show/197/structured-concurrency-
in-pythonB)

------
andrewstuart
I absolutely love async Python.

It lets me do stuff that would be hard with ordinary Python.

No doubt it takes some effort and practice to grasp, but once you have it
worked out you'll find it's a powerful tool in the toolbelt.

Many languages now have async and await - it's not exclusive to Python. The
reason so many languages have gopne this way is that so far it's the easiest
and most sensible solution to writing concurrent code in a way that won't
become incomprehensible.

And if you are trying to get your head around it, the easiest analogy is the
web browser and JavaScript. If you have grasped JavaScript and the event loop
then essentially you have grasped async Python, or not far from it.

Some people misunderstand async Python - they think the point is to make
Python faster - it isn't. The point is to enable concurrent programming.

I do understand the haters - at first, until you grasp it, async Python so
incredibly foreign and so different from ordinary synchronous Python that you
might think "AAAARGH why did they do this and make it so hard"? But really
it's not hard once you grasp the mental model of what is going on.

~~~
nurettin
I had so much trouble getting the python tcp client to work like I wanted it
to. Then I tried my hand on the async tcp client and was pleasantly surprised
how simple it was.

    
    
        reader, writer = await asyncio.wait_for(asyncio.open_connection(), connect_timeout)
        line = ""
        while True: 
            line = await asyncio.wait_for(reader.readline(), read_timeout)
            if line == b"":
                break
            yield line

~~~
andrewstuart
I've written a bunch of things that were really well suited to async Python
including an mpjeg server, a queueing server and a process management server
that monitors the stdout and stderr of various processes and reads them in
real time and acts upon the messages it reads.

Each one of those applications was made possible comprehensible and even easy
by async Python.

~~~
nurettin
I made a python clone of multitail in just under a day thanks to being able to
read multiple files simultaneously with async. It really is overpowered.

------
holografix
I’m very uninformed when it comes to async in Python so please bear with me.

Given a “web API type” python app, where about 30% of the APIs also reach out
to other, external APIs, before responds to the incoming requests...

Is that _the_ classic example where async would provide a solid benefit?

Is that why Node has become so popular = it’s all json + non blocking API
responses?

~~~
hansvm
That's where it _could_ provide a solid benefit (though you could probably
usually do better by instantiating an appropriately sized thread pool and
avoiding async entirely), but that's missing the real power of async which is
that it makes concurrency easier to reason about. Except at an await the
application behaves serially; you can't get interrupted in between lines or in
the middle of an operation that it turns out isn't atomic.

~~~
arethuza
"it makes concurrency easier to reason about"

Doesn't it do that by not really having concurrency? There is only a single
locus of control so a whole class of problems should simply cease to exist.

~~~
hansvm
We might be nit-picking definitions, but I don't think that's a fair
characterization. From the perspective of the user programming with async,
when you gather the results of two coroutines you don't care at all the order
used for each stack frame. The mental model is one of concurrent operations
which might suspend at yield points.

------
zwieback
What this really highlights is that you need to learn the basics to make the
right decision for your application and platform. There's no one solution for
everyone but if you don't take the time to understand context switching and
the role of your OS then you will be doomed to tilting at windmills.

------
luord
Didn't know that about Flask. One learns something new every day, even with
tools one's been using for years.

Actually, I gotta take a closer look to greenlets in general.

------
ds0
tangentially, I'd like to give thanks to Miguel Grinberg for getting me
started on flask and sockets in python via his posts.

------
shermanmccoy
Any emacs users know of a linter that will tell me when I forget to await a
coroutine?

~~~
war1025
Pretty sure I added a lint rule to pylint the other day that will do just
that. I imagine emacs has a pylint plugin.

Basically you add a linter that overrides `visit_callfunc()` and make sure
it's not an async function being called.

~~~
shermanmccoy
Cheers.

