Hacker News new | comments | show | ask | jobs | submit login
What lies beneath async/await in C#? (foreverframe.pl)
175 points by goorion 401 days ago | hide | past | web | favorite | 65 comments

Async/Await is especially important to use in .NET and .NET Core web servers if high concurrency is desired. .NET uses a managed thread pool that will only spawn up to something like 50 threads quickly by default. If it sees that more threads are needed, it will slowly add them.

This means that a Synchronous web endpoint (think API call that does not use "Async/Await") will hog an entire thread for the life of the request. I graphed the difference between Synchronous and Asynchronous API endpoints in a blog post last year, and Synchronous endpoints were far less efficient: https://caleblloyd.com/software/net-core-mvc-thread-pool-vs-...

Note that just because endpoints are declared as "async" does not automatically make them efficient - Asynchronous I/O must be used all the way to the lowest level. This can be an issue when using library code that you don't control. For example, Oracle's official MySQL package provides "async" methods, but it uses Synchronous I/O in these methods. Therefore, "async" calls into their library will really cause synchronous performance.

Async/Await is a really cool feature of C#, but I wish that the language had more protection that forced users to do things "the right way", like disallowing Synchronous I/O in async methods. In order to write libraries that do I/O properly that include both Synchronous and Asynchronous methods, maintainers have to write two separate implementations of the same logic, which is a bit of a pain.

This is very true and a better explanation of the benefits of async/await than "Efficiency and Comfort" from the article. There are a lot of articles on async/await in .NET that use the example of off-loading a long-running operation from the GUI event thread. While that's a legitimate use, it obscures the true benefits which are most closely tied to throughput scalability. In other words, maintaining a relatively small pool of threads to service I/O completion events allows a machine to minimize the overhead of thread context switching that can become a bottleneck in a thread-per-operation model.

Additionally, many people have the misconception that async operations are somehow faster than their sync counterparts. In fact, they're generally slightly slower, but a fully asynchronous server will be able to maintain a much higher level of throughput, usually with similar latencies, than a fully synchronous server. In other words, the principal class of applications that benefit from async/await are high-concurrency server applications.

One more place I see these kinds of misconceptions is in naive benchmarks that purport to test async performance without using a concurrent workload. That tests only straight-line latency of individual requests and isn't the point of async. The benchmarks in the blog post above properly use a concurrent workload for testing.

> In fact, they're generally slightly slower

An async method is much slower (40x) compared to a normal sync method. If you're writing a high performance library you need to be careful to not repeatedly call a method decorated with async in a tight loop. Of course 40x times slower of extremely fast is still really fast but I found the overhead can add up really quickly.

Good video on async/await and performance: https://channel9.msdn.com/Events/Build/BUILD2011/TOOL-829T

Did that 40x number come from that video? In my experience, any non-trivial async operation (e.g., a 2ms database insert in a fast OLTP system) is going to be on the order of 10-20% slower in straight-line operation latency than a sync operation while handling vastly more throughput.

It came from the video but I have personally found async methods to be significantly slower most of the time (although I've never sat down and figured out the exact amount).

The reason for the performance hit is that a lot the time your async methods are actually completed synchronously, e.g. you read data from a FileStream with ReadDataAsync and instead of going to disk, the data was already cached in an in-memory buffer. When you hit that situation the overhead of the async - the state machine, not being able to inline the method, nesting everything in a try/catch, etc - makes it orders of magnitude slower than a normal method just fetching some data from a buffer.

If your method is always async and is already taking 2ms then it doesn't matter, but you want to avoid the async in methods called in tight loops in performance critical code. The overhead can be significant.

I'm curious where the 40x slower figure came from. It would really depend on what you are doing inside the method.

If the method is a small amount of "synchronous" code in a tight loop, then yeah, the overhead of the async call is going to outweigh any benefits (a few orders of magnitude slower).

However, if your method is doing something like reading in a 40MB file to do some processing and calls the ReadAsync methods then the difference will be negligible and you get the benefit of better parallel throughput.

It's also worth considering why .NET is conservative about spawning threads. The basic reason is that threads are really big objects. The default stack space allocation for a 64bit thread in .NET is 4MB, so you do not want hundreds of these things getting spawned when your traffic spikes. Since most of your threads handling requests would be spending most of their time waiting for IO operations (network requests to DB servers being the most common wait state) to complete, this is a recipe for having huge blocks of committed memory sitting around, each holding a small amount of stack state for a single request (most of the request data being strings and dictionaries will be on the managed heap anyway, the stack will just contain a few references to those heap objects), waiting for a DB query to complete.

A web server using few threads and async/await state machines can transfer those stack states out into managed heap objects too, and handle many times more concurrent requests without running out of RAM.

Aync/await just seems like really heavy handed solution to this problem. I wish .Net runtime provided lightweight threads (with low memory and switching overhead) so that it would be possible to tune how application runs without needing to modify the code.

if someone can chime in and say why .NET cannot have the lightweight threads like Haskell I would be interested. I am guessing because being pure/immutable there are no "stack frames" to store with the thread, just a reference to the (immutable) data passed in to a function.

You can find more info about how Haskell does it here: http://community.haskell.org/~simonmar/papers/multicore-ghc....

what language does "lightweight threads" exactly like this?

I think JVM with Quasar gets close. It uses bytecode instrumentation to support suspending and resuming method execution. It modifies methods so that they know how to store and retrieve their stack state on heap when they are suspended and resumed.

I think the Quasar API is not exactly what I described in my previous comment but the above building blocks are what is needed to achieve something close to it.

Haskell for one, but lots of others: https://en.wikipedia.org/wiki/Green_threads

he added an additional requirement "to tune how application runs without needing to modify the code."

So I assume that means not using things like forkio/goroutines etc.. You still have to tell haskell "fork a green thread here" instead of what he wants, which is it to be completely transparent.

Additionally, I think one can make the case the haskell green threads aren't very different from async await, in which "awaiting" is just creating some data that a scheduler maps onto a threadpool (rather like how GHC maps it's 'green threads' onto it's 'capabilities' via a scheduler)

Furthermore, if you actually want to pass data round in Haskell rather than just fork threads, it leaves it completely up to the user to figure out how to do that.. MVar's are canonical but noisy syntatically. the 'async' package is very nice, but gives the gives the same syntax OP is complaining about!

The transparency I'm after is that majority of your code can be oblivious to whether it's running in OS or green thread.

So, for example, you could configure your web server to use use green threads to handle incoming http requests but configure other part of the system to run with real threads (for whatever reason).

In the end, internally, it will of course end up similar to async/await - you need to capture the execution state and resume it which is what async/await does. I just want to be able to do it without rewriting majority of my code.

In a managed platform like .NET or Java there's no reason in principle why any code should be able to tell whether it's in an OS or a green thread - the decision of how to actually accomplish having multiple simultaneously active callstacks with pre-emptive scheduling is made by the runtime. Early versions of Java used green threads within the JVM rather than relying on native threads, presumably as part of an attempt to provide consistent write-once-run-anywhere behaviors, so this is not just theoretical.

I think the async/await model arrives as a reasonable compromise in terms of supporting an alternative to OS threads in that it allows you to get the benefits of green threads (i.e. that you can process a lot of different stack contexts simultaneously without needing to spin up a huge amount of OS threads) without getting into the messy situation of having a user-space scheduler pre-emptively switching green-thread stacks (which probably needs the scheduler itself to run on another OS thread and definitely needs to be done at a 'runtime' level) - instead, you just rely on co-operative scheduling, which can be done at a 'library' level.

> configure your web server to use use green threads to handle incoming http requests but configure other part of the system to run with real threads

this just seems like an odd feature imho, but sure i think i understand you better now.

I can't imagine the fun you could get into in such a system where there are both green thread and OS thread locking primitives available...


I would argue that async/await is far more important for native applications. I've seen plenty of apps doing significant work on the UI thread and hanging. Making it easy for developers to run things in the background is great for performance. Web browsers automatically add a level of asynchronous interaction.

It's also important on a web server but it won't improve performance for a single user in isolation. As you say, it allows more requests to be serviced concurrently. This won't speed up an individual request unless the server is already heavily loaded.

Edit: You don't need to build two library APIs. You can just build an Async method as the default implementation then have a blocking call to that as the synchronous method. I believe that all new MS APIs are now Async by default.

> You don't need to build two library APIs. You can just build an Async method as the default implementation then have a blocking call to that as the synchronous method

This can lead to a nasty problem: Thread Pool Starvation. I contribute to MySqlConnector, an MIT Licensed MySql driver for .NET that implements truly Asynchronous I/O. We did this at first, but had to do a large refactor because we hit Thread Pool Starvation in our high concurrency performance tests. Here's the original issue: https://github.com/mysql-net/MySqlConnector/issues/62

Stephen Toub has a pretty good article explaining why you shouldn't do "Sync over Async": https://blogs.msdn.microsoft.com/pfxteam/2012/04/13/should-i...

>>I would argue that async/await is far more important for native applications. I've seen plenty of apps doing significant work on the UI thread and hanging. Making it easy for developers to run things in the background is great for performance. Web browsers automatically add a level of asynchronous interaction.<<

You can achieve the same thing in a GUI app by simply off-loading the long-running operation to a separate thread. The same is _not_ true for a highly concurrent, I/O-heavy server workload. That's where async/await is necessary and not just a syntactic convenience.

Async/Await does not imply multiple threads. Often, the async version will use fewer threads than the equivalent synchronous code.


> In order to write libraries that do I/O properly that include both Synchronous and Asynchronous methods, maintainers have to write two separate implementations of the same logic, which is a bit of a pain.

A solution to this that I am particularly proud of can be found in the Nim programming language. It is a macro that transforms a single procedure into a synchronous and asynchronous procedure[1].

1 - http://nim-lang.org/news/e027_version_0_15_0.html#multisync-...

> Oracle's official MySQL package provides "async" methods, but it uses Synchronous I/O in these methods. Therefore, "async" calls into their library will really cause synchronous performance.

It's all about scale. If you're an Nginx server serving thousands of files concurrently, or a websocket server, you need async.

But if you can spare a thread for a connection to MySQL, it's the MySQL server that's going to be in trouble, because the server already uses a thread per connection.

Writing high performance applications on Windows has been an 'open secret' for long time. Good free presentation I know of are Rick Vicik ones:

  - https://blogs.technet.microsoft.com/winserverperformance/2008/04/25/designing-applications-for-high-performance-part-1/  

  - https://blogs.technet.microsoft.com/winserverperformance/2008/05/20/designing-applications-for-high-performance-part-ii/  

  - https://blogs.technet.microsoft.com/winserverperformance/2008/06/25/designing-applications-for-high-performance-part-iii-2/
A simplified explanation is that for good throughput one creates a completion port and binds a thread pool to it with as many threads as CPUs. Then all IO is done via the completion port. If the system is partitioned (NUMA) then one should create a completion port per node and partition the thread pool. I skip the details, read the links for a better explanation. The gist of it that the code written this way has no procedural flow when you read it. Instead of a sequence of function calls (read socket -> parse -> read file -> write socket), it must be a state machine which always reacts to some IO completion, modifies the state, posts some more IO and then returns the thread back to the pool (post socket read, socket read callback -> parse -> post file read, file read callback -> post socket write, write socket callback). Writing it as FSM makes it lot easier to understand.

C# async/await is like a DSL that generates the FSM while you write fluent sequential code, easy to write and easy to read.

so, what I'm seeing here is, use continuations.. i.e. async/await

Also worth reading to understand how Async/Await are implemented: https://blogs.msdn.microsoft.com/ericlippert/2010/10/21/cont...

And https://codeblog.jonskeet.uk/2011/05/08/eduasync-part-1-intr... is a useful reference, although I believe the implementation details have changed slightly since those posts were written.

I'm a heavy C# user, but I come from a classical background and I have yet to start using async/await stuff. My reason (and I know this is terrible and I need to get over it) is I can't make heads or tails of it when I'm looking at the jitted code in NTSD.

This question I answered many moons ago may be useful:


Combined with this approach to debugging using `.load sos`:


The SOS debugging extension makes following JITted code much easier.

Of course if you're looking at async code, or the body of an iterator method, or a lambda body that's capturing variables, you need to understand those high-level transformations first.

I was in the same position but I would recommend you start using it. Make sure you start with a small project and really step through each line so you can predict what will run in which thread. I notice that some people use async/await but don't really understand it which can cause problems in some situations.

It definitely makes life much easier. Can't wait to use it in JavaScript.

This, so much. My first deep dive into it I ended up in async hell.

Definitely start one call at a time in a small corner of a small application. Once you understand async Task Method() vs async Task<Result> Method(), then how the async methods all chain back together, you can expand it.

The funny thing is you can use async successfully without having a clue. But one day it will bite you hard.

This is really a problem for .NET itself. The pervasive availability of thread-static state is the only way async/await could really go wrong.

There is more that can go wrong. For example in a desktop app a button click finishes before the async code has been executed. The code looks like it's blocking but it isn't. I have seen that a few times now.

In addition, many ASP.NET server-side projects had a fun time making changes required to properly survive cross-thread transfer of per-request data. (This was possible in the past but rarely happened.)


You can see similar issues with lifecycle events on load... load/render may happen in different threads as well under heavy load, I've seen a lot of race conditions with these kinds of assumptions.. also, a lot of people tend not to understand static in a web server context as well.

I did a talk a while back on async/await in C# for anyone interested: https://vimeo.com/191077931

Thanks for posting! I believe you also have a Pluralsight course on the subject, which would you recommend for someone who wants to dive deeper with async/await?

I'd certainly recommend the Pluralsight course, it goes into greater detail and shows a lot more examples. Thanks for mentioning it!

One thing I really like about async/await is that it hooks into existing .NET infrastructure for issuing calls on other threads. As a result, you can implement your own synchronization context and control how the async callbacks run and what thread they run on - if you have your own job scheduler, you can run them inside that. Technically this means you can use async/await for single-threaded workloads, by running all the callbacks yourself on the main thread.

Writing your own synchronization context for these purposes is incredibly easy, as well: https://github.com/sq/Fracture/blob/ed408e0a84afb0c68d7e81be...

Any advice when saving new DB records ? We usually return the new ID in our return result, but it is not available when using async/await. Is it possible to send a second result with the ID and how does one create a listener for the second result in Javascript ?

Edit: Because we use async HTTP requests, we aren't blocking the UI. On further thought, the problem I described is not a good use case for async/await.

It can be a good use case, to increase the number of concurrent users you can serve. See my comment elsewhere in this thread.

You should still be able to output an ID when using async/await. It just means that the calling thread can do something else while it waits for the DB to get back to it. Execution will resume in the same place.

As mentioned elsewhere in this thread, for a distributed system you can use GUIDs to remove a immediate dependency on the DB. You can adopt eventual consistency and merge later, but this adds lots of complexity. If you do go down this route, add the GUIDs as an additional key and stick with something sequential for PKs, to avoid clustering issues.

Define your method like this:

  public Task<MyResult> ExecuteDatabaseQueryAsync()
    return yourDbObject.SomeMethodAsync();
  MyResult result = await ExecuteDatabaseQueryAsync();
The actual code in ExecuteDatabaseQueryAsync will differ depending on what data access technology you use. Often, the method is called something like ExecuteScalarAsync.

In addition to what nathanaldensr suggested I would add that you should try to architect your application in a way that you don't have to wait for the DB to give your ID back.

Using GUID as ID is an option, but there are plenty of other methodologies to generate the ID in the client and not be held hostage by the DB to get your IDs.

I was thinking of mentioning this, but I wanted to avoid given design advice for systems I haven't seen. In virtually all of my own projects, I use business-logic-generated GUIDs for IDs. There are a whole bunch of easily-Googleable benefits for GUIDs over database-generated IDs.

While I agree that using GUIDs here would work to solve the issue, I wouldn't blindly use GUIDs without looking at the performance impact. The use of GUIDs as the primary key in something like SQL Server for large tables could cause read performance issues down the line.

async/await is not just to avoid blocking UI. It's to avoid blocking any thread, so that it can be reused for a different task while yours is awaiting completion of some dependency.

It's not entirely clear what your problem is from that description. Why is the new ID not available when using async/await? If your "add a record" operation produces an ID in sync version, then the async version of it should not be considered logically completed until said ID is returned. Are you using some library that works differently?

Doing this with Go using goroutines and channels pretty simple: https://play.golang.org/p/fattakS6aj

But something struck me, the low level execution flow was much much more intuitive. I think this might be an example how simple syntactic sugar can make code much harder to reason about intuitively. I do write a lot of Go so maybe its just that... you tell me!

> But something struck me, the low level execution flow was much much more intuitive.

A), I don't view this as lower level, just more verbose and explicit.

B), the logic is the same, but it's harder to read the operations—not knowing go, `await` makes a lot more sense than `<-`.

Good feeedback, further thoughts:

>I don't view this as lower level, just more verbose and explicit.

Well in this case, aren't these the same? The Go flow shows the explicit spawning of a (go/co)routine and the reading and wring to a thread safe queue. The C# abstracts this so the Go is closer to what is actually happening (ie low level). I should point out, you say more verbose, but there isn't much, if any, of a line count hit here for Go compared to C#.

>but it's harder to read the operations—not knowing go, `await` makes a lot more sense than `<-`.

Fair point, but if you swap <- for await I'd think the Go is more clear for people equally unfamiliar with Go and if you are learning Go <- is a core operator you will learn early.

Is using the await keyword basically like a shortcut instead of having to add a callback method to an asyncworker and then get the result manually?

I am always curious which enhancements to C# are just syntactic sugar vs. requiring IL and/or CLR enhancements.

Microsoft released the CLR code required for async/await separately, enabling back-compat for .NETv4.0. The F# language has supported it back to .NETv2.0.


Async in F# works quite differently from C#. This article highlights some key differences [1].

[1] http://tomasp.net/blog/csharp-async-gotchas.aspx/

Yes, and it pays off to know that. It explains a lot of behavior you will see.

In short, yes.

The slightly longer story is also yes.

But there are some optimizations and tricks. E.g. if the Task is already completed the async method will just continue to the next statement instead of attaching a continuation that will later be called.

And in case continuations are used async await will automatically capture the current SynchronizationContext and will assure that the continuation runs on the same.

Is it just me or someone else sticks the infamous .ConfigureAwait(false) on every instance just to be safe? ;)

This. And I hate doing this. I've talked myself out of the performance gains, because it just makes everything feel verbose and hard-to-read.

I still wish they would've added a keyword for this. i.e. await-nocontext

I think that would be a big mistake to add a keyword. It apparently doesn't give much of a performance gain:


It's not just about performance gain. The real problem is that if you don't ConfigureAwait(false), you don't know where your continuation is going to be executed, and how long it'll take to get there.

Worse yet, posting to the original sync context increases potential for deadlocks. The UI thread often has to do something synchronously (e.g. invoke an API that doesn't have an async version), thereby blocking the event loop. If anywhere down the line it happens to wait on a task, and that task gets posted to the UI synchronization context, you have a deadlock.

Note that the answer you link to is specifically for ASP.NET, where there's no UI thread. But for libraries, you don't know if they're going to be used in a web app or a GUI app. So, the safest thing is to always ConfigureAwait(false).

Here's a async library I'm working on written on top of clojure/core.async. Currently ~ 100 LOC.


I think you are being downvoted because that looks more like a parallelisation library rather than an async/await implementation?

Possibly. I did not claim it to be an async/await implementation, though.

Applications are open for YC Summer 2018

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact