One thing to note is that async programming like in JavaScript is "threadless" programming which means all your data-structures are by definition thread-safe.
It's one less complication in your logic when you don't have to think "what if some other thread modifies this data while I am doing it?". And you never get hard-to-detect and hard-to-fix errors caused by that nor do you need to try to set up locks that might cause performance problems.
So some might say that threads and locks and synchronization primitives are the solution while others might say they are the problem.
Note though, that many of the problems that come up with threads can still come up with single-threaded async, especially when building a server. As soon as you await, any object visible to another promise chain can potentially change out from under you, and this can cause analogous problems to multithreading. One example where this can get hairy is if you have a database connection shared between two promise chains running "in parallel". You'll get undefined query ordering, which can cause very bizarre behavior.
Don't get me wrong: it's still easier. But there's an interesting property I've noticed of statements of the form "by definition, problem X is impossible when you have Y", which is that X-analogous problems absolutely can happen, and they are often more complicated to solve. For another example: "by definition, memory leaks are impossible when you have a garbage collector".
Concurrency !== parallelism. You can have concurrency, as in async javascript, and all the issues that come with concurrency, even when no two execution paths are executing simultaneously. The main benefit is that your lock structures don't have to worry about being accessed in parallel.
Valid point that async is easier than parallel. I've done a lot of both, and parallel is harder. But, I realize that some people say that concurrency in software is not parallel, but I think that's a poor choice of terminology that is prone to confusion and argument. The word concurrent is literally synonymous with "simultaneous". We have a good word for asynchronous. Let's use async instead of saying that concurrent means async but not parallel in spite of it's common meaning.
> Let's use async instead of saying that concurrent means async but not parallel in spite of it's common meaning.
No, let's not. They mean different things. The execution of Python generators is concurrent with your code, but not parallel or asynchronous. And it's also completely possible to have concurrent synchronous parallel execution [1].
I think you agree with me- the comment I replied to appears to conflate concurrency and asynchrony. The article is about async. I’m saying use async when talking about async.
And I avoid using concurrent if I can. It does not have a strict CS definition, and it is prone to miscommunication. Many people use “concurrent” when they are talking about parallelism. Some people’s CS definition of concurrency is not very well matched with the common non-CS definition of the word.
The dictionary definition of “concurrent”, according to Google, is: “existing, happening, or done at the same time.”
> I think you agree with me- the comment I replied to appears to conflate concurrency and asynchrony. The article is about async. I’m saying use async when talking about async.
I'm not sure we do. The comment you replies to does not conflate those. "You can have concurrency as in async Javascript" is exactly correct: an example of concurrency is async in Javascript. People don't say "an example of asynchronicity" because that's not what they want to talk about. You can't really change the word unless you change the topic they are trying to talk about.
> And I avoid using concurrent if I can. It does not have a strict CS definition
It very much does. It means an operation overlapping in time with another one, i.e. one beginning before the other one ends. The colloquial definition is irrelevant in a technical context. It's like suggesting scientists stop using the word "theory" on the grounds that laymen don't use it properly. No, people should just use the right word in the right context.
To deal with this exact issue, I wrote a 'Promise Queue' for use in our codebase. The implementation of this queue is simple. It is initialized with a resolved promise. Each operation (i.e. a function that returns a promise) is enqueued by calling `.then(theFunction)` on the queue, while replacing it with the returned value from the `.then(...)` invocation. This ensures that two different operations do not interleave.
You lose some potential speed this way, but you are able to reason about the states your program will take far more easily.
That sounds sort of similar to what Selenium webdriver does with promises, although it does it so that the promises can be automatically managed without chaining .then calls. In webdriver's case, this causes async/await to deadlock. Does your queue system have that problem, or is it something you had to work around?
I ask because at work, we have a lot of browser tests that we'd like to start transitioning off of that deprecated queuing mechanism. If I can rip out Selenium's version and provide my own mechanism that doesn't deadlock on async/await, it would make the transition a lot easier.
var lock = new AsyncLock();
(async () => {
for await(var _ of lock) {
console.log("here 1a");
await sleep(1000);
console.log("here 1b")
}
})();
(async () => {
for await(var _ of lock) {
console.log("here 2a");
await sleep(1000);
console.log("here 2b")
}
})();
/*
When Lock is working should print:
here 1a
here 1b
here 2a
here 2b
Without the lock it would print:
here 1a
here 2a
here 1b
here 2b
*/
It requires Chrome 63 (at this point Chrome Canary) because it uses async iterators. I used this because I can close the lock automatically in case an error is thrown, the loop is broken. It only iterates one value and only after the lock is unlocked.
Not sure if this helps but I hacked this together the other day to avoid adding
a real queue to a very simple application. It uses event queue ordering
semantics to get FIFO behavior.
The read/write combinations to the semaphore variable need not be atomic since we know
that if this thread is running there are no other threads that can be reading
and then modifying it.
MyObject.queueSemaphore = N;
MyObject.emitter = new EventEmitter();
MyObject.queue = function( ... ){
// if this run can take place go
// otherwise wait for a run-complete to try again
if(this.queueSemaphore > 0) {
this.queueSemaphore--;
} else {
// must return a promise that resolves and possibly requeues
return new promise((resolve, reject) => {
this.emitter.once("run-complete", () => {
resolve(true);
});
}).then(() => this.queue( ... ));
}
this.promiseBasedProcess()
// ... work
.finally(() => {
this.queueSemaphore++;
this.emitter.emit("run-complete");
});
}
Almost. In threaded programming a thread will wait/block until the lock is released and then make use of the protected resource. In async world you would probably just check some flag and if it's "locked" then register a callback to some "unlocked" event to mimic this.
Not gp, but I used Twisted Matrix for a few years. It is usually implemented with a boolean flag and a list of client waiting to be called back when the lock is released.
So users of the lock would call acquire and get back a deferred (a promise in other language), but say in the most basic way, a callback is passed to it. When lock can be acquired, that callback is called (or deferred is fired).
When done with the resource, must call the release. Release then will trigger the lock to inspect the list of available waiting clients, pick one and fire their callback letting them know they can access the resource, and so on.
I don't have access to the code anymore, unfortunately. But it kept track of a list of people waiting on the lock and then notified the first caller off the list when the lock was released. It was only beautifully simple with respect to what it would have been in a non single-threaded evnironment.
I needed it in order to use a library that had some annoying global state in it.
With asynchrony you get out of order execution and non-determinism by definition.
That you don't have the concerns of shared memory concurrency, that's actually not such a big relief, especially given the right abstractions. And the underlying problem, that you need to deal with concurrency, doesn't go away.
The reception was actually negative, having lots of people telling me that I'm concerned about a niche use-case, but functional programming isn't a niche at all, being one of the two known ways to write reasonable, composable code that deals with concurrency and asynchrony, the other way being to have a draconian borrow checker, like that of Rust.
Mainstream developers are in general deluding themselves that the tools they are using are adequate, in spite of repeated evidence to the contrary, whereas most current breakthroughs in productivity were born out of functional programming, starting from garbage collection, but also actual libraries that people are now fond of, like React.
Haskell does shared memory concurrency. The JVM / Scala does that too. In Haskell you work with things like IO, MVar, STM (transactional memory) and many more. High level abstractions that have been borrowed by Scala as well, my current languages of choice.
I feel much more confident writing concurrent Haskell or Scala code, with 0% test coverage, then I am writing JavaScript code with 100% code coverage.
> all your data-structures are by definition thread-safe.
It is thread safe from data being accessed by two different threads, well because there is only one thread running. But surprisingly it is not free from application level data races in general, and you might still need async mutexes or lock type constructs.
That is because it is possible to have multiple callback chains each partially modifying the same data structure, or trying to write to the same tmp file, or modify some global resource.
In general with callback programming each callback chain specifies a sort of a poor man's IO concurrent context of execution. In a thread you spawn a thread to execute some code in a function, so it executes instructions A, B, then C. And there can be another thread executing the same A, B, C instructions in parallel. With callbacks it's the same idea, a chain of A, B, C callbacks can execute IO requests in parallel. Two client request coming almost at the same time, would run through A, then B, then C functions. A, B, might be called by first request. Before it finishes, and calls C, another request might start A, B chain again.
IMO thread programming via locks, mutexes and such is in the same category as assembly programming - they're necessary low level primitives that are extremely easy to get wrong, and hence they should ideally be left to specialized experts, who can use them to develop higher level abstractions (e.g. worker pools, event queues, tasks) that normal application developers can use for most of their use cases.
Agreed. I've been programming primarily in Scala the last few years, which has great high-level concurrency abstractions, and the code honestly stays readable and easy to reason about. We get the odd concurrency-related bugs, but they're reasonably rare, no more common than people screwing up things like time, text encoding, not properly using db indexes, etc.
Thread-based concurrency need not be overly scary. Any concurrency model will take some time to wrap your head around, but with the good ones, once you get comfortable with them, they should feel very safe, intuitive and deterministic. Scala's approach of "Futures + immutability when you don't need mutable state, or Actors when you do need mutable state" has worked well for every problem I've encountered in the last few years, and I've certainly heard great thing's about Go's approach too.
ES6 has improved the state of concurrency in JS, it's now similarly easy to read and reason about as Scala's model, but the lack of actual parallelism is a pretty big limitation - often a deal breaker for server-side code, and still often an issue client-side.
> but lack of actual parallelism is a pretty big limitation - often a deal breaker for...
Well, in my experience, it has been a deal breaker in a select few, rare cases. But often?
Do you routinely come across problems that could NOT be solved using a multiprocess approach (in Node) or webworkers on the client side? Could you share some of these scenarios?
All the time. Web development is a sad excuse for experience in HPC environments. But node guys always think I/O processing in an event loop must work for everything!!!!
With that approach we can go back to the happy openmpi/mpich days and take the gargantuan performance hit that ended MPI clustering for any serious real time scientific processing.
30 arc second permutation on a TC forecast track as it impacts multiple points of interest across multiple impact models. Heavily parallel amenable and in RT not an async endeavor.
Not giving you more 'hints'. You webdevs are kind of dull anyway. I/O this and I/O that and you'll accept untrusted input until the sec industry all drive ferraris on your mistakes.
Disagree. Threaded programming is not that difficult. Design matters. If you are trying to back fit threads into legacy code that has a traditional process based model with massively shared data structures it can be a little ugly.
Most peoples problems come with being over clever with their per thread data/stack, priorities, affinities, attributes, etc..a simple threaded program with a couple of locks for shared data and a flat schedule is efficient and easy to understand on multi-core systems.
Event loops and aio are nice but they are not a superior standalone solution as evidenced by nodes use of libuv.
Stupid question. Does this mean that a web server written in node is running single-threaded? I know that there are callbacks and promises to hand over control between different execution flows. But doesn't running on a single thread put an upper limit on the amount of work a server can handle?
And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently? Much like, say, Python Django?
> Does this mean that a web server written in node is running single-threaded?
Yes.
> But doesn't running on a single thread put an upper limit on the amount of work a server can handle?
Is not about how much work it can handle, it's about how much it can offload. Async servers can handle a much greater volume of I/O bounded tasks. So it can handle more connections. When the task is CPU bounded you can either create a thread (which does not really scale well) or offload it to some other servers that can scale horizontally (i.e doing micro services)
> And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently? Much like, say, Python Django?
Yes, to take advantage of all CPU cores you have to create more server instances. But why would they talk to the same database instance? It could be a replica or a shard. You can even have a pool of shards connections per server instance.
The problem with offloading to a thread is that you can't structurally share data with another thread. So any copying of e.g. arguments can be very expensive, and awkward/inconvenient as well.
If the other thread/process is running on a different machine (or you want to keep that option open) that's what you're going to have to do anyway though.
>Sometimes you want to use a thread to keep the CPU from blocking (e.g. long database operations).
On the contrary. That's the prototypical use case for non-blocking event based IO. No threads needed.
>Also, sometimes you want to use the CPU to its full capabilities.
You can use all CPUs by using multiple processes. That's not an issue.
Threads are useful when you want to run multiple algorithms in parallel on the same bigish in-memory data structure, especially one that has a lot of pointers.
Something like an in-memory graph database or a desktop application that lets you work with huge files in-memory, or even complex user interfaces. For instance, I'm not convinced that the cross process bridging that React Native has to do is a great idea.
So yes there are use cases where threads are very beneficial. But on the server side it's essentially the database/analytics systems themselves, not the code that accesses them.
Database access, likely the most common I/O operation from a web server, is certainly running in another process (or on another machine) - I'm not sure that "if" is as big as you suggest.
> > Does this mean that a web server written in node is running single-threaded?
> Yes.
Just to expand, this (like most things) is a simplification, as I'm sure nitely is just being too brief to explain. It's true for Hello World, and a little further, but real-world web servers in non-trivial contexts typically utilise techniques like clustering, workers, and other ways of delegating tasks to external processes.
>Yes, to take advantage of all CPU cores you have to create more server instances. But why would they talk to the same database instance?
Why not? Unless you've exceeded the capacity of a single DB and have a real use for sharding etc, it would not make sense to have difference DBs (+ replication overhead) for different Node processes.
The parent is asking how comes async I/O can handle higher volumes of requests given it's single threaded and even if it does how comes the database is still not the bottleneck. I answered both of these questions.
Yes, and I had an objection with the answer to the second question, that it might leave the impression to the parent that sharding or replication and pooling is required to have good DB IO performance with multiple Node processes -- when in practice it might or might not be an issue.
You can have a 12-processes node cluster and still not need a second db.
>Does this mean that a web server written in node is running single-threaded?
Yes -- node is by default a single threaded, single process server.
>But doesn't running on a single thread put an upper limit on the amount of work a server can handle?
Not any more than this is the case with Python, PHP, Rails, etc -- which also don't do multi-threaded (or don't do it well and not by default), and which on top of that don't have asynchronous capabilities (again, not by default) and are even less performant than a single Node app.
Which is why a simple Node running with its single process and single threaded execution can e.g. beat a Python server with two dozens of workers (e.g. gunicorn) in handling simultaneous connections (assuming Node code is properly async in the most part).
>And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently?
Databases take care of serialization of multiple queries for you -- and for more complex cases (with or without transactions for fuller control).
>Not any more than this is the case with Python, PHP, Rails, etc -- which also don't do multi-threaded (or don't do it well and not by default), and which on top of that don't have asynchronous capabilities (again, not by default) and are even less performant than a single Node app.
uWSGI makes running a threaded or multi-proc python webapp trivially easy (and as of Python v3.6 async comes as standard)...
>Which is why a simple Node running with its single process and single threaded execution can e.g. beat a Python server with two dozens of workers (e.g. gunicorn) in handling simultaneous connections (assuming Node code is properly async in the most part).
Node can beat a threaded python app for sheer volume of concurrent connections to clients, yes. But for a lot of traditional backend work (e.g. talking to a DB) async is no faster (indeed it's often slower) than a threaded approach.
Node (or async in general) is great for terminating inbound client connections; talking to local, in-memory caches or making backend calls to remote, non-local REST services.
For making local DB connections or doing any CPU work (e.g. parsing XML docs returned from an API) single-threaded async rarely yields better performance over threads/multi-proc. A good illustration of this is pgbouncer (async on the client facing end; threaded - i think? - on the db facing side).
Basically, all node really does is reduce the number of front end app servers you need to serve X incoming client connections. Just because node can handle a high concurrent connection count doesn't mean the rest of your backend services can. Regardless of whether connections originate from a single node instance or a large fleet of php/python/rails servers; you still need reverse proxies like pgbouncer/haproxy/twemproxy/squid to manage and shape those connections before they get to things like your DB or internal micro-service APIs.
Because node is single-threaded you also need to keep a very close eye on any CPU bound activity to avoid blocking all your connections. This is not always obvious and can crop up in unexpected ways (see: https://news.ycombinator.com/item?id=15477419)
>I thought the difference was in fact that PHP fires up new threads for each connection.
PHP doesn't get to decide what happens for each new connection. That's determined upstream, and there's a lot of different pieces of software people choose to do that. Could be directly a webserver like apache's mod_php, or something like fastcgi, php-fpm, etc.
All of those front-ends chose to implement a model of a php process pool. Multiple processes, each with a single php interpreter running. Incoming connections are sent to a process in the pool. So, if the pool is 5 processes, and you get a 6th concurrent connection, that one waits.
Node is written using libev, which under the hood uses system calls like select, poll, epoll etc (whatever is fastest for the combination of the IO task at hand and the current kernel) to provide an abstraction called an event loop.
It acts as an intermediary between your application code and the kernel, notifying you as soon as some IO action was completed by the kernel via an event callback.
This notification is provided to you as a single queue of events; the event loop is hence single threaded.
The important thing this facilitates is making it easier to reason about your application code, since you can be assured that only one of hundreds of callbacks in your application can be running at any one time.
Does it put an upper limit on the amount of work a single server process can handle? Depending on your use case, possibly yes. NodeJS shines when most of the work each call to your server involves mostly IO, i.e. are IO bound tasks.
If on the other hand, if any of the calls are CPU bound (some complex mathematical calculations say), you're probably going to hit this limit much sooner.
Even in cases where you have to run CPU bound tasks, it is far 'simpler' to offload these to an entirely different process that uses some sort of IPC to run the calculations and communicate the completed results back to your main server process, rather than spinning up a new thread in your server process to handle those CPU-bound tasks.
Does it now not possibly involve multiple threads accessing the database concurrently? Well, yes. Databases though are rather good at handling races. Most mainstream databases provide some sort of locking mechanism to make sure that some shared record cannot be erroneously modified by two processes at the same time.
If database locks are not for you, there are other solutions possible for these kinds of issues as well. By implementing a proper message queue, you can filter out calls that access this shared record into a separate synchronous queue, while all the other calls can be made to the DB simultaneously.
Why bother with mutex locks and races in your application code when other people (authors of libev/databases) are willing to do it for you?
"In our tests, however, we found that React’s renderToString() takes quite a while to execute — and since renderToString() is synchronous, the server is blocked while it runs. Every server side render executes renderToString() to build the HTML that the application server will be sending to the browser."
"The average renderToString()call with this configuration took 153.80 ms."
> Does this mean that a web server written in node is running single-threaded?
JavaScript is single threaded. Node.js is not.
Node executes the entire JavaScript code in a single thread. However, the I/O requests dispatched by the JavaScript code (file I/O, network I/O, db I/O) can be executed in separate threads.
That is why Node is efficient for applications with lot of I/O (typical web apps), and can still provide a default thread safety for global objects.
But I think, it will not work efficiently for number-crunching CPU intensive applications.
NodeJS userland is effectively single-threaded, but NodeJS is not single-threaded. NodeJS outsources some event loop scheduling to libuv, and libuv is multi-threaded (by default, 4 threads, but this can be configured. See:
I'm a noob to concurrency, but my current understanding is that a "thread" is "virtual". Fundamentally, your CPU has finite cores so no matter how many threads you spawn your (individual) CPU cores will resolve the threads into a synchronous set of instructions to execute.
Having a single threaded (asynchronous) server simply displaces the abstraction of threads.
Please feel free to correct if I'm wrong about this...
No, async & single-threaded is not an abstract replacement for threads. Single-threaded async code can only use 1 CPU core at a time. For example, JS running in the browser without web workers. Multi-threaded uses multiple cores simultaneously.
Whether a thread is "virtual" - I don't know what that means to you, but threads are a primitive the OS provides. Some CPUs have hardware support for threads, and some don't. So, I suspect they are less virtual than you're thinking.
Yes, but you need at least as many thread or processes as you have cores to take advantage of them.
A multicore cpu is wasted on single threaded program unless you are lucky enough that your problem is so embarassingly parallel that can be handled by N indipendent the processes
Node doesn't eliminate parallelism, it just pushes it outside of JavaScript. A node-based system is parallel at multiple levels: within the process itself library calls can run in parallel, and like you suggest, you have many parallel processes running instances of the same code.
Worth noting that it is possible to have threads without the complications of concurrent modification. You just need to pass things between threads as immutable values. Tcl does it this way (prior to having threads added, it used only an event loop, like JS).
I'm seeing a lot of confusion about the internals of NodeJS, so I hope to clarify:
NodeJS userland is effectively single-threaded, but NodeJS is not single-threaded. NodeJS outsources some event loop scheduling to libuv, and libuv is multi-threaded (by default, 4 threads, but this can be configured. See:
"Libuv by default creates a thread pool with four threads to offload asynchronous work to. Today’s operating systems already provide asynchronous interfaces for many I/O tasks (e.g. AIO on Linux). Whenever possible, libuv will use those asynchronous interfaces, avoiding usage of the thread pool. The same applies to third party subsystems like databases. Here the authors of the driver will rather use the asynchronous interface than utilizing a thread pool. In short: Only if there is no other way, the thread pool will be used for asynchronous I/O."
It's possible there is some internal enforcement, inside of libuv, of timeouts, because there are some bizarre problems that come up when NodeJS is under enough load. See:
"A surprising NodeJS failure mode: deterministic code becomes probabilistic under load"
For example in Chromium there's threads for GC, JS compilation, IO, image decoding, audio processing, video decoding, rastering, compositing, animations, indexeddb, and more. It also uses a thread pool and a task scheduler that handles prioritization both on the main thread (where the DOM is) and inside the thread pool. (ex. Handling touches is usually more important than handling xhr on the main thread).
Other browsers like Servo and Firefox Quantum use the thread pool for handling layout and style tasks as well.
Some hard earned wisdom: Callbacks (whether asynchronous or not) have the advantage of being extremely modular. The callee can just do what needs to be done - since the contract is simply a call to a function with a very simple signature, this will hardly ever be a (dependency / maintenance) burden.
I suppose experienced developers do as little as possible in callbacks (basically store the event away and make a note that there was something, at a central place). Then try to do everything else in easy to understand main control flow, some time later.
> I suppose experienced developers do as little as possible in callbacks
(talking about C/C++ mostly)
I have personally come to embrace the opposite approach: let callbacks do absolutely anything someone could do outside of a callback. This way, code gets simpler and clearer. For example, if a Socket object contained in HttpConnection calls back to HttpConnection telling it the connection is dead, the HttpConnection class should be able to just do something like "server->connections.remove(this); return;". Yes that remove is HttpConnection self-destructing; after that statement HttpConnection and the Socket no longer exist.
Ensuring safety of this is really simple: whenever you call a callback, the next statement must be "return" and more specifically you must return back to who called you without touching anything which might be dead now. Or perhaps more idiomatically you can "return callback();" when both the callback and the calling function return void.
In the case that you would want to do something after the callback is done, instead ask the event loop to call you "soon", and make sure your destructor will cancel that. For example this can be just a timer with zero timeout, or a special event queue designed for this purpose (in which case LIFO scheduling will get you natural "recursive" evaluation order).
> I suppose experienced developers do as little as possible in callbacks (basically store the event away and make a note that there was something, at a central place). Then try to do everything else in easy to understand main control flow, some time later.
This is why there's a trend toward frameworks that are a giant pub-sub, like React/Redux. The pattern is essentially to forward UI events (mouse clicks, etc) to the pub-sub, and that way you can keep all code together in the main control flow, in a somewhat "functional" style.
Async is necessary sometimes... but I'm really hating the trend of designing APIs that make it unavoidable, with the assumption that it's necessary always. It just makes code illegible and thus buggy.
And strangely enough, people still reinvent the wheel, reproducing a scheduler in userland, which is both insane in term of code complexity and debuggability, and in term of performances.
Yes, although to be fair they tend to do that because kernel threading never quite does what it needs to do (e.g. doesn't scale to the # threads applications need these days).
Your comment is spot on though: I'm only aware of two modern implementations that really work : Erlang and Go.
Kernel threading does scale to the number of threads applications need these days.
What can be slow are context switches, but they aren't slow in absolute terms. The vast majority of applications, including Web servers, are perfectly fine with 1:1 threading.
It doesn't really scale. Kernel level threads are quite expensive in the terms
of memory. Erlang's processes take up few hundred bytes. It's rare to have
dozen thousands of kernel-level threads running on Linux, while it's quite
common for Erlang servers to have that many processes.
It's not really up to a distribution to lessen this cost. Large part of it
is what task descriptor in the kernel takes.
Also, if nothing else, you'll run out of PID numbers, as they're usually still
16 bits, even today, though there was a kernel compilation option to change
that, from what I remember.
I wrote a server application that runs with about a million goroutines and performs quite well. It's not a webserver, but it responds to "requests" within hundreds of micros. Surely this would not be possible with OS threads.
> Threads do scale. On Linux, O(1) scheduler solved this non-issue a long time ago.
yup they do.
till you start making sure that your code doesn't end up with deadlock, data-corruption, races, performance issues due to lock-contention etc. etc. designing efficient locking schemes is notoriously hard alternating between:
- too coarse grained : resulting in serializing activities which could have (should have) proceeded in parallel, thereby sacrificing performance and scalability.
or
- too fine grained: with space+time for lock operations sapping performance, error recovery and not to mention understanding etc. etc.
In the former we have the dragons of deadlock and livelock roaming freely, and in the latter we have race conditions. Somewhere in between is a razor's edge which is both efficient and correct.
Almost always, things start with ‘one big lock around everything’ and the vague hope that performance might not be abysmal. When that is dashed, big lock gets broken up, and the prayer is repeated. Each iteration increasing complexity and decreasing lock-contention, and hopefully with some luck, modest performance gain as well.
Having high numbers threads and switching between threads are different things. There is still a huge constant in front of that O(1) scheduler that makes it unattractive.
Full interfaces for managing threads are often unnecessarily flexible and complicated for the relatively simple use cases where JS programmers typically need some sort of promise, though. Even if it’s still OS level threads implementing the behaviour behind the scenes, if you’re only spawning a new thread to determine a value in some potentially time-consuming way and then return that value when it’s available, the concept of a function that runs asynchronously and returns a future value is quite neat and intuitive.
Async is always necessary in JavaScript, and always has been. (And yes I know it's possible to write a fully synchronous JS program. No front ends do that, because obviously the site would be unusable, and no backends do it because it's a bad idea- inefficient, wasteful, more code than needed, etc. etc..)
Which APIs are you seeing that are async and seem like they shouldn't be?
Promises make code more legible than callback chains, and async/await make code more legible than promise chains. Do you have any examples of buggy/illegible code and the more legible less buggy alternative?
I quite like async/await except that it's annoyingly easy to produce a deadlock, and given a snippet, it's not obvious that such a deadlock should occur.
Ironically it is essentially the event loop which causes the deadlock in C#.
Depending on what framework/runtime you're in .NET will schedule the await continuation on something called a "SynchronizationContext" which has ~3 different forms but it's basically an event loop/message loop which queues up each continuation on the original thread.
The problem occurs when you use .Wait() or .Result instead of 'await'. This causes the function to spin waiting for the Task to finish, which of course it never will if it has a continuation trying to dispatch into that same event loop.
This problem doesn't really happen at all under some circumstances, such as if the async chain starts on a background thread, or in .net core where they've removed the SynchronizationContext, hence no event loop, hence no problems.
Do you have an example? I’ve never seen a deadlock in Js. I’m trying to think of how it’s even possible? Async/await is just super around promises anyway.
Oops, should've specified C# specifically, in which deadlock has become notorious. I don't think there's anything wrong with async/await per se. It looks to me like the linearity of js engine event loops wouldn't produce the same problem.
Lua coroutines are a much better & simpler solution to handling async callbacks (and callback hell) than what JS offers. Here's a simple wrapper I made for it: https://gitlab.com/snippets/1678104
async function bar(){...}
function baz() {...}
async function foo() {
var x = await bar();
var y = baz();
return x, y;
}
Lua:
function bar() return something_that_may_yield_deep_inside(...) end
function baz() ... end
function foo()
local x = bar()
local y = baz()
return x, y
end
Iow, in Lua it is irrelevant whether something yields or not, so you don’t care if it is async.
I know that other code, outside bar(), may have run during the execution of that statement.
Also, the JS approach makes composing asynchronous operations simple:
var x = bar();
var y = bar();
return await x, await y;
Both bar invocations can run in parallel. If, say, each invocation of bar fires off an Ajax request that takes a few seconds to come back, that can be a significant saving.
It's unclear (to me) how that would be done in Lua without complicating the API.
It is a false security. If you call a function either you know what it does (so in lua you would know whether it call yield) or you don't and it could still be calling arbitrary code, so you have to code defensively and think about reentrancy. The only reason for await is that it normally needs to save less state (a single frame) than a full coroutine yield (a whole stack).
Also lua-like stackfull coroutines don't prevent firing multiple asynchronous operations at the same time (like in your example), they only make the waiting much more peasant.
OpenResty, for one, allows that via grouping queries, rather than results:
local requests = { "/mysql", "/postgres", "/redis", "/memcached" }
local responses = { ngx.location.capture_multi(requests) }
for i, response in ipairs(responses) do
print(response.foobar)
end
But instead ngx could lazy-evaluate responses with help of metatables (see my other comment).
In theory, someone could write a Babel plugin that wraps all files with `(async () => { /.../ })()` and prepends all function/method calls with `await ` (I think). I personally like the explicitness but maybe it's just because I'm used to it. I also wonder how that would work with `Promise.all()`.
We're now sorting out things for third-party js library written manually that way. It is a mess.
>Promise.all()
Objects returned from yields_deep_inside() may be implemented as lazy-evaluated, i.e. only `print(result.items)` or explicit `await(result)` will yield upon use, while request will be sent immediately. Thus the order of execution will depend purely on natural use case, not on programmed await sequences/groups. Since js is not parallel, you'll touch A or B first, not both.
How to sort it all out is a responsibility of an event framework, not of green thread abstraction that is coroutine.
One more thing: idk how js optimizes endless closures that are spawned as promise callbacks, but Lua coroutine's yield/resume is as cheap as return from / pass control to a VM. No closures are created to retain state across async calls, because VM stack is state itself. I suspect wrapping everything in async-await will simply trash GC and VM performance. In a sense, asynchrony is only emulated in javascript with a cost, though I may be wrong and it all is optimized out. We have to wait for jsvm implementation experts to [dis]prove that.
The entire green thread is paused at yield() and resumed when other [controlling] thread calls resume() on it. Event loop has to account which thread yielded on which event source and resume corresponding thread when event arrives. Program as a whole doesn’t stop, only sequential blocking paths are put on waiting list. If each incoming query starts a new thread, then all queries are run concurrently around event/io loop, implicitly switching on deep yield() that programmer never spells out loud. That’s how e.g. openresty (nginx+lua) handlers work.
Surely it's nice not having to worry about it, but how is that better over explicit await?
If a sub-sub-sub-function suddenly decides to yield, you might not even know that your whole path is waiting for that. Right? Perhaps you need baz() to run asap but that isn't clear without looking deep into bar().
One big advantage is that higher order code can work with both sync and async code. The simplest example would be a for each iterator but this also applies to anything else that calls a function or method that is passed in
Well, I do want to care if it's async. I don't think it's a good design. Async keyword is making asynchronous code easier, but it's still asynchronous code with its pitfall, I think that hiding it under ordinary calls is a bad design.
I mean, select(2) (and thus event loops) was in 4.2BSD ~10 years before windows 3.1 was released, and it wouldn't surprise me if there were earlier examples of I/O multiplexing.
The Javascript community is something of an echo chamber, and often thinks it is innovating when actually it has rediscovered spokes, rims, and axles, and is now starting to wonder if they can be combined into some form of rotating load-bearing device.
I think the prize for doing event loops without a big fuss and before it was cool probably should go to tcl. They also did multithreading right, whereas Python botched that up. Tcl model: if you need to, run interpreters in separate threads in the same address space. No need for GIL, no need to pickle uncle pickle every message.
When will JS programmers pull their head out of the sand and realize that this is nothing new? All this stuff was present in Windows 1.0 and they moved on since. Now we have multi-reactors like vert.x and concurrency models based on process philosophy (Clojure). Please wake up.
The first example in this article is misleading, imo, since the actual execution is synchronous, not asynchronous. The "programming style" is asynchronous, yes, but the execution will ALWAYS be the way its described since there are no other way it could be executed and therefore synchronous. Unless there are any way of things to happen out of order in relation to the execution flow, its effectively synchronous. If there is an possibility something can be executed out of order, its asynchronous. JS doesn't provide that because the event loop. It will always be executed in the order of the loop and that will be synchronous (even if and event re-queued because it's not done yet).
Some databases are able to provide serializability while not grabbing a global synchronous lock for all transactions.
Even if the execution must appear synchronous to an observer, it's sometimes possible to do some async work.
That could be because it's performing an operation which is atomic to the JS application (e.g. a sort operation can be implemented with parallelism because the JS app is paused the whole time it's running).
Or, the JS interpreter could perform static analysis on control flow and determine that two or more computations do not depend on eachother for a period of execution.
Of course the they can, but that's not the thing here and it's about JS. The "lock" here is the event loop, and that's the issue.
I don't understand the part where the database is doing work is in any way related to what the JS app is doing. Even if you are running on the same machine, they are bound to have different runtime scopes and not related on how they execute.
That could be because it's performing an operation which is atomic to the JS application (e.g. a sort operation can be implemented with parallelism because the JS app is paused the whole time it's running).
Or, the JS interpreter could perform static analysis on control flow and determine that two or more computations do not depend on eachother for a period of execution, and can thus be performed in parallel.
Or, as long as the JS isn't performing IO, the execution environment can use optimistic concurrency[1] and back out the changes if the codepaths did have interdependency or tried to perform IO.
But that is still not asynchronous in relation to the program flow. Running other unrelated tasks is not asynchronous like that, it's concurrent execution. Asynchronous would mean that something that share context, time or execution, is run out of order in relation to each other. And even if it's using optimistic concurrent execution, in relation to program flow it's still synchronous.
The event loop is still synchronizing program flow, even if it does optimistic concurrency control. Concurrent doesn't mean "out of order", it means "at the same time".
Chiming in a bit late here, but for years I have said that there are really only two kinds of UI that I want: a cli REPL (maybe with something like ncurses), and a full powered 3d game engine. Everything else in between is going to be a poorly implemented version of one or the other of those two, so just pick which end of the spectrum you want to be on for the purposes of rendering and user interaction, and build your browser apps like cli/ncurses, or implement them inside of quake3 and force your users to login by learning how to plasma surf.
One of the first things new computer users latch on to (including myself, long ago) is their graphical capabilities, and one of the first things budding geeks learn - which was once even more true than now - is that there is a vast array of incompatible dedicated hardware and software used for them. This disincentivizes learning, as no one wants to spend a bunch of effort acquiring knowledge that is useless in the future.
If everyone was using a similar 3D engine on similar hardware (or with the hardware abstracted away) for everything non-text, with comparable libraries, they could just jump in and start learning how to manipulate the graphical environment immediately as children, and build on those same methods anywhere they went for a lifetime. [The 2D stuff would obviously be a subset.] It would dramatically increase efficiency of all graphical development across the board.
Also, it became apparent to me very early on that windowed GUI environments were inherently less productive and reliable for common tasks than "something like ncurses" as you say (the old AS/400 interfaces come to mind, a great many of which are still in use for good reason). The 2D GUI interface is much more finicky & time consuming to design properly, as well.
In short, I couldn't agree more. One of the biggest frustrations I have with the entire field is how much time is wasted on counterproductive efforts. How many man-hours have been flushed away on 'perfectly aligning' graphical text boxes that never needed to be graphical in the first place? I understand how this is driven by underlying economic & social dynamics, but it's still galling.
I guess I mean poorly implemented not in the sense that they don't work (early 90's word is an engineering marvel) but the sense that in order to get them to work you are left with enormous amounts of baggage that will come back to haunt you in the future.
The 'classic' windowing toolkits and WYSIWYG implement the 2d portion of a 3d engine using a whole bunch of really cool hacks (necessary at the time) in order to run efficiently on a 286. Now days GPUs give us the luxury of not really having to worry about the cost of repainting the screen. The way QT, gtk, windows, etc. implement their interaction paradigms redraw/etc can all be subsumed by a 3d engine, and when it comes time to say, render a 3d graph inside of a hypertext document, then you don't face the huge abstraction impedance mismatch of having to drop into opengl because nothing else exits.
I'm not saying that you don't need the constructs that were developed to build the classic 2 interaction paradigms, in fact you will find most if not all of them recapitulated in every 3d engine. The point is that 3d engines are essentially the logical end of any input/rendering loop you can imagine and if you develop tools long enough the infrastructure and abstractions they provide will eventually be needed and if you were using a 3d engine from the start then you won't have to implement all that functionality from the beginning, usually without the benefits of decades of experience and tens of millions of users banging on your code.
To give a more concrete example -- think about how much it still sucks trying to get multimedia or good user interaction _inside_ of a word document. If word were implemented on top of a good 3d engine, then when you get the digital version of a paper the 2d projection graph is now suddenly trivial rendered because that is how it was created in the first place and the document format and editor have those capabilities from the start.
Somehow this reminds me of the first programming I ever really did in Squeak [0].
And there are runtimes that do that, like Erlang's BEAM. At least Node concurrency is a step in the right direction, as it's much easier to reason about than threads on the JVM.
There is a proposal for shared memory and atomics, which are absolutely opt-in at the moment. You can easily use the event loop if you don't need parallel processing.
It's one less complication in your logic when you don't have to think "what if some other thread modifies this data while I am doing it?". And you never get hard-to-detect and hard-to-fix errors caused by that nor do you need to try to set up locks that might cause performance problems.
So some might say that threads and locks and synchronization primitives are the solution while others might say they are the problem.