- node is also often the best for - 'backend for your frontend' ( batching/caching, ...) which adds a bit more complexity
- structured logging should really be tracing given the heavy async + services nowadays
- This architecture is normally done as part of some scalable m:n mapping, which is alluded to, but can be a full article to itself (tools, patterns, manual/auto operations, ..)
We have been moving Graphistry's regular stateless web backends to python as it converges with node on async but is easier to do our GPU + AI data services (and other reasons).. but been generally keeping our 'backend for the frontend' parts here. This could have been part of a primer in our internal docs, so bravo!
I’m a huge huge proponent of tracing but god I wish it wasn’t necessary. The inability for most programming languages to serialize and produce a sensible async stack trace is a source of constant pain that tracing bandaids over. The state is right there dammit! There’s one thread, the event order is centrally controlled and serializable, you have all the stack frames, just put them together, don’t make me keep a log just to know what actually happened in a program that isn’t at all random.
I’ve seen a few Express applications that were mostly the same.
I also cringed at that premise. JS sucks a lot, having it on both sides would make things worse. Yes TS can help here, but only so much. You cannot make up for a languages design mistakes by bolting on a type system. Also: making a SPA-like browser app makes thing think about the state of the application twice! This comes at huge cost. Simple page-to-page applications with SSR are waaaay easier to build an maintain.
When I do browser apps I prefer a strong-typed API spec from which I generate (1) client libs and (2) server stubs. This gives me some safety guarantees over the API barrier.
Using a language like Elm with an "elm-graphql" generated client lib is a bliss. Compile gQL queries in a type safe manner: if it compiles it runs. I feel there is soo much more "tao" (simplicity, peace of mind, zen, less moving parts) in this approach than in the JS-article I just read.
I've been using Elm since it came out and I use it for everything, but you could write a "Tao of Elm" that's just as in-depth as this. Just watch rfeldman's talks about how to structure large Elm projects (https://www.youtube.com/watch?v=DoA4Txr4GUs) or watch the community chat about how to do it, it's an open question.
For example, the Tao of Elm would give pointers on what you should send over ports, how granular you should even make ports, how to pick Msg types (LogIn User | LogOut vs. SetUser User), how to build a good Model, when to nest models, (a big one:) how to build parent/child communication, what should be implemented on the JS side instead of Elm side, how to represent fetch/wait/response loop when making http requests, etc. and then the trade-offs of all of those suggestions.
Being able to compile a list of best practices doesn't mean something sucks. Though the comparison to Elm isn't all that great because it doesn't even compete with Node, and if you do use it on the server with Elm.worker, you could write a whole other guide of best practices around that.
Anyone who has built a massive Elm client is shimmering with trail/error knowledge that they could share in this kind of format.
Anyways, be wary of developing a knee-jerk "JS sucks" response.
Well, that is not where one usually uses multi-threading (OS level threads), as the threads would possibly be bottle-necked by IO and there would be no use of having the overhead of multiple threads. IO does not become magically faster by using multiple threads.
Which languages "rely" on doing that? Many popular languages have green threads and many have asynchronous concepts. I would assume, that programming language designers are aware and do not prescribe using multiple threads for IO of multiple concurrent "execution units".
Note: Little typo in the TOC: "Cetralize error handling".
The real success story behind node is libuv. It's a shame it was married to such a mediocre language, but that language did have some of the least-bad async ergonomics at the time (nearly by accident - client-side JS had to yield to the browser frequently, and the language was mostly designed to run small code blocks instead of application lifecycles).
Thank you for providing code examples of both what not and what to do. So many of these architectural principle articles just stay floaty and academic instead of having practical examples.
For people who generally follow this approach, what's your take on structured logging transaction traces, or more generally, propagating request-contextual information through the call stack? TFA seems to prefer a res.locals approach, but it's coupled to the framework in a way that the article avoids elsewhere, and needs manual propagation beyond the express handoff. I've seen these alternatives:
One team in my organization uses explicit context objets that get passed to every function call as the first argument, in a style very similar to what is idiomatic in Golang. I find this to be viral and not to compose well with third-party code that ends up calling your own code, which is rare with npm modules but somewhat common with company-internal libraries.
Another team was an early adopter of node async hooks to create the equivalent of a continuation local storage, with cls-hooked at around the time of node 8. They reported several problems that were implementation related, but seemed to prefer the developer experience of the approach, and context was preserved through third party code just fine.
E: (Apparently this now ships standard in node with AsyncLocalStorage)
Starting with just the handlers and introducing layers as you make sure its time is great way.
It reminds me of this story about a secretive master kung fu artist who perfected his technique but taught his style to no one. Yet the master still wanted to document his style in a kung fu manual. So he did, but as a security measure, he wrote everything backwards so that anyone reading it would get the wrong set of instructions. If a move involved a user blocking a hit with his arm he would write down that the hit should be blocked with his foot. Every move was a complete reversal of what should be done.
One day a thief stole the kung fu manual with the goal of reading it and becoming a great kung fu master. The thief read the manual day and knight and practiced just as much. And guess what? the thief became an incredible master surpassing the original author himself. Nobody could figure out his hilarious style, when you tried to punch him in his face his foot would immediately come up to deflect the punch. Everyone was WTF? How does this shit work? The thief ended up founding a kung fu school and taught everyone this style... proliferating the backwards art all across China, much like the same way Node has proliferated across the world.
I made the last part up because I thought it would be a more apt analogy. What actually happened is the thief went crazy because the the backward moves caused his chi to spin in the opposite direction... driving him insane. Anyway, this is what the Tao of Node reminds me of. I should also say so does the Tao of CSS and HTML and maybe the Tao of the entire web reminds me of this story.
Do these things really need to be associated with Tao as if it's some great art or philosophy? Feels a bit pretentious, but that's just me.
Thank you for taking the time and writing this!