
Interview with Ryan Dahl, Creator of Node.js - WhiteSource1
https://www.mappingthejourney.com/single-post/2017/08/31/Episode-8-Interview-with-Ryan-Dahl-Creator-of-Nodejs
======
_nalply
A serious problem with Node was the callback pyramid of doom.

This means deeply nested callbacks such that the indentation wanders steeply
to the right. When you do loops or have exceptions programming got very
confusing. In effect you have to do a continuation passing transform. It's not
difficult but the results are utterly unreadable. For example a simple for
loop could be transformed into recursive function because this way it is
easier to keep state between callbacks. It's a nightmare.

What I found surprising in the early years of Node that some people in the
mailing list had a «Real programmers don't use Pascal» attitude.

Bruno Jouhier's developed streamline to simplify and automate the continuation
passing transform. One could almost program in a «blocking» style. I found his
work really amazing. However for that Bruno got attacked in the mailing list.

Then Marcel Laverdet developed fibers. A different and equivalent way to solve
the callback pyramid are fibers, coroutines and generators. This way we don't
need the continuation passing transform. But the reception in the mailing list
was at best only lukewarm. Anyway, the developers of Meteor saw the potential
and decided to base the server side of the platform on fibers.

And now JavaScript has generators and async/await. And lo and behold: with
these official solutions the hard core programmers swayed and accepted
«Pascal».

In my opinion, Ryan Dahl has missed an opportunity. Node was by large not
ready and finished when he left. He should have tried to convince the
community to find a solution for the callback hell.

I think I understand people like him. They are always on the lookout for fresh
ideas.

~~~
nas
This was easily foreseeable when he started on Node.js. The Twisted framework
for Python had existed for years (decades?) before Node.js was created. Anyone
familiar with programming in that style knew of the issues with "callback
hell".

I was the primary instigator of generators for Python. Part of my motivation
was to do something like CSP (e.g. call-with-current-continuation, call/cc)
without having to redo the whole language runtime and also kill off
alternative Python implementations.

Generators in Python are limited to a single stack level. They have been
extended to better support async IO coding. They work and are probably nicer
than using callbacks but are still pretty ugly IMHO. The message passing that
Erlang does is the best solution to this kind of concurrency. You can't really
bolt that on a side of an existing language though.

Anyhow, I think Node is successful largely because Javascript is the language
understood by web browsers. The non-blocking coding style is not some kind of
magic bullet as Ryan originally suggested it was.

~~~
k__
Somehow I never experienced callback hell.

It always felt like every other code-nesting problem to me.

I mean, nobody talks about conditional hell, everyone acknowledges that you
should flatten out your conditionals.

~~~
sah2ed
Callback hell was real, especially if you worked on projects that decided to
join the node.js bandwagon in its early days. Then promises happened.

[https://twitter.com/winterbe_/status/541868081308790784](https://twitter.com/winterbe_/status/541868081308790784)

~~~
mikewhy
Why post a contrived meme as an example? Especially one with a sane
solution[1] just below.

[1]: [https://pbs.twimg.com/media/B4W2AmaCAAA-
heo.png](https://pbs.twimg.com/media/B4W2AmaCAAA-heo.png)

~~~
sah2ed
It's an exaggeration, yes, but it does correctly summarize how I felt when I
had to work with node.js code bases before stuff like _promises_ became
mainstream.

~~~
andreime
An exaggeration could be written for any language. A decent programmer would
not write code like the one you showcased as an example for callback hell.

This [https://docs.spring.io/spring/docs/2.5.x/javadoc-
api/org/spr...](https://docs.spring.io/spring/docs/2.5.x/javadoc-
api/org/springframework/aop/framework/AbstractSingletonProxyFactoryBean.html)
is not an exaggeration and I still would not use it to showcase how I felt
when I had to work with Java developers.

------
bit_logic
Many are forgetting the initial reason node.js became popular. Consider the
popular server-side landscape before node.js. It was dominated by Java,
Python, etc. The primary way these ecosystems handle concurrency is OS-level
threading. There was nothing else in the popular languages. Each language had
some niche library that did non-blocking I/O (Twisted for Python, Netty for
Java), but these all had a critical flaw, which is the rest of the ecosystem
didn't support it. Basically every library, almost all existing code used the
threading model. And when you mix threaded libraries with a non-blocking I/O
server, it completely falls apart because the threaded code blocks.

Then came node.js. Look at the ecosystem it came into. JS itself has very
little standard library, and nothing for I/O. It has a highly optimized VM
supported by the resources of Google. It's a language known by many
developers. node.js took this well developed clean slate and built a non-
blocking I/O server on top of it. It offered a concurrency model on the server
side that's completely new to many developers, an alternative to the
traditional threading model everyone knew. Since server-side JS didn't exist
yet, it forced all libraries to be written in this non-blocking way. Every
package in NPM is written to support the core design of node.js which is non-
blocking I/O. And it was exciting to many developers, it was a reason to
rewrite everything in this new way.

~~~
nulagrithom
On that note, coming back from node.js has been a really frustrating
experience for me. I want to scream every time somebody suggests threading as
a solution to non-blocking database calls and HTTP requests. It's made me
painfully aware of how often parallelism is suggested in place concurrency.

C# in particular does a great job at conflating concurrency and parallelism.
Both fall under the same Task namespace and it drives me mad to no end. I've
also seen a couple instances of Execute and ExecuteAsync _both_ blocking
because the underlying driver didn't support async, with no indication other
than my async calls blocking.

It's quite different from the async-by-default sort of thing that's going on
in the node.js world.

~~~
cdelsolar
but why? Go's model is perfect. They're not real threads, it's basically very
similar to Node, except that you can write code without a mess of callbacks
and it's easier to reason about...

~~~
a13n
async await though

~~~
paulddraper
And no shared-memory CPU parallelism.

~~~
a13n
No threading bugs though.

------
s_kilk
> That said, I think Node is not the best system to build a massive server
> web. I would definitely use Go for that. And honestly, that's basically the
> reason why I left Node. It was the realization that: oh, actually, this is
> not the best server side system ever.

Really interesting. I can imagine others would refuse to give up on the thing
they'd worked so hard on. But Dahl has the self-awareness to just step back
and say "huh, guess this isn't so great after all".

Huge props to the guy.

~~~
jgrahamc
Agree. It takes a lot to walk away from something and say you were wrong.

But, sadly, part of the saga of nodejs could have been avoided by people
looking at the history of computing. If you look at Go's concurrency model it
was entirely mapped out in the 1970s in CSP. There wasn't really any need for
go the 'single thread, non-blocking' route that nodejs took and evangelize it
as nirvana. There was a ton of distributed systems work done in the 1970s.
Just look at Lamport's clocks paper.

I'm not criticizing Dahl directly here, more a sense that people ignore
history in a field that has only had history for far less than a century.

~~~
amasad
The fact that Node survives as a platform for Fullstack JavaScript shows how
serendipity works in invention: he set out to build a high performance network
server and ended up building one of the largest platforms for building web
applications.

The reasons it ended being very big are clear in hindsight but are hard to
predict:

\- Sharing code between server and client

\- Server side rendering

\- Lower barrier to entry than other platforms (learn once write anywhere)

\- Well-designed package system

It was also helped by the fact that folks decided to write JS tooling in JS,
be it minifier, transpilers, or test runners, without nodejs we would've built
then on top of something else.

~~~
stephenr
I wouldn't call npm "well designed".

It's progres bar was the bottleneck causing slow downs.

The size of node_modules directories is basically a meme now.

~~~
amasad
It's the least shitty of all other designs. It picked the best design
decisions from existing package managers. By default it is project local (as
opposed to global like what rubygems, pip, and others are), it's recursive and
largely reproducible (fully reproducible using lock files). The resolution
algorithm is also deterministic and simple, you can implement it in an hour.

Although other languages are catching up they have to do it as add-ons or
third party tools -- for example python pipenv
[https://www.kennethreitz.org/essays/announcing-
pipenv](https://www.kennethreitz.org/essays/announcing-pipenv)

------
afshinmeh
Ryan is so smart. I like the way he thinks. This blog post [1] is still one of
favorites.

[1]: [http://tinyclouds.org/rant.html](http://tinyclouds.org/rant.html)

~~~
mrec
In a similar vein, see "We Who Value Simplicity Have Built Incomprehensible
Machines":

[http://prog21.dadgum.com/139.html](http://prog21.dadgum.com/139.html)

------
Abishek_Muthian
"And then, you know, maybe little servers to... maybe little development
servers, and here and there, maybe some real servers serving live traffic.
Node can be useful, or it can be the right choice for it. But if you're
building a massively distributed DNS server, I would not choose Node." -Ryan

------
dsacco
I find it interesting that he ended up at Google Brain working on deep
learning research after writing Node. There seems to be a trend in the
industry of taking people who are exceptional in one area and putting them on
AI problems (e.g. Chris Lattner). I wonder how effective that cross
pollination is.

~~~
technofire
He has two math degrees from decent (R1, AAU) schools, so I wouldn't say it's
a huge leap; he probably has better training for modern AI than most people
working in the area with CS degrees.

------
amoorthy
Pretty cool how he took a very unusual career/personal route to become such an
important figure in the programming world. Good reminder for a parent like me
that getting your kids into a "top" school isn't a must to succeed.

~~~
vqc
I mean, he went to two "Top 50" schools studying math. Do you mean he didn't
go to an Ivy League or a school traditionally known as a top CS school?

~~~
amoorthy
Yes correct (and not saying that top schools are really top; good article on
selection bias a few weeks ago on this).

I was primarily pointing at the fact that he started in Community College,
which is sometimes looked down upon.

------
georgecalm
I was always curious why Ryan Dahl left the community. He finally answered
that question; not the way I thought he would though:

"I think Node is not the best system to build a massive server web. I would
definitely use Go for that. And honestly, that's basically the reason why I
left Node."

Go may be an excellent choice for massive non-web servers, I don't have enough
experience in it to say. For the product I work on, though, Node.js is the way
to go. It's the best framework that allows us to use the same exact code on
the server and on the client to create a fast progressive site.

------
qaq
Given free choice I'd use Elixir, but working with Node and Python at work
have to say that for async web services to my surprise I prefer Node more and
I def. like Yarn more as package manager.

------
rmrfrmrf
I can't help but feel like Google gently encouraged him to promote Go over
Node. Or that being told by coworkers (goworkers?) for years-on-end that Go is
better than Node has had an effect. Especially considering he talks about
green threads like he's not quite sure what he's talking about. At any rate,
there are many innovations where the creators weren't fully aware of their
impact and eventually come to hate their own designs, so no love lost there.

~~~
tgtweak
Node lets you do single thread non blocking.

Go lets you do multi threaded non blocking.

That's a pretty substantial delta in the server world where you can easily
have 24+ cores available.

Memory and cpu footprint of go is also drastically lower to node as it is a
compiled language. That makes a difference when you are running in a cloud
paying per GB of memory and per core.

I say this while actively developing in node.

~~~
rmrfrmrf
The thing is, in real-world scenarios with databases, http, and regexs, golang
and node.js have very similar performance with node beating golang in some
cases.

Furthermore, not everyone is developing cloud-based apps and trying to
optimize for metered cost structures.

If you need raw, parallelized, computational power, then I will without-a-
doubt agree that Go is a better choice. However, to say that one would always
use Go for any server (even webapps? come on...) truly doesn't make any sense.

The only other benefit-of-the-doubt explanation I could come up with for such
an explanation is that it seems like the creator has spent more time in
academia than in "the real world" of development, and perhaps the persuit of
"the ultimate async system" is more important to him than practicality.

------
dfabulich
It seems weird to me to have an interview with Ryan Dahl today, August 31st,
without talking about the political struggle the Node Foundation has been
going through over the past week. [http://www.zdnet.com/article/after-
governance-breakdown-node...](http://www.zdnet.com/article/after-governance-
breakdown-node-js-leaders-fight-for-its-survival/)

~~~
bdcravens
If you think back to when he quit node, it seemed to when it was due to not
being interested in anything not involving the technology.

~~~
z3t4
You wont get much done if your focus is on politics and fashion.

------
tambourine_man
There's an audio version for the lazy like me (it's a podcast)

[https://api.soundcloud.com/tracks/340298200/download?client_...](https://api.soundcloud.com/tracks/340298200/download?client_id=cUa40O3Jg3Emvp6Tv4U6ymYYO50NUGpJ)

------
chj
A lot of people talk about callback hell. For me, the biggest issue of
Callbacks is that you don't have a single point to catch thrown errors. We
need finer control than a global catch-all handler.

------
quocble
Ryan Dahl's advice for building high performance web server: Use Go.

~~~
romanovcode
nginx is faster then caddy

~~~
tgtweak
And better all around.

------
killjoywashere
But, but what were the colorization projects? What was the domain?

~~~
limsup
[http://tinyclouds.org/colorize/](http://tinyclouds.org/colorize/)
[http://tinyclouds.org/residency/](http://tinyclouds.org/residency/)

------
explodingcamera
They need to fix their ui on mobile, there's no way to listen to the podcast
and most people listen to podcasts on the go I would imagine.

------
thinbeige
Even if Node is not the perfect server-server environment, you can go quite
far just with knowing JS.

------
robinduckett
In this thread: people who don't use Node.js making wild presumptions about
how it works. Callback hell? Been a few years since that's been a problem.

~~~
vorpalhex
It depends :tm:.

There are still a lot of codebases that are callback heavy. For one, callbacks
are faster than promises in a lot of node versions by a very large margin. In
other code bases, legacy rules still apply.

Promises don't work well for some more complicated logical flows (though
they're pretty damn perfect for the common ones) and async is still pretty
new.

Just because it's not a problem you deal with doesn't mean that it doesn't
still affect plenty of devs.

~~~
robinduckett
You can mitigate callback hell using classes, which are far quicker than
promises and easier for humans to parse than callback chains.

~~~
vorpalhex
Genuinely curious, how? Do you have some example code to post?

------
gaius
_this new paradigm of model view controller_

New... in the 1970s
[https://en.wikipedia.org/wiki/Model–view–controller#History](https://en.wikipedia.org/wiki/Model–view–controller#History)

It boggles the mind how little "web devs" know about the history of the field.
No wonder they keep reinventing the wheel.

~~~
topgunsarg
Maybe he's talking about when it got popular, not when it was literally
invented...but I guess you can just be pedantic and miss the point.

~~~
bdcravens
I was seeing frameworks in ColdFusion referencing MVC back around 2001.

~~~
gaius
Yeah, MVC was well known to the Visual C++ community in the 90s, and it was a
well established concept among commercial (as opposed to academic) programmers
even back then.

------
jgrant27
Software Engineering in 2017 : Personality cults and mostly a lack of
appreciation for the history of CS.

