
The Way of the Gopher: Making the Switch from Node.js to Golang - 0xmohit
https://medium.com/@theflapjack103/the-way-of-the-gopher-6693db15ae1f
======
forgettableuser
I'm disappointed by the overall tone here. It's as if this were some kind of
failure that everybody is piling up on.

\- She made three passes to fix the existing code base.

\- After that didn't work, with the CTO's involvement, this Go solution was
devised.

\- It took only 2 weeks to ramp up with Go and build a replacement that solved
the problem.

\- The number of instances needed dropped from 4 to 2

This sounds like a resounding success.

So congratulations Alexandra and Digg. And thank you for sharing. The article
was well written and I learned some things about how the Node.js event system
works under the hood.

~~~
rgbrgb
I think the disappointment here is that the OP seemed not to understand the
root cause of the problem and implemented a solution in a different language
(load balanced worker pool) that could have also worked in the original
language without a total rewrite. Then the new language is trotted out as the
savior. It sounds like the file fetch workers and request handlers were
running on the same process, so the longer-running workers ended up blocking
requests. The way it's discussed, it seems like pure luck that they stumbled
on a solution that solved this (running the workers on a separate
queue/process).

I totally agree that it's impressive that it only took 2 weeks to build the Go
version, it just seems like it would have taken 2 days to try the worker pool
implementation with Node.js.

~~~
zaphar
But using a process pool is _far_ more heavyweight than threads. There is a
cost there that you pay in hardware. Using Go allowed them to do this with
less hardware cost than Node. That's a win that continuing to use Node would
not have been able to provide.

~~~
ilaksh
Node does actually have threads with modules like webworker-threads.

------
ysolo
I've been coding in Node for a couple of years. I find it interesting. But I'm
aware that the async model of Node inverts the flow of control, turning code
inside out and makes application logic difficult to scrutinize and reason
about. Stack traces in traditional multithreaded languages are easy to
understand, whereas in Node a stack trace is necessarily filled with unrelated
calls - or perhaps no stack at all in the case of next tick deferred callbacks
- due to the single-threaded nature of the model. This necessitates passing
around context via closures (or god forbid global variables!). The "context"
in a threaded language is typically just the stack. With Node's async Promises
and equivalents we can trick ourselves into simulating linear program flow,
but it's a pale imitation of the real thing.

This got me thinking - why did the single-threaded "green" threading
programming model like how the Java JVM was initially implemented all those
years ago fall out of favor? It would be as performant as Node presently is
but without the difficult async logic for the end user programmer. Async
programming and green threads are duals - both use event queues behind the
scenes to drive the scheduling. The complexity of the actual I/O and timer
event callbacks is hidden from the programmer in green threads. The pseudo
"thread" context switches would only be done upon blocking I/O. No need for
actual mutexes as the program is still actually single threaded. Stack traces
of green threads within a program would be easy to decipher. Computation on a
green thread would also block all I/O as it would in Node, but we'd know that
going in - no different from Node in that regard. Just the programming model
would be simpler.

~~~
tankenmate
Node's javascript engine v8 can only run in one thread, if you want to access
it from a different thread you need to take a lock (for example in Chrome /
Chromium you need to take an isolate lock; each page runs it's own isolated v8
engine (read only code pages are shared, all other javascript pages (data plus
read/write code pages) are private); in this sense it is very similar to the
GIL in Python. So although node/v8 allows for concurrency it does not allow
for multi-threaded parallel execution.[0]

Because of the way tasks are scheduled in node.js you can have head of line
blocking; i.e. a task that is scheduled to run first can block all the
subsequent tasks, the issue that the OP suffered from. You can sometimes
reduce this by calling process.nextTick(), but you need to remember that it is
only IO is non-blocking in node.js.[1]

[0] [https://stackoverflow.com/questions/14409609/does-
the-v8-jav...](https://stackoverflow.com/questions/14409609/does-
the-v8-javascript-engine-have-a-gil)

[1] [http://greenash.net.au/thoughts/2012/11/nodejs-itself-is-
blo...](http://greenash.net.au/thoughts/2012/11/nodejs-itself-is-blocking-
only-its-io-is-non-blocking/)

~~~
ysolo
Since Google changed the V8 API to mandate use of Isolate pointers passed in
to V8 functions as a parameter rather than implicitly using thread-local data
for engine storage it is now possible to run separate V8 instances within the
same thread (one at a time), or from different threads as they do not share
state. One of the many issues Node has to contend with in using the V8 engine
is a perpetually changing API surface.

------
firasd
The crux of the difference seems to be, in the Node.js service:

 _" when any request timeouts happened, the event and its associated callback
was put on an already overloaded message queue. While the timeout event might
occur at 1 second, the callback wasn’t getting processed until all other
messages currently on the queue, and their corresponding callback code, were
finished executing (potentially seconds later)."_

And in Go,

 _" the service has a worker pool and a delegator which passes off incoming
jobs to idle workers. Each worker runs on it’s own goroutine, and returns to
the pool once the job is done."_

~~~
awjr
Node.js has cluster to solve this issue
[https://nodejs.org/api/cluster.html#cluster_cluster](https://nodejs.org/api/cluster.html#cluster_cluster)

~~~
Sanddancer
Which brings us back to 2001 and Apache's prefork model. Threads are pretty
battle tested at this point, and allow for different reasoning about the
concurrency of a problem. The problem with node is that it pretty much only
allows for one type of scaling if you start running into performance problems.
Go gives you a much bigger toolbox to work with if, much like the post's
author discovered, your problem set does not match up with the answers node
provides.

~~~
qaq
All is well until you have to deal with SEO for SPAs :) and then there is
little choice other then node. [https://github.com/audreyt/node-webworker-
threads](https://github.com/audreyt/node-webworker-threads) spawn real threads
with web workers compatible API.

------
maleck13
Just wanted to add my thanks to the author for writing this enjoyable piece.
After working with node for 4 yrs & golang for 1yr, I think they both have
their place. If forced to choose one, I would currently pick golang due to its
excellent tooling, strict typing, excellent std lib and simple deployment.

------
kjksf
For other stories of X => Go rewrites, I started keeping a list at
[https://quicknotes.io/n/vzb7-rewritten-tales-of-
rewriting-s](https://quicknotes.io/n/vzb7-rewritten-tales-of-rewriting-s)

~~~
feylikurds
That list can easily be made longer. Off of the top of my head I know that
there are many more projects which have switch languages to Go. What they say
most of the time is (from the article):

Since our transition to Golang we haven’t looked back. ... we’ve begun the
process of modularizing our code base and spinning up microservices to handle
specific roles in our system.

~~~
kkowalczyk
I agree, it's a work in progress.

If you have more links, I'll be happy to add them.

I have a much longer list of companies that use Go
([https://quicknotes.io/n/1XB0-companies-using-
go](https://quicknotes.io/n/1XB0-companies-using-go)) but it's harder to find
descriptions of rewrites than find out who's using Go.

------
BogusIKnow
When you switch from Node to Go, two languages that are totally different in
style, library support, performance, compile/web turnaround, client/server
reusability, I always wonder why you chose Node in this case in the first
place.

~~~
pron
For the exact same reason they chose Go now, and the reason they'll switch to
something else (much better!) in two-three years: because that's the current
fashion.

Nothing says "I'm keeping up with my professional development" than relearning
IO libraries and build tools, and rewriting in-house code, over and over
again, every time in the new langue-du-jour.

~~~
zappo2938
Meanwhile Facebook is enjoying working with a large PHP code base. When asked
they say they can write code that takes time to compile that runs fast in C++
or write code fast in PHP that runs slower. Writing features faster gives them
they edge. They have an engineering team that does nothing but optimize PHP.
For them an optimization is an acre of servers saved. They if something needs
to run blazing fast can write code in assembly language.

What I'm told about Node is the JavaScript is running in a single thread but
all I/O is being passed to native code which is written in C++ and running on
multiple threads. It does use all the CPUs.

------
partycoder
Blocking the event loop in node can cause severe issues. If you are juggling
with 5 sticks but one stays in your hand for 1 second, your juggling will
certainly fail.

Instead of flamegraphs I use nodegrind, which I can use with
KCachegrind/QCachegrind.

------
KirinDave
I think this is great engineering, and it shows that the Node runtime needs to
consider how to actually address the needs of developers rather than smugly
pointing to really frustrating solutions like Cluster that just don't provide
equivalent functionality to what Go (and its predecessors in this style such
as Erlang and Haskell) are providing.

There are actually a whole host of languages that would let you approach the
problem with this technique and get there, so the only thing special about Go
is that it is syntactically simple (although it's conceptually quite hard to
build reusable code because of poor generic support the resulting bad error
handling), so it makes for an easy migration.

We could see similar stories about going from Node.js to Erlang, Nim, Rust,
F#, C#, Haskell, or even the newer Python & Ocaml runtimes. So it's not like
Go is special here. But it's timing is special, because it's a harbinger of
the industry's slow and reluctant admission that pretending everything is
single threaded is okay. Even I/O multiplexing with coroutines simply cannot
keep pace with even casual demands on modern infrastructure.

------
nissimk
Originally, the web server was intended to be a very simple program with most
of the complications in the browser. Most web app backbends are similar to
this where all they do is query a database, cache or other web servers. This
type of thing is perfect for node. If you are doing more complex stuff in your
backend then go may be more appropriate.

------
cryptica
I have read so many articles which incentivize switching from Node.js to Go
and every single one of them (this one included) blabber on about the Node.js
event loop becoming congested - This is completely misguided - It only shows
that the engineer didn't understand the problem.

A single Node.js instance runs its business logic in a single process. A
single Go instance can run its business logic in multiple processes. Of course
if you're comparing a single instance of Go running on 4 CPU cores, it's going
to be faster than a single instance of Node.js running on 1 CPU core.

But the thing the author fails to mention is that there are a lot of Node.js
modules which help you easily scale across multiple CPU cores. E.g. PM2
[https://www.npmjs.com/package/pm2](https://www.npmjs.com/package/pm2)

Had the author of the article actually understood the real cause of the
problem, he would have saved himself and his company a lot of time.

Nodejs is still outpacing Golang according to Google Trends and yet I have
never seen a single article about switching from X language to Node.js. My
guess is that the Go community probably wouldn't exist if it wasn't for this
constant form of aggressive propaganda/marketing. Node.js on the other hand
doesn't need any marketing/propaganda; it just sells itself.

If you have a scalability problem with Node.js (or any language for that
matter) and you can't find a simple solution that doesn't require rewriting
your whole app, you're probably not qualified to manage this project at scale
- Because scaling across multiple CPU cores is kids' play compared to scaling
across multiple machines. Heck, my grandma could scale this thing across
multiple CPU cores.

Personally I think Node.js is great because it encourages you to think about
these problems sooner rather than later.

~~~
spriggan3
> Node.js on the other hand doesn't need any marketing/propaganda; it just
> sells itself.

If Node.js used language X,Y,Z instead of Javascript, it would certainly not
have been the success it was. "Javascript on the server" was the
marketing/propaganda . Nodejs came at the right moment, Js was exploding,
websockets were exploding, coffeescript and co were exploding in popularity.

What Go does better than Nodejs is a better standard library, true
concurrency, static typing. With Go it doesn't matter if an operation is
blocking or non blocking, that fact can totally be abstracted from the client
code. Writing callbacks is tedious, promises are tedious, co routines with
yield need plumbing and async isn't in the spec.

> you're probably not qualified to manage this project at scale - Because
> scaling across multiple CPU cores is kids' play compared to scaling across
> multiple machines

With Node.js any heavy computation needs to be launched in a separate process,
which is not the case with Go. sure you can have one/2/4/8 process dedicated
to heavy computation, but it will still be the bottle neck. In the end you'll
have to use a different language in order to solve the single threaded
problem.

Finally there is no mystery as to why some businesses are moving from Node.js
to Go: They didn't need anything Node.js provides at first place. They didn't
need to use javascript, they didn't need to write "universal" applications
with the same logic running in the client and in the server, they didn't need
single threaded async programming.

Go is not a silver-bullet, but I was a bit sick of having to write a callback
every time I wanted to make an http request, just to prototype something.

~~~
Lukasa
I agree with almost all of your post except this:

> With Go it doesn't matter if an operation is blocking or non blocking, that
> fact can totally be abstracted from the client code.

No, it can't, and pretending that it can is misleading in a way that allows
large teams of developers to cause themselves real problems.

The only sense in which this is true is that you can write a Go function with
a simple signature like this:

    
    
        func DoSomeStuff (error) {}
    

And the caller has no idea whether this involves concurrency under the hood
(e.g. the function can spawn its own goroutines and channels as it sees fit).

 _But_ , make no bones about it: this function blocks until it completes. This
means that while this function may do concurrent work it is absolutely a
blocking function. This is fine: sometimes blocking functions are good. But
you cannot write a non-blocking Go function in the same way.

To make a function non-blocking you can return a channel out of it for the
return value to appear on, like this:

    
    
        func DoSomeStuff (chan error) {}
    

In this model the caller really doesn't have to care whether the function is
synchronous or not (though the fact that it was written this way strongly
suggests that it is going to return asynchronously, or at least that the
developer believes it will have to in the future).

Except...that return value just there? _That 's a Future_. It's a terrible,
half-implemented version of a Future, but that's exactly what it is. It's a
promise to return some kind of result at some point when the underlying
process has returned.

And if you don't want to block your current goroutine, you cannot block on
that channel receive either. That means that you need a callback. There are
two patterns for doing that: you could have some kind of central loop that
selects over all channels like this and calls the callback functions (boy that
looks a lot like Node's event loop, doesn't it!), or you can manually spawn
your own callback functions in their own goroutines. Either way, you have
callbacks and futures here: you're just building them yourself and calling
them something different.

There are lots of good reasons to switch to Go: it's a language that makes
lots of developers remarkably productive, it has an ingrained philosophy of
building concurrent programs, it's pretty damn fast, and it runs on all kinds
of awesome platforms. But claiming that Go has learned something magic and new
about how to write concurrent software in such a way that you don't have to
care whether your code is async or not is just not true: you always have to
care.

~~~
agentS
> But you cannot write a non-blocking Go function in the same way.

The caller can make function "non-blocking" by wrapping the call in a
goroutine themselves. (There's some subtle differences, but they are mostly
irrelevant here). For this reason, I'd say there is (almost) no reason to
introduce asynchrony in your API in the way you suggest. The rest of your post
built on this example seems shaky to me, since it seems built on an example
API that doesn't need to exist.

I'd say that "you don't have to care whether your code is async or not" is a
overstating the case. I would append the qualifier "unless you're introducing
concurrency". Considering that almost no low-level APIs are asynchronous, this
usually happens rarely (or happens in low-level code like the HTTP server).
Examples that have come up for me: making N parallel RPCs, writing a TCP
server. In those situations, you care about async vs not.

In event-loop based systems, it seems like async is in my face all the time,
even when doing things that are entirely sequential.

~~~
Lukasa
> The caller can make function "non-blocking" by wrapping the call in a
> goroutine themselves.

Sure, but if they want the return value then either they need to construct the
Future-y wrapper I just described or they need to assemble it together in a
collection of other function calls wrapped inside a function that itself is
either Future-y or uses a long-lived channel to communicate results.

It is not novel to build up a non-blocking system from purely blocking method
invocations. We've been doing that for years: it's called threading. Doing
things this way has many advantages when written with appropriate diligence,
and I'm not pretending otherwise. _However_ , if you actually care about
communicating between these arbitrary threads of execution than you either
need Futures or queues (both of which are essentially just channels in Go),
and at this point you've got the exact same problems as you get in NodeJS or
any other asynchronous programming environment.

> The rest of your post built on this example seems shaky to me, since it
> seems built on an example API that doesn't need to exist.

I don't think that's fair: as I mentioned above, the fact that you as library
author would not write the Future-y extension doesn't mean that the Future-y
extension isn't built: you just force your caller to build it. That's fine,
it's a perfectly good architectural decision (probably you should't be making
those decisions for your user), but it doesn't remove the problem.

> I'd say that "you don't have to care whether your code is async or not" is a
> overstating the case. I would append the qualifier "unless you're
> introducing concurrency".

Sure. The thing that matters here is that Node is _always_ introducing
concurrency, because Node is concurrent. This is why all Node programs have to
care about concurrency: they are all concurrent because their system is
concurrent.

This is desperately inconvenient for many one-off programs, which is why I
personally don't use Node for anything like that: I'd much rather use Python
or Rust or Go. But that was never my argument. My argument was about OP's
assertion that "with Go it doesn't matter if an operation is blocking or non
blocking, that fact can totally be abstracted from the client code. Writing
callbacks is tedious, promises are tedious, co routines with yield need
plumbing and async isn't in the spec."

The first sentence is dangerously misleading (while technically true, any
system that does that is usable only in that one context), and the second one
misses the point, which is that those things get effectively built _anyway_ in
any moderate-scale concurrent system in Go.

But my biggest point is this: Go isn't magic in regard to concurrency, and
there is a weird amount of magical thinking around Go. Go is a very good
language with a lot to like, and I like it quite a lot. But when boiled down
to it, Go's concurrency model is threads with a couple of really useful
primitives. And that's great, and it works really well. But it's not new or
novel.

The sentence "with Go it doesn't matter if an operation is blocking or non
blocking, that fact can totally be abstracted from the client code" is equally
true if you replace "Go" with "C", or "Python", or "Java", or any language
with a threaded concurrency model. There's no magic here. It's the same
building blocks everyone else is using.

~~~
vishbar
Go isn't just a threaded concurrency model, it uses an M:N greenthreads
pattern. Also, when you say that Go I/O operations are blocking, it is true
that they'll logically block a goroutine. However, under the hood, it uses the
same libuv-style async IO (or IOCP on Windows) that Node does. An operating
system thread doesn't get blocked; the goroutine is "shelved" and woken up
again when the I/O is complete. It accomplishes the same kind of thing as
Nodejs does, it just abstracts the async nature of the IO away from the
programmer. I have to say I like it: procedural execution is easier to reason
about.

~~~
Lukasa
Honestly, I think the distinction between M:N threading and straight OS
threading is pretty minor. It grants some advantages to the language runtime:
it can control the stack size, for example. But in terms of how it affects the
development style and what kinds of bugs it encourages/discourages I don't
think it dramatically differs from the OS threading model.

~~~
sagichmal
It is _categorically_ different. OS threads are orders of magnitude more
expensive, which makes them a nonstarter for most problems that are a good fit
for lightweight conceptual concurrency.

~~~
Lukasa
It is not categorically different.

As I said above, green threading has advantages over OS threading, but they
behave exactly the same in terms of design patterns and potential bugs.

This is what I was getting at when I said "not that different": compared to
the difference between event-loop concurrency and threaded concurrency, M:N
green threading is basically just a subcategory of threading.

~~~
sagichmal
No. Green threads enable entirely new classes of design patterns that are
categorically infeasible with OS threads.

------
planetjones
_> for each request, Octo typically fetches somewhere between 10–100 keys from
S3_

I'm really curious as to why this is neccessary - what are the 10-100 keys you
are fetching per request ? Just reading the high level article it sounds like
there is some issue with what this service is doing, irrespective of the
language it's doing it in.

~~~
meonkeys
Another idea: stress-test a Node.js process then hard-limit the number of
requests each Node.js process handles based on that stress test. Add more
Node.js processes (one per core on the same or other servers) until there is
sufficient capacity to handle all requests. This assumes the Node.js code is
already distributable across cores and servers, which seems to be the case
based on the article.

Might have been a day-long workaround rather than a weeks-long rewrite.

~~~
zaphar
Part of the goal was fixing this without having to just throw hardware at it.
Every Node process has a non-trivial amount of overhead that requires
additional hardware resources. Overhead that the Go app doesn't have. They
could have thrown more machines at this but that costs money. The better
solution technology wise was Go. It was better suited to their workload.

------
justicezyx
Another evidence that the language that is narrowly focused and force "more"
correct practices tends to work better in practice.

Or, coders are getting worse at master the more powerful language...

~~~
XorNot
99% of the time what is called "mastering" is actually a whole bunch of stuff
you shouldn't do if you want a maintainable codebase.

Tricks are just that: tricks. No one sets up a business based on how you pull
a rabbit out of hat, nor should they.

~~~
justicezyx
No, mastering a complex and powerful language means use the minimal feature
for the majority of your needs, and knows how and when to use the "dangerous"
part for a small fraction of your needs.

I do not know much of JS, but people write C++ with loads of patterns are
prominent examples of people who do not master the language. That's not trick.
That's real engineering with (often) bitter experience.

------
0xmohit
Some comments prompt adding another link here:
<[https://medium.com/@tjholowaychuk/farewell-node-
js-4ba9e7f3e...](https://medium.com/@tjholowaychuk/farewell-node-
js-4ba9e7f3e52b>)

~~~
sah2ed
[https://medium.com/@tjholowaychuk/farewell-node-
js-4ba9e7f3e...](https://medium.com/@tjholowaychuk/farewell-node-
js-4ba9e7f3e52b)

FTFY

~~~
0xmohit
Thanks. Didn't realize that HN screws up links within angle brackets.

------
takno
On the face of it this sounds like a problem where the optimal solution would
have been to throw more hardware at it while you looked at the underlying
choice of datastore.

------
diegorbaquero
The EC2 medium instance has 2 cores. Were you running 2 node processes in
cluster mode? The only thing I asked myself when reading this. I'm keeping my
vote for node.

~~~
dminor
m3.medium only has 1 core.

------
dalyons
im obviously missing something here, hopefully someone can explain?

I understand the eventloop/nexttick/etc architecture of node, but i dont
understand how in this case he was blocking the loop? Shouldnt all the
operations to S3 be async (and thus non-blocking, even if waiting for a
timeout)? What was the specific part in this scenario that was causing the
loop to stall?

~~~
amalantony06
If I understand correctly, the problem was that they were making several
thousands of requests to S3. While the requests to S3 themselves were
asynchronous, the callbacks for these requests were queued up for
(synchronous) execution in the event loop. Due the large number of callbacks
in the queue already, new callbacks for the incoming requests were queued up
for execution behind the previous callbacks, leading to latencies in serving
up responses.

~~~
dalyons
Ah ok! Perfect, thanks, that's what I was missing. Makes total sense now. In
any lang (even in ruby!) you could have separate thread pools(or EM loops,
whatever) for the s3 requests & web handlers. But because node only has one
EL, and node interprocess comms is awkward, its tricky. Gotcha, cheers.

I wonder if using something like async.eachLimit would have helped; it might
prevent the s3 batches from flooding the loop & give a chance for web reqs to
interleave, but probably at a cost to the median resp time.

------
romanovcode
2010 - Making switch from PHP to Ruby

2013 - Making switch from Ruby to Node

2016 - Making switch from Node to Go

2019 - Making switch from Go to {hype}

~~~
tptacek
"So what I'm saying is, we should all still be using PHP."

~~~
bliti
What is star fighter written in? Curious, no bad intentions.

~~~
tptacek
Go, Ruby, and React.

------
bitwize
I lol'd at the bit about blocking Node's event loop.

If Unix weren't so committed to doing I/O the stupid way around, maybe Node
wouldn't have to provide fragile, ersatz asynchronicity the way that it does?

~~~
fooyc
Care to explain?

~~~
bitwize
Two iron-clad rules of data processing:

* All data has a type

* All I/O is asynchronous and interrupt (event) driven

At its inception Unix doubled down on stupid, first by encouraging data to be
stored and migrated in the form of untyped text streams, which require ad-hoc
parsing and unparsing logic at the endpoints; and secondly by failing to
provide asynchronous I/O primitives to user space. POSIX later provided an AIO
standard, but it was cumbersome to use, bolted on rather than integrated into
the OS, and virtually nobody used it. What you _want_ to be able to do is
submit I/O requests to the kernel, and have the kernel notify you when they
are completed while you wait or do other processing; you could even put the
process to sleep until pending I/O is completed and have it take up _no_ CPU
during this time. Instead what people do is burn CPU cycles in select/poll
loops, which are the I/O equivalent of asking "Are we there yet? Are we there
yet? Are we there yet?" over and over. You can fake asynchronicity by doing a
bit of processing before asking "are we there yet?" again, but you have to
write your processing code in such a manner as to be done piecewise in a loop,
and the lower latency you want the more frequently you have to poll (and the
more CPU you have to burn polling). This is how Node does "asynchronous" I/O;
the VM stops every few instructions to poll for ready FDs, dispatches to
callbacks as necessary, and performs parts of pending large I/O operations
which can be done non-blocking. (That's the other sucky bit: you can't just
call read(2) and write(2) for large chunks of data on an fd that's been opened
O_NONBLOCK and expect it all to work; you have to keep reading or writing in a
loop when the fd is ready, subtracting the number of bytes successfully
written until the entire buffer is processed.)

But if something -- a call into a C library for instance -- stops the VM from
doing the poll and I/O bits of its main loop, all I/O simply... stops. And
your throughput goes into the toilet. Whereas if the runtime had been based on
an OS that natively supports interrupt-driven AIO -- like VMS, Windows, or
AmigaOS -- it would be extremely difficult to stall the I/O pipeline
completely this way. You might be able to stall _further_ I/O calls with a
really long operation, but any pending calls already initiated would complete,
and the runtime would be properly notified of their completion.

And all this stems from the fact that Unix was the Node.js of its day -- an
environment designed to make things easy for casual programmers, not to do
things correctly.

~~~
wtbob
> At its inception Unix doubled down on stupid, first by encouraging data to
> be stored and migrated in the form of untyped text streams, which require
> ad-hoc parsing and unparsing logic at the endpoints

I tend to agree with you here: it'd have been better had Unix offered a typed
abstraction over byte streams, but OTOH its approach really was a Worse is
Better deal. Building a large set of tools which can deal with typed streams
(or even trees, or graphs …) takes a long time, and is error-prone: building a
few simple tools which treat everything as text is quick & easy. Of course, it
does mean that the next four and a half decades are spent dealing with a lack
of types, but it means one can ship quickly.

> Instead what people do is burn CPU cycles in select/poll loops

I thought that select/poll blocked, but it's been a long time since I did
anything low-level like that, so I could be wrong.

> And all this stems from the fact that Unix was the Node.js of its day -- an
> environment designed to make things easy for casual programmers, not to do
> things correctly.

Well, Worse is Better.

~~~
bitwize
> I thought that select/poll blocked, but it's been a long time since I did
> anything low-level like that, so I could be wrong.

Well, yes. That's just the problem. ALL I/O ops in Unix block. Even the ones
on O_NONBLOCK fds, which simply read or write -- blockingly -- until the
underlying kernel buffer is full/empty, then return.

You don't have the option to request I/O, do some processing, and then wait
until the I/O completes. If you want to interleave processing with I/O you
_have_ to buzz in a select loop, doing your I/O and processing in tiny
explicit chunks. Or spawn subprocesses to do I/O, leadibg to _more_ complexity
and confusion.

The facilities in the Windows kernel for I/O are _strictly_ more robust and
flexible than what Unix (eve Linux and BSD with kqueue/epoll) provides. The
designer of the Windows kernel, Dave Cutler, also designed VMS, knew what
real-world I/O requirements were, and famously mocked the way Unix handles
I/O.

------
bliti
Wait until they discover Elixir and Phoenix. It's the power of Go (a bit more
I think) without the ugly syntax.

~~~
akshatpradhan
What sets Elixir apart from Go?

~~~
bliti
IMO, it's simpler to write and easier to read. It's been a bit faster in t he
tests I've run but not by a lot. Probably due to my skills. I just find its
nicer all around. It does not carry over anything from C.

------
kseistrup
For old farts like me gopher is a protocol.

------
illumen
I think the author made a pretty good decision to choose a solution which was
proven to meet the requirements by another author. However Amazon does provide
a tool to batch download multiple things from S3 though. Let's pretend for a
moment those tools don't exist already... Let's also pretend Amazon doesn't
suggest another solution for when there are lots of GET requests [0]. Let's
pretend that using something like nginx which is a very well tuned web proxy
might be way better because of SPDY, SSL connection pooling and such [1].
Let's also pretend that node.js can not use multiple processes with cluster
cluster [2].

Why wouldn't one use python for this job? Since the company already does a lot
of python, and this job is actually easy to do in python (I've done something
similar).

400 processes with python3.5 is way less than the 4gig on one medium instance
(less than 2.5MB each). Just farm the work out to a ProcessPoolExecutor, and
have a timeout on the S3 get requests. That would let you have enough
resources to match the spike of 20000 requests per minute (334 per second).

A lot of an S3 GET request could be the SSL, AWS auth and such. All quite CPU
intensive. So using an async framework that doesn't do SSL+AWS auth async is
obviously not going to work well once the requests go up.

There's even an example in the concurrent.futures of downloading urls [3].

Made with the beautiful python3.5

    
    
      import concurrent.futures
      import requests
      from awsauth import S3Auth
      ACCESS_KEY = 'ACCESSKEYXXXXXXXXXXXX'
      SECRET_KEY = 'AWSSECRETKEYXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
      
      URLS = ['http://www.foxnews.com/', 'http://www.cnn.com/', 'http://europe.wsj.com/',
              'http://www.bbc.co.uk/', 'http://some-made-up-domain.com/']
      
      def load_url(url, timeout):
          return requests.get(url, timeout=timeout).data
          # return requests.get(url, timeout=timeout, auth=S3Auth(ACCESS_KEY, SECRET_KEY)).data
      
      with concurrent.futures.ProcessPoolExecutor(max_workers=400) as executor:
          # Start the load operations and mark each future with its URL
          future_to_url = {executor.submit(load_url, url, 60): url for url in URLS}
          for future in concurrent.futures.as_completed(future_to_url):
              url = future_to_url[future]
              try:
                  data = future.result()
              except Exception as exc:
                  print('%r generated an exception: %s' % (url, exc))
              else:
                  print('%r page is %d bytes' % (url, len(data)))
    

[0] [http://docs.aws.amazon.com/AmazonS3/latest/dev/request-
rate-...](http://docs.aws.amazon.com/AmazonS3/latest/dev/request-rate-perf-
considerations.html#get-workload-considerations)

[1] [https://coderwall.com/p/rlguog/nginx-as-proxy-for-
amazon-s3-...](https://coderwall.com/p/rlguog/nginx-as-proxy-for-
amazon-s3-public-private-files)

[2]
[https://nodejs.org/api/cluster.html#cluster_cluster](https://nodejs.org/api/cluster.html#cluster_cluster)

[3]
[https://docs.python.org/dev/library/concurrent.futures.html](https://docs.python.org/dev/library/concurrent.futures.html)

~~~
mh-
_> However Amazon does provide a tool to batch download multiple things from
S3 though._

out of curiosity, what tool are you referring to here?

------
DannyBee
How does one live without is-positive-integer and isArray?

~~~
blitzprog
The real question is: How does one live without is-thirteen?
[https://github.com/jezen/is-thirteen](https://github.com/jezen/is-thirteen)

~~~
ysolo
Best post here.

------
kelkes
How many instances of node octo where running to handle the load?

~~~
biot

      > And those 4 load-balanced instances we were running Octo
      > on initially? We’re now doing it with 2.

------
erlich
Long live Node.js!

As everyone has mentioned this seems to be a simple case where the rewrite is
a completely different architecture, which could have been done in Node.js.

I'm interested in why this continues to happen.

Seems like Node.js is a victim of its own simplicity. The baked-in non-
blocking concepts of Node.js makes everyone feel like single-threaded is the
only way to go, and think that Node.js is inherently flawed when it comes to
parallelization.

Because Go doesn't give you a simple and relatively scalable solution without
even thinking about it like Node.js does, instead you have to choose what to
parallelize and think about the problem more. With PM2 scaling Node.js over
CPUs becomes incredibly trivial.

Also, I think programmers naturally like rewrites/starting from scratch, and a
new programming language provides an opportunity to rewrite everything (which
the OP from this article and every other "switcher" article seem to do).

~~~
zongitsrinzler
Recently I rewrote a medium sized Websocket app from Node to Go. Hoping for
better performance and having heard a lot of good about Go.

Since the application relies heavily on sharing resources between sockets I
found out the hard way that concurrent programming in Go is not much easier
than in Java. Sure, it has a better syntax and channels but compared to Node
where you never have to think about things like these the complexity is way
higher.

Long live Node.js!

~~~
59nadir
> Since the application relies heavily on sharing resources between sockets...

Which resources?

~~~
zongitsrinzler
Connections are grouped in various ways. A lot of maps, etc. You can't even
use a regular int safely, need to use an atomic for that.

I'm not bashing Go here, the memory usage is about 3x lower (which was the
most important for me). And for a static language it's pretty cool.

~~~
59nadir
Am I understanding you correctly when I assume that you mean you share data
across threads?

------
spullara
Switching from one essentially single threaded language to another wasn't
likely to solve your issues. Instead of Go you could have chosen any JVM
language or even C++ and had similar success.

~~~
sagichmal
Go is not a single threaded language.

~~~
spullara
Obviously. I was just pointing out that switching to Go wasn't the interesting
bit. Maybe everyone misread my comment.

~~~
_ak
If you didn't want your comment to be misread, you should have been more
specific after the "another".

