
From Node.js to Go - kiyanwang
http://blog.scaledrone.com/posts/nodejs-to-go
======
Gonzih
This blog post feels very empty. No code snippets? No actuall before after
benchmarks/graphs? How can you state anything without showing numbers?

~~~
Cyph0n
Agreed. I posted a comparison between Node and Go UDP socket performance a few
months back with some code snippets. My submission was completely ignored. HN
can be weird sometimes, huh.

Edit (and shameless plug): [http://assil.me/2015/11/07/roadomatic-node-vs-
go.html](http://assil.me/2015/11/07/roadomatic-node-vs-go.html)

You might need to read the previous post (and only other one!) to understand
how the GeoJSON geospatial query works.

~~~
andrewstuart2
There was an analysis done recently that showed a few votes very early on (I
think 4 may have been the number quoted) is often enough to bump you to the
front page. Have ten developers on staff, each with an HN account? Some might
see that as a free ticket.

At best it's only a very temporary traffic boost, though, unless the content
is legitimately interesting enough to warrant return visits.

~~~
throwanem
> Have ten developers on staff, each with an HN account?

I think that's what HN calls a "voting ring" and attempts to detect and
counteract.

~~~
adrusi
I don't think it can be detected the first time its attempted with any
particular group of users.

------
lossolo
I use Go and Node in production. Node is used for cold paths and Go for one of
hot paths (where GC latency is still not a problem). Go for sure is more
readable than Node callback hell and i really like more writing synchronous
like code inside gorutines than do everything around callbacks/promises.
Static typing and compiler is also big + for Go.

~~~
SonicSoul
correct me if I'm wrong but in Node you could use something like a Generator +
Promise combination to achieve synchronous looking code with no callbacks

[https://eladnava.com/write-synchronous-node-js-code-with-
es6...](https://eladnava.com/write-synchronous-node-js-code-with-
es6-generators/)

~~~
adamtulinius
In practice a lot of libraries don't support promises, so in my experience
you'll just end up with a codebase with callbacks and promises mixed. Not very
pretty.

~~~
bilalq
Any library that accepts a standard node callback can be easily promisified.

See:

[http://bluebirdjs.com/docs/api/promise.promisify.html](http://bluebirdjs.com/docs/api/promise.promisify.html)

[http://bluebirdjs.com/docs/api/promise.promisifyall.html](http://bluebirdjs.com/docs/api/promise.promisifyall.html)

~~~
spriggan3
Now compare that kind of patching with writing Go code where non blocking I/O
doesn't require any special syntax. I'm not fan of Go by a long shot, but it
is undeniable the blue/red function problem is a pain [1] when dealing with
JS.

[1] : [http://journal.stuffwithstuff.com/2015/02/01/what-color-
is-y...](http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-
function/)

~~~
jerf
I love that blog post, because then this makes sense: Promises are basically
an inner-platform [1] that replicate all the control flow constructs of the
given language, except as a different "color". Go simply _is_ that inner
platform. Go does not use promises because all of its code is already running
in the promise color. You don't need anything special to "deal with" a for
loop that may end up waiting for something to be read from a socket, because
the for loop is already a "promise", and so forth. You can rearrange your code
as much as you like, and you are not constantly juggling with stuff the
compiler ought to deal with.

Manual promise transpilation is better than manual callback transpilation, but
it's still manual transpilation. Why are you working so hard?

Sometimes I get mistaken as an advocate for Go. I'm not really. What I _am_ an
advocate for is using languages where you don't need an inner-platform of
promises because the language is already that color. Go just happens to be the
most familiar Algol-esque of the bunch right now. Don't want to learn Go? Go
learn Erlang or Elixir. Or maybe if you're feeling saucy, Haskell. Or some of
the other choices popping up lately. If you're writing network servers you
will not be able to understood how you ever functioned without this.

[1]: [https://en.wikipedia.org/wiki/Inner-
platform_effect](https://en.wikipedia.org/wiki/Inner-platform_effect)

------
fighting
Go seems to be still a bit of a black box in terms of performance/memory
profiling.

On one recent project, the go memory profiler was showing about 35MB allocated
while top showed memory usage at 500MB to 1GB. In another instance the
performance profiler would show microseconds elapsed while the user
experienced tens of minutes with no response from the program.

I really think they should consider including a function level profiler
instead of just a sampler, as well as provide more insight/control over all
the goroutines that are generated.

Just saying golang is better than node.js may not be saying all that much.

~~~
lossolo
Can you provide more details? Because i didn't encounter such problems. Was
this 500 MB in top was RSS ? What you describe should not happen, did you
report a bug?

~~~
fighting
They were more like things the go runtime was doing but which I had no clue or
information about, rather than bugs. The top info was indeed rss.

~~~
lossolo
For me it seems like you probably leaked gorutines/not closed os resources or
you encapsulated some data structures in themselves which then can't be
garbage collected. It really looks like memory leak caused by app logic. But
as i said you should report this if you are certain that is Go bug.

------
sspiff
TL;DR: Using a lower-level, compiled programming language results in lower
memory consumption and faster software. Shocker.

Besides the lack of data to illustrate their point, their point is well know
and has been well documented for decades. The reason people use these higher
level, higher overhead languages is generally time-to-market, maintainability
and development cost.

I don't so much care about the performance difference data, I can measure that
myself easily and cheaply.

Measuring frustration, development time and bug-hunting time is something that
would be expensive to measure (you have to build a real world product in both
languages with users and maintain it, which they seem to have done). Yet this
part is glossed over and never quantified in the article.

~~~
gbersac
> their point is well know and has been well documented for decades.

Node.js and go are 7 years old, so no it has not been documented for decades
:p But your comment is still right.

~~~
untothebreach
the point that has been known for decades is not, "Switching from Node.js to
Go gives you better performance," it is "switching from a high-level language
to a low-level language gives you better performance." This point _has_ been
known for decades, but new generations of programmers keep re-discovering it.
(note: I am not critizing new programmers, I am one of them. in fact, having
recently switched from python to rust, you could say I am one of the many re-
discovering the performance joys of lower-level languages)

------
mccolin
I love the anecdotal account -- especially since I am researching a similar
change -- but would truly appreciate more numbers for transition stories like
this. Memory usage before and after. Throughput, etc.

Also what was the cost in time and man hours of conversion?

~~~
lossolo
It really depends on how much experience someone has with Go. It will be more
hours for sure than Node, probably more code also. You need to watch out on
couple of things also that you don't need in Node. For example default passing
by value to functions so you need to use pointers for large structs if you
want performance, you need to know that strings are immutable which means that
if you do something like "some" \+ "ting" you are basically creating new
string and both strings are copied to that string, if you have a lot of that
kind of operations (like really a lot) then use bytes.buffer etc. A lot of
things that are not so obvious for beginner in Go. But you got great tools at
your disposal by default like pprof which is great profiler, data race checker
etc. Standard tooling is really great.

~~~
adamtulinius
Strings in javascript/nodejs are also immutable.

------
qaq
How did this make fp? 0 factual content. X is better then Y

~~~
oblio
It made first page by having popular values for X and Y.

------
CraigJPerry
Why 89 upvotes, there's zero technical details?

~~~
d23
And considering the recent leftpad disaster, it makes this "point" a bit tone
deaf:

> Node.js has been around the block longer it boasts more than double the
> modules

~~~
nkozyra
Well the leftpad disaster wasn't really a symptom of the # of modules. I think
it's probably fair (and unsurprising) to say there are a lot more
"significant" modules in the Node world than Go, even if we discount the
10-line basic function modules.

------
grigio
Is this really the best compromise?

What about NodeJS + Rust (for the heavy tasks) via Neon?
[https://github.com/rustbridge/neon](https://github.com/rustbridge/neon)

------
lpinca
I wonder what was the WebSocket module used in Node. There is this new module
[https://github.com/alexhultman/uWebSockets](https://github.com/alexhultman/uWebSockets)
which is very promising.

------
KAdot
> Node.js has been around the block longer it boasts more than double the
> modules

Yes, Node.js has more third-party packages available, but the quality of Go's
packages is usually higher. Also, you can do a lot with built-in packages in
Go compared to Node.js.

------
BuckRogers
It's hard to make a great analysis because there's no hard data on the memory
used, the code creating that memory use or their infrastructure. But if I may,
due to them being less than forthcoming, I'd like to speculate. Anger many,
please a few.

If they needed something better than Node they may simply be doing it wrong
and I'd lean that way first before subscribing to Go being the solution. Which
has me thinking it's the most convenient band-aid nearby.

There are alternatives to the language/platform hopping. My case would be a
decent example. Most of my stuff is in Python2. I have a newer Django project
that I'm vetting CPython3 with, but will remain writing long running CPU
intensive services in Python2 on PyPy. I see CPython + PyPy as a pretty
reasonable approach and have been able to scale quite well for most situations
which don't involve battling millions of concurrent users and petabytes of
real-time data analysis. There's also the advantage of a single language
backend and developer productivity. Node.js callbacks never matched gevent or
Python3's async I/O and Go is adding yet another language to the stack where
PyPy (or Node in their case) would likely handle it. Difficult to say without
looking at how they've implemented everything but I have doubts whatever
they're doing couldn't be done with resourceful coding. Most likely, these are
people who want an excuse to try something else and they're just excited.

Given the self-professed extremity of their case, Node, Python, Go, et al
would not resolve something so seriously memory-bound well enough that it
wouldn't be worth calling out to a non-GC'd library. I would steer away from
GC-anything and drive a stake through the heart. Writing a small limited-scope
Rust extension to do what they needed and called that from Node.

------
josep2
Beating the living shit out of this dead horse.

------
andrewstuart2
> Another thing I found bothersome were the generated GoDoc pages. They were
> ordinarily not as easy to understand as hand written Node.js GitHub readme
> pages and came with very few, if any, examples.

I'll definitely agree here. It's not obvious right away how to write examples
that get included in the generated documentation, and definitely unintuitive
that it's a completely different process than the comment-based approach.

------
Twirrim
a.k.a. from one hotness to the next?

------
wooptoo
This page is worded like an advertisement.

------
lopatin
If anyone is planning on building a semi-ambitious web system soon, you should
evaluate Go. It's kind of scary how productive it allows me to be now. All of
the following benefits, of course, may not apply to your project but I wanted
to share my experience regardless.

I started my project as a collection of (micro?) services, all talking to a
distributed kv store, containers all orchestrated, etc ... That's just how we
build things nowadays right? Well one day I said screw it and rewrote the
whole thing as a Go monolith. Including the DB layer, which now runs on
BoltDB, an embedded k/v store. So now I have a web analytics system, reverse
proxy, rest api, pixel api, license key server, reporting engine, emailing
service, and a custom stripped down time series database (uses Bolt as a
storage engine) complete with a custom SQL-like language (stripped down
version of InfluxQL). It all exists as a single binary, which magically works
on Windows, OS X, and linux, with zero run-time dependencies. This has allowed
me to provide my customers with a self-hosted version of my software if they
don't want to use my cloud service. My cloud service is just a differently
configured version of the self hosted binary. Compilation takes just a few
seconds.

Regarding performance. A single node handles thousands of requests per second.
It's possible for me to optimize that further (I suspect by a lot), but I
don't need to. And up to 1TB single node storage. Again, there have been
larger BoltDB deployments but I simply don't need to push it very far since I
can easily run multiple nodes when the time comes, each handling some subset
of user accounts.

Another delightful thing is my dependency situation. I have 8 library
dependencies in total, and have never even had to think about which version
I'm using past the initial download. They're mostly lean and mean and stay out
of the way. Things like CORS middleware, JWT library, request mux, Bolt, and a
few others. And they all work together since they all conform to Go's built-in
defined interfaces. What I appreciate about the library ecosystem is that most
of the libraries you will use are _done_. Go has no dependency versioning,
semver, etc... It sounds concerning at first but I was pleasantly surprised
about how much of a non-issue that is in Go. I suspect that a project of
similar scale would have resulted in many lost hours in Maven, Gemfile, or
package.json.

Also, the concurrency makes me happy. I've never been one to multi-thread my
programs. But Go makes it easy to build concurrent pipelines and services
without needing a team of engineers to review it for correctness.

Anyways I'm rambling, I guess I should list some of the frustrations and
trade-offs I made. For one, yes you will miss generics at first. It seems like
a weird thing to not have in the language. It kind of takes getting used to,
but you will figure out how to write the same functionality in terms of
interfaces. It's certainly unique, a bit more typing, but in the end, things
still get done. But, yes, you will have to shift your mind to a different
programming style. Which some find annoying. On the plus side, at least for
me, Go is one of those languages that has changed the way that I think about
programming.

Also, not really speaking to the Go language, but if you're considering Bolt
as I mentioned, don't expect it to handle replication or distributed fanciness
for you. I'd have to say the biggest tradeoff I made is switching to a
scheduled data file backup system in the place of an actual database service.
With naive scheduled backups, you might lose a few minutes of data in the case
of a total node failure. This can be improved on my end, but that starts to
get pretty involved.

... I don't know why I just wrote an essay. Hopefully this encourages someone
try out Go for their next project.

------
fithisux
The right decision. If only I could do the same with java and Python.

------
kiril-me
Yep, leak of numbers exists in this blog post. Node is certainly not for
websocket communication, until it's simple chats or message count is not so
huge.

