
Go vs. Python for a simple web server - amarsahinovic
http://blog.kowalczyk.info/article/4dep/Go-vs-Python-for-a-simple-web-server.html
======
noobface
Hardware/Network guy here. I realize this thread is probably only tangentially
appropriate for this discussion, but I digress, I haven't seen anything too
close in my daily visits here.

There's a ton to be said for per-thread connections. This author certainly
recognizes the benefit of truly parallel communications, however, there are
some networking considerations that might be of interest. Especially regarding
the tendency to simply "open more sockets" to increase overall speed.

Most software guys I know tend to ignore or just be ignorant of how their
choices affect the underlying hardware. I realize this is a brash
generalization; there are some full-spectrum people out there, and I
appreciate you.

That said, opening new sockets is not without diminishing returns. The sheer
volume of port numbers for those sockets can lead to memory exhaustion in
consumer network devices, especially when something else is running. In this
respect, bit torrent is atypically aggressive.

Basic consumer routers have hard time keeping up with their internal processes
when flooded with new TCP sessions for hundreds/thousands of ports. NAT tends
to run slowly as memory exhaustion takes hold and many times the only option
is to shorten the timeout window on connections. (Admittedly, the default
settings for NAT table timeouts is ridiculous; sometimes up to 300 seconds).
With long default timeouts, it's no wonder these simple MIPS systems are
overwhelmed.

Count how many tabs you have open now, and consider that they all may be using
the same number of ports the services you're building do. Not to mention the
ad networks...etc

All I'm really trying to get across is, more ports, especially on consumer
devices, does have diminishing returns despite its apparent immediate benefit.

Yes, newer routers do have better hardware. Yes, your AWS machine can handle
tons of connections. But your end users may be like you, they may not know how
much memory their local router has or particularly care.

I'm not advocating anything other than moderation the number of sockets you
employ. I just wanted to get this out there.

~~~
jlouis
NAT (Or more correctly PAT) was never a good idea and this is the main reason
why the cheap network device suffer. It should not need any kind of
intelligence. When some of us "Software" guys are up in arms over the slow
IPv6 adoption it is because the current IPv4 state translates to thousands of
extra code lines for little to no benefit.

We also need those cheap consumer devices to understand Active Queue
Management, but that is another problem :)

~~~
dsl
Having seen the "sausage factory" of consumer gateway devices myself, I can
tell you the problem is not NAT or any other technology you'd like to blame.

The specs are written by non-technical product managers in the US, a
contractor in Taiwan selects the OEM boards that most closely fit the spec
requirements, pieces are combined and packaged in China with firmware
outsourced to a completely different company in South Korea working off the
OEM specs not the designs of the final product. (none of this is an
exaggeration and is an example from a major broadband router maker)

They literally don't know if a feature works enough to print on the box until
the first batch is unloaded off the boat.

~~~
noobface
Heh. You've seen what I've seen.

------
bishop_mandible
Note: this blog posting is 2 years old. In the meantime Go 1 was released with
full Windows support and the ecosystem has matured a lot and it's used in
production.

------
mseepgood
"Written on Jun 13 2010"

Why are people posting ancient and outdated stuff to Hacker NEWS?

~~~
scott_s
We regularly post old articles. Not everything we post is "news." However,
common practice is to put the year in the title if it's not recent.

~~~
S4M
That is true if the content of the article contains a truth that is not time
dependent, or if the article is posted for historical purpose. But this
article is comparing one new language against a more established one, so in
two years you should expect the new language to evolve significantly.
Especially, the problems mentioned in the article about Go may be now fixed.
Talking about the Go from 2010 is just not relevant in 2013!

------
riquito
Where is the math? Really, what's the point of making assumptions on the whole
subject and invent a conclusion? You may be right, but you need facts to prove
a point (which here is "go is as easy or easier than Python, faster, use less
memory and has better concurrency"). At least the source code of both
implementations..

------
jebblue
>> Go is used internally at Google, so it won’t be abandoned.

That's what I used to think about GWT.

------
asimjalis
Do you have a link to your source code? It would be interesting to see the
difference in the actual code.

~~~
jlarocco
Yeah, an article like this is almost useless without being able to see the
author's code and see why he came to the conclusions he came to.

~~~
dsl
The author didn't post this to HN. It's over 2 years old and reads like a
"brain dump" type blog post.

------
jstanley
"Speed and memory efficiency

I haven’t measured."

This is the part I was most interested in. It's my understanding that
Goroutines are not currently implemented very efficiently and I wanted to see
how the current implementation compares to a typical Python version.

~~~
jff
Goroutines are "green threads", small threads managed by the Go runtime and
scheduled within multiple OS processes. I've never seen any indication that
they're inefficient; I regularly create and destroy thousands of goroutines in
less than a second. Oh, and Go doesn't have a global interpreter lock, so I'd
expect the threading to work better than in python.

------
Goopplesoft
> I haven’t measured.

I'm extremely interested in the measurements if someone could do them. I want
to know how much the web concurrency/gorutines adds in terms of load testing
and response times.

~~~
voidlogic
This is an extremely simple comparison, but unless you are really pushing to
garbage collector in go 1.0.3 these results are pretty typical across the
board: <https://docs.google.com/open?id=0ByTAIYFOarh0cm9aSXlzc0NWaXc>

Go tip (aka 1.1 alpha/in progress) is even faster!

~~~
voidlogic
My previous link is a more general performance comparison across many
languages. I realized upon re-reading that you were more interested in
concurrency/scaling, this might be more what you are looking for:
[http://eric.themoritzfamily.com/websocket-demo-
results-v2.ht...](http://eric.themoritzfamily.com/websocket-demo-
results-v2.html)

~~~
Goopplesoft
Thanks!

------
SeanDav
I would like to see the equivalent done with node.js and something like
express for comparison purposes.

~~~
voidlogic
If you do this comparison and are doing performance testing yourself, make
sure to test the scaling. At low concurrent connection counts I find their
request handling latency to be pretty similar, but I have found Go scales much
better.

Also, another factor (and different type of scaling), is how big the test app
is. Go is for the most part written in Go (stdlibs etc). But in node.js,which
is mostly C under the hood, as the app gets larger and larger and the ratio
time spent in C vs JS changes to favor JS, things can take a downhill turn.

Enough said: <https://twitter.com/felixge/status/275320642059001857>

------
jeremyjh
> The downside of that is that at any given time Tornado can process only one
> HTTP request. If that processing takes a long time, it blocks all other
> requests.

What? The whole point of an async framework is to service multiple HTTP
requests concurrently.

~~~
smilliken
Yes, concurrently, but not necessarily in parallel [1]. Here's an example of
what the author meant:

    
    
        while (True);
    

Executing this within a request will block all other requests in a single-
threaded web server.

[1] [http://stackoverflow.com/questions/1897993/difference-
betwee...](http://stackoverflow.com/questions/1897993/difference-between-
concurrent-programming-and-parallel-programming)

~~~
jeremyjh
The author is talking about concurrency, and so am I. Why the downvote? That
quote comes from his paragraph headed "Concurrency". It also refers directly
to concurrent HTTP requests. Well, a single Tornado or Node.js process can
generally handle many thousands of concurrent HTTP requests if they are
implemented correctly without blocking on I/O.

~~~
spankalee
But even if the request handlers don't perform blocking I/O, the interpreter
is still single threaded, so any computation the handler does blocks all other
handlers - only one runs at a time. "Concurrency" in the context of node.js
usually is referring to simultaneous connections, not parallel handling of the
connections.

------
ternaryoperator
>Go is used internally at Google, so it won’t be abandoned.

I don't see how A implies B. Given the number of APIs Google used and
abandoned, I think the best that can be said is: Because Google uses Go
internally, it's less likely to be abandoned.

------
laurent123456
He wrote that "In Go, writing concurrent code is more natural and efficiency
of goroutines is much better than threads." I would have assumed goroutines
were threads too. If not, how are they implemented?

~~~
wmf
Goroutines are M:N green threads but they're not 1:1 native threads.

------
st3fan
These comparisons always forget about packaging and deploying.

~~~
prewett
I would imagine that the Python deployment process is a few `pip install
package-name` and a `hg clone ...`.

For Go, `go build -o name` creates a complete, statically linked binary, so
deploying is `cp name dest/`, but I'd guess to deploy to your server you
usually just do `hg clone ...` and call `go run name`.

~~~
voidlogic
Most shops that use a compiled language like Go take advantage of the fact
that they do not need to take the risk of storing source code on production
hosts.

The deploy process is most likely an scp or git pull, etc from a binary/asset
repo. Go binaries are statically linked to you have no dynamically linked
libraries ( _.so/_.dll files) to worry about either.

If you have the binary you don't need to use "go run" which is a compile and
run, just run it (nohup ./myProgram &) or run it with which whichever
supervisor script/daemon/etc you use for other services.

