
Go After 2 Years in Production - treeder
http://blog.iron.io/2013/08/go-after-2-years-in-production.html
======
programminggeek
Go seems to be hitting some kind of tipping point where it's going from being
more of a niche thing with a small user base to something with a broader
appeal. I don't think that it's because go is changing so much as the kinds of
problems people are encountering writing web services that need to scale (or
at least have that option).

I've been comparing go to scala lately and what has really got me into go is
the development speed. The fast compiles and fast web page auto-reload make it
feel every bit as fast to develop in as ruby or python or php, but with type
safety, fewer tests, and very clean code.

Scala is a fantastic language, but even with jrebel autoreload you still have
2-3 second page reloads and a 5 second test suite reload. That seems like a
small thing, but the faster I see code change results, the more hooked I am to
the process. A 5 second wait is probably enough to get me out of the zone.

With go, on just a default simple test of a small thing it is less than a
second to compile/run. In revel, pages reload/update as fast/faster than they
do in rails/sinatra.

Oh, and with go, each web request will run in a separate goroutine, so you get
excellent nonblocking IO performance without dealing with the callback soup of
node.js.

It might just be irrational exuberance because I haven't built anything big
and messy yet, but so far go is seriously fantastic and solves a lot of real
world problems elegantly.

~~~
illumen
So you've enjoyed all the marketing and Google employee upvoting?

~~~
tptacek
Comments like this never, ever improve threads.

~~~
weego
The other thing that never improves threads is everyone only posting what they
think might get upvoted instead of saying what they actually think, especially
when there might be a grain of truth in it. Tedious comment back-patting is
the death of good communities.

~~~
gruseom
How about we avoid both trollish sarcasm and tedious back-patting, and say
what we really think politely? That's not so hard.

------
jd007
"Two years in, Go has never been our bottleneck, it has always been the
database."

I would expect this to be true with any language, if you code well. Regular
web applications do not have state so they are very easily scaled horizontally
anyway. Databases on the other hand are trickier to scale the same way and
will end up being the bottleneck almost all the time.

~~~
bhauer
In my experience, with slower platforms/languages, while it may be
conventional wisdom that the database is the bottleneck, that's not actually
the case in many circumstances.

Certainly in circumstances where you're doing a complex query involving fields
that are not indexed or several joins, you're going to be waiting on the
database.

But if you're just fetching rows by ID or indexed fields, slower platforms and
languages end up being a bigger bottleneck than modern databases. Sometimes
this is masked somewhat by the fact that the database drivers and/or ORM are
slow, so from the application's perspective, the "database" is the bottleneck.
But one should not confuse the drivers and ORM for the database.

~~~
jd007
If you are talking about a breakdown of time consumed during a single
request's processing, then yes on slower platforms/languages/frameworks, the
database access portion may not be the most significant percentage of time
used. But this is not that relevant as even the slower platforms usually can
handle a single request reasonably fast.

What I was talking about was more about in the scaling of a system, i.e. what
happens when your architecture needs to handle lots of requests. In this case,
it is very rare for the application server part of your architecture to be a
bottleneck in scaling because it is generally stateless (for normal web apps
at least) and hence very easily horizontally scaled out. Of course a faster
platform will allow you to use fewer servers but 15 servers on Go vs. 20
servers on Python is not that big of an issue.

~~~
bhauer
"Reasonably fast" is in the eye of the beholder.

In my opinion, many popular platforms and frameworks are not reasonably fast
at providing a response in real-world applications. As a result, many web
sites are frustratingly slow in my opinion (for example, a popular site used
for hosting source code repositories). If those sites were to capture and
share their profiling data (including time spent in drivers and the ORM), I
would guess the database proper would not be as great a bottleneck as
conventional wisdom says.

Perceived slowness is latency, and horizontal scaling doesn't necessarily
address latency. Horizontal scaling may help alleviate an over-taxed CPU
dealing with too many concurrent requests, but if a single request in
isolation runs in 300ms, it will not run quicker than 300ms. It may run
_worse_ when contending for CPU capacity versus concurrent requests, but not
better, unless a faster CPU is dropped in.

Performance matters, even in the world of horizontal scalability. Performance
brings reduced latency (user experience) and reduced cost (size of cluster).
If we can get that paired with an efficient, enjoyable developer experience,
then yay for us.

Finally, "15 Go servers == 20 Python servers" seems a little unfair to Go.

~~~
jacques_chester
I think the confusion arises because jd007 is discussing _throughput_ and you
are discussing _latency_.

Latency and throughput can be inversely related depending the precise
architecture of a system (queueing is the classic mechanism that trades them
off).

But in terms of "horizontal scaling", the goal really is to improve total
throughput. Often imposing a tax on latency due to coordination costs.

~~~
bhauer
True enough. This thread of conversation was kicked off by the premise that
most applications are bottlenecked on their database server. When I said, "not
necessarily," I meant as far as latency is concerned--for any given request on
a slower platform, it is likely that the database's contribution is actually a
minority.

That is the conventional wisdom I find in need of disruption.

But you are correct. Since a conventional database server can be more
difficult to scale horizontally on its own right, even that small latency
contribution, when multiplied by the number of queries being run by a wide
array of application instances may ultimately mean the database server is the
first observed bottleneck. By which I mean, the first device to reach 100% CPU
without the simple recourse to just throw more money at Amazon and spin up
another instance to solve the problem.

So I buy that.

But when I see slow web applications--when I criticize a site for being slow--
I am always talking about latency. When we look under the hood, assuming the
application is not being silly with its queries or doing a fundamentally
challenging work-load, user experience slowness (latency again) usually
originates from the application's own code or slow platform.

For example, I've observed applications that require 200ms of server-side time
to render a login page. Behind the scenes, I may observe that they are badly
designed and include one or two trivial (but utterly unnecessary) queries.
Still, those two queries can be fulfilled by modern hardware in ~5 to 10ms.
The remaining ~190ms of server processing is on the application. To my mind,
that is unacceptable. A login page should be delivered in ~3ms of server time
(under load!) on modern hardware.

And back to the OP, Go is a platform that brings JVM-class speed (the
capability to return a login page in ~3ms) to those who can't stomach Java.
Bravo to Go!

~~~
encoderer
I don't entirely disagree with you but I do think you're not doing anybody any
favors by "disrupting this conventional wisdom."

When I was an engineer at Formspring I profiled our social graph service which
was basically PHP client hitting a Python service that queries against
Cassandra. Thrift was used to communicate between PHP and the Graph Service
and, being Cassandra, Thrift was used between the Python Cassandra client and
the Cassandra server. So, two-way serialization, twice.

In the end, this isn't a bad design. I didn't write it then but if I were re-
writing it now I'd probably use protobuffs but aside from that, it was a clean
separation of concerns and fit in nicely with our larger SOA.

Point being, though, that serialization is CPU expensive. Reading from
Cassandra was blazingly fast compared to the work Thrift was doing.

All that said, I think noob engineers should be taught that network operations
(db queries included) are at least an order of magnitude slower than, say,
opening and writing to a local socket. Experienced engineers can see the
balanced view of things and agree with the point you're making, so advocating
it, IMO, will only serve to make you look smart and confuse noobs.

~~~
bhauer
I appreciate your point of view. It's not my intent to confuse noobs.

You're right, with that in mind, the conventional wisdom is worth retaining so
to instill the proper fear of treating database queries as trivial. Thinking
of queries as cheap--or not thinking about queries at all--is what gets
applications into a state where a single request runs dozens if not hundreds
of queries to deliver what is effectively static content! :)

That said, I've met more than a few senior folks who continue to fiercely
stick with the premise that database queries trump all. As someone else in
this thread said, profiling can be illuminating, even for senior developers.
But it seems on that point, we're all in agreement.

------
neya
Another Scala user here. And I've been using Scala for the last six months.
Mostly developing web applications on Scalatra and Play. I love Scala - It
still has its goodiness and it's an excellent alternative to Java.

If you notice in all of the Scala books (I've read the one by Martin and the
One by David Pollak as well), they all seem to push you towards the functional
programming model instead of the much more common imperative model.

But you know, there's a problem with that. Not with the language itself, but
with the web development side of Scala. There's only certain ways by which you
can program with a particular function that accepts a request and returns a
response. So, most of the time, I would find myself wasting time thinking "How
can I make this more functional?"

Functional programming is really great when you can actually enforce it - When
writing algorithms and so forth, but otherwise, you are forced to fallback to
this imperative model which sucks, because you have to constantly keep
changing your mindset between these two models.

Another downside of Scala is its HUGE syntax. Processing strings? Here's a ten
million ways to do it.. (a bit over-exaggerating). Oh and if you chose this
way, then you should use this syntax. No, not that one, THIS one. Oh no, not
that one, this variation of that one that looks like this.

I've advocated Scala to many HN members here in the past, you can even check
out my comments, for instance. But from my experience, I think Scala is an
Academic language. But it's superior in its own ways - I just LOVE the concept
of avoiding nulls and passing an Option. It's beautiful. But the downside is
its huge academic complex syntax. I want to be able to write code which
shouldn't reduce my hair count even if I look at it after 6 long months. I
don't want to refer to an 800 page book each time I'm confused about
something.

That's why I think Go is beautiful. The syntax is just enough to keep it
inside your head forever. I fear that as the language matures, this might also
evolve into something as complex as Scala, but let's hope not so.

Go isn't a magic bullet though. It has its own downsides, but nothing w.r.t
performance or similar. For the most part, it's awesome.

Once you go Go, you never go back.

P.s - I still love Scala as well ;)

~~~
rybosome
I've historically been pretty pro-Scala 'round these parts, so my response to
this should be considered in that light. So, with that disclosure out of the
way... =)

I have to admit, I don't agree with the assertion that the cognitive
dissonance associated with programming in an imperative way versus a
functional way is a major problem to Scala. I realize personal preference is a
very big part of this, but there are many problems which are just easier to
solve in a functional way versus an imperative way. Of course, the opposite
holds true as well. This is the beauty of Scala; one can write performance-
critical imperative code while exposing a functional interface; or, one can
consume a fluent functional interface to produce imperative code.

Frankly, I guess I've become something of a functional zealot, so my problem
with Go is that it's so stubbornly imperative. This is why I can't get behind
Go, as much as I want to. I feel like it doesn't consider many of the lessons
that have been learned about FP making life easier. Set operations (map,
filter, reduce) are insanely common, yet doing them imperatively sucks to the
point of discouraging them. Excessive mutability is difficult to reason about.
Nobody expects the nil/null/none inquisition. We tend to forget about side
effects. Without an extensible language, which requires truly first-class
functions, you're at the language designer's mercy.

Hell, Scala probably isn't the be-all, end-all answer to all of this. I just
don't think that a doggedly-imperative, non-composable, statement-oriented
language is the future of programming "in the large", not when the past has
shown us that we tend to produce buggy, unmaintainable software this way. I'm
pragmatic enough to realize that pure FP isn't a solution, but I feel strongly
that sticking to traditional imperative because it's familiar is a costly
mistake.

I can't argue with your take on the syntax, though, since that's personal
preference. =) If you have any thoughts on why you feel productive in Go, I'd
love to hear them; as I've said, I've been really struggling with motivating
myself to learn and enjoy it.

~~~
realrocker
Have ever tried programming in Go?

Edit: Functional Programming in Go 1\.
[http://stackoverflow.com/questions/4358031/functional-
progra...](http://stackoverflow.com/questions/4358031/functional-programming-
in-go) 2\.
[http://golang.org/doc/codewalk/functions/](http://golang.org/doc/codewalk/functions/)

~~~
gnuvince
Go doesn't lend itself to functional programming, in part because of its lack
of type parametrization. In a language like Scala or Haskell, using functions
like map, fold, and filter is clean an idiomatic. In Go, it is so ugly and
convoluted that you immediately reject this approach and use for loops
instead.

~~~
mseepgood
Standing firmly in one paradigm is a good thing for a language IMHO. I've
worked in multi-paradigm languages and they suffer from the fact that their
communities can't find a common style. "Should I use a fold or a loop or a
tail recursion?", or some library designers want to be as typeful as possible,
others don't, etc.

~~~
eru
> "Should I use a fold or a loop or a tail recursion?"

Neither, use a combinator, like map or traverse. (And that applies to all
languages that support this style.)

------
bitcrusher
I can't be the only person who thinks Go is really interesting, but can't get
over the 'package' hump... It's just too wonky for 'real-world' from my
experience. For example, how do you create clear lines of separation with
internal modules? If you want to use 'namespaces' then each namespace has to
be it's own package, which then requires its own Repo that you have to 'go
get'. There's unsustainable for a project of any useful size.

I must be missing something.

~~~
Game_Ender
Google has said they don't use 'go get' internally to manage dependencies, so
they appear to agree with you.

~~~
gt384u
I would imagine this has something to do with having an existing package
dependency management system company-wide and it causing friction trying to
get the two to coexist.

I have similar issues where I work with trying to integrate an internal build
system and rubygems. Our answer is to essentially mirror the gem version
internally into our own repo. It's not the best answer I could hope for.

------
Pxtl
I know it's cliche, but I still can't live without generics or a macro-like
approximation thereof. I suffered under Java 2 for too long to go back to
that.

~~~
rprospero
It's not cliche, it's just personal. For instance, most of the code I write is
numerical code. To that end, any language without operator overloading is a
non-starter, since it's far harder to find a bug in:

div(add(mult(-1,b),sqrt(sub(pow(b,2),mult(mult(3,a),c)))),mult(2,a))

than it is in

(-b+sqrt(b^2-3 _a_ c))/(2*a)

On the other hand, if I wrote server code, operator overloading would be far
less useful. I'd probably curse any programmer who used it and thank the gods
that it was left out of Go.

Conversely, since I write a lot of numerical code, I don't care about generics
or typing, which is crucial to many other programmers. Generics don't matter
since everything I work with is either a Double or a collection of Doubles.
Similarly, static typing doesn't help, since most functions just have the type
(Double -> Double) and the type checker can't tell a sine from a logarthim. Of
course, the reverse is also true. Since everything is either a double or a
collection of doubles, the fancy tricks that dynamic languages offer don't
give me a damn thing, so I'm extremely ambivalent about the typing debate.

Of course, on other projects, I've written code that benefited from static
typing and I've written valid code that would choke any type checker. I've
written code that heavily needed generics. When I did that, I used the
languages that had the features I needed.

Go just won't work for the programs I write and it sounds like it doesn't work
for yours, either. That's why we won't use Go. I've heard it works wonderfully
for a certain class of server software and I'm glad those guys have a useful
language for their domain. If I ever have to write another server, I might
even teach myself Go. But don't feel guilty that you're using an old hammer
instead of the shiny new saw.

~~~
howeman
It's not quite that bad in Go,

(-b + math.Sqrt(b _b - 4_ a _c)) / (2 _ a)

though if you're using a ton of matrices it could be.

I for one have found Go great for computing. The really quick compile times
with static checking plus the composability are great. It definitely depends
though; while the native concurrency is great there aren't a lot of easy
solutions for non-shared memory computations. (I saw an MPI package at one
point, but I haven't tried to use it)

~~~
howeman
Sorry, new to posting on HN. How do I get asterisk literals?

~~~
prodigal_erik
[https://news.ycombinator.com/formatdoc](https://news.ycombinator.com/formatdoc)
says two leading spaces formats code verbatim:

    
    
      (-b + math.Sqrt(b*b - 4*a*c)) / (2*a)

------
dancric
As a developer on the Python stack, I would love to know when would be a good
time to start using Go in serious production work. It seems to me that it
solves a lot of the backend services infrastructure problems associated with
interpretive languages (one of the reasons I was considering diving in Scala
or other JVM languages), is relatively reliable, and has a fairly strong core
library. It still seems bleeding edge, but the language seems to have
developed far faster than Python did over the last decade or so.

~~~
kintamanimatt
> I would love to know when would be a good time to start using Go in serious
> production work

Now would be a good time. No language, runtime, compiler, library, or
framework is ever going to be perfect, but now is a great time to dive in.

> It still seems bleeding edge

This is probably a good thing in many respects because Go doesn't have the
baggage from yore, and it was created by some pretty smart and capable people.

> but the language seems to have developed far faster than Python did over the
> last decade or so

Language designers are getting better at marketing. No language succeeds
without fantastic marketing.

~~~
danieldk
_This is probably a good thing in many respects because Go doesn 't have the
baggage from yore, and it was created by some pretty smart and capable
people._

As was Javascript plus Node.js two years ago, Ruby and RoR five years ago,
etc.

You'd think that more reasons are required than 'it is new, doesn't have
baggage in was created by smart people'.

~~~
kintamanimatt
Go isn't a framework. NodeJS is a framework, as is Rails. JavaScript and Ruby
are very old languages, both laden with baggage.

~~~
ricardobeat
node.js is not a framework, it's a platform written in c++ and js.

~~~
kintamanimatt
Yeah, you're right. Thanks for pointing that out.

------
txutxu
I wish we could know what happens "after 12 years in production" (just 10
more).

Update:

Sorry if someone did feel offended (the negative voting), I said it with my
best and more constructive intention.

In my opinion, the problem is not in the content of my comment. Well, maybe,
if someone understands it as: "oh, can't argue against that, is attacking the
language" instead of, "let run your imagination to 2023".

Have a nice day/night.

------
melling
The memory footprint for Go seems to makes it ideal for mobile devices. Also,
considering that Dalvik performance is much slower than Java, it really seems
like have Go on Android would be a huge win.

[http://en.wikipedia.org/wiki/Dalvik_(software)#Performance](http://en.wikipedia.org/wiki/Dalvik_\(software\)#Performance)

Go compiles to native, so "compile on install", would probably be needed.

~~~
programminggeek
Go on Android would be great, but Google used Java in the first place so that
they could leverage all the Java devs in the world to build Android apps.

Go might be a much better choice from a pure dev standpoint, but from a
getting everyone to build apps standpoint, it's a fail in the short/middle
term.

~~~
melling
I didn't say get everyone to build apps. It would simply be another option.
App development is getting more competitive, of course, so people that want to
stand out might need to move to Go to gain an edge. Plus, I image that a Go
environment might provide a better interactive development cycle.

~~~
programminggeek
Yeah totally. I'm with you there. It would be great and I hope they do it.

------
pjbringer
It's a shame they have actually converted a real system, and they rely on the
language benchmark to make an argument about performance.

That said, not having run into a problem is a worthwhile feedback.

------
ulisesrmzroche
Anyone got a good tutorial on Go? I'm interested in fiddling around with it.

~~~
BarkMore
[http://tour.golang.org/#1](http://tour.golang.org/#1)

~~~
ulisesrmzroche
Sweet, thanks Barkmore!

------
toqueteos
Not so relevant but for any of you who prefer Rust to Go and haven't tried Go,
don't do it.

Go's std, visibility by case and gofmt, among other things will make you cry
for using Rust.

I really hope Rust get's better with time and it really focuses on being
developer friendly not just a bag of nice features.

~~~
pcwalton
> Not so relevant

Then why post?

~~~
axaxs
I, as a user interested in both, did find value in the comment regardless of
his/her intentions. This comes off as a 'shut up' when really, it could be a
good opportunity to look at what Go does right for implementation in Rust, no?

~~~
pcwalton
Because it's plainly a troll. ("Go will make you cry for using Rust" made that
clear, especially considering the user's comment history, in which I've
already explained why at least visibility on case doesn't make sense.)

~~~
toqueteos
Not trolling at all, just sharing my opinion as noob with Rust coming from Go.

That particular quote is related to "cleanliness of the language" obviously
it's not an apples to apples comparison.

My apologies to all of you Rust devs who might get offended with my comment,
wasn't my intention, you are doing an awesome job I'm looking for 1.0
meanwhile I'll keep toying around with the language.

------
dodyg
This is awesome. I much prefer this to "we rewrite some stuff with Go"
articles.

------
zurn
Interesting that Go lost so heavily to Ruby on amount of code (~ 3x more on
average.)

------
knodi
Being using go for a year now. Really enjoying using it.

------
stephenitis
I would love to see codeacademy, codeschool, treehouse, and the online course
community to jump on creating go curriculum.

~~~
enneff
[http://tour.golang.org/](http://tour.golang.org/)

------
netcraft
on an unrelated (to golang) note, does anyone have any experience they would
like to share about iron.io? especially it vs rabbitmq or amazon sqs?

~~~
carimura
[Disclaimer/Warning I work there and this is about Iron.]

Vs RabbitMQ: \- Native cloud service over HTTP transport \- Clean easy API \-
Scales to unlimited queues/clients \- Push queues can have URL endpoints as
subscribers \- Highly available (our #1 priority is to keep it running at all
times) \- Nice UI to manage queues, stats, rate, etc. \- IronWorker
integration (workers as a service)

Vs SQS: \- Fast, clean API \- FIFO \- One time delivery \- Multi-cloud \- Push
queues / pubsub / fanout as first class feature \- IronWorker integration
(workers as a service)

Best way is to just try it out. It's already one of the leading cloud MQ's out
there and we have a lot of big plans for IronMQ to make it the safest "bet
your business" cloud message queue available.

~~~
rabbitmq
RabbitMQ supports HTTP and has a clean easy API. Scales in lots of ways.
Supports very high throughput on a single cheap AMI. Has awesome GUI and UI.
Etc. YMMV.

