
Go Is a Great Fit for Lambda - craigkerstiens
https://brandur.org/go-lambda
======
seniorsassycat
I don't understand why AWS released Go support instead of binary support and I
don't understand why they chose to rely on go's net/rpc.

The Go 1.x runtime will happily execute any binary, go, rust, c++. Events are
sent over a TCP stream. That stream uses Go's net/rpc which encodes objects
using Go's special [gobs] binary format.

If they had chosen a more common exchange format it would be much easier to
support new runtimes. I haven't found any implementation of gobs outside of
go. If you write one for a language you could use that language to write a
native Lambdas.

[gobs]:
[https://golang.org/pkg/encoding/gob/](https://golang.org/pkg/encoding/gob/)

~~~
sreque
part of the "magic" of Lambda is making it easy to use from your programming
language of choice. If they made a lower-level model, like, for example, "we
start up your binary, and it needs to bind to port X, then we'll send HTTP
requests to it and it should respond appropriately using the following JSON
specification," it would take a lot more work to set up your Lambda function
and get it working correctly than it does now. It's certainly possible for
them to do it, but I don't think it fits in with their style.

Worst case, you can implement this interface yourself now easier than ever.
Since golang is statically linked and has super fast startup times, you can
build an interface where Lambda launches your golang process, and that process
binds to a port and spawns a sub-process that implements the actual Lambda
function. The golang process would implement the "socket" based interface for
you.

People have been doing the above already, but all the existing runtimes,
including Node.js, Java, Python, and .NET, have non-trivial startup times and
memory usage compared to a golang binary, which should be able to start up in
10s of milliseconds and use less than 1 MB of memory while acting as a proxy.

~~~
donatj
I don't understand why it can't all be done over standard in / standard out?
Why bother with ports at all?

~~~
ninkendo
Because then you’d need a (inefficient) fork/execing webserver, or some
convoluted (error-prone) way of multiplexing multiple connections over one
stream. The OS file descriptor model already handles this for you with
listen() and accept().

CGI only works if you’re willing to take the hit of launching an entire
process (good luck using any interpreted language, and even go needs some
runtime initialization at startup) for every single web request.

~~~
nivertech
Lambda/FaaS is more like FastCGI

[https://en.wikipedia.org/wiki/FastCGI](https://en.wikipedia.org/wiki/FastCGI)

~~~
ninkendo
Replying way late here, but FastCGI doesn't work by printing to stdout, it
talks to a domain socket or an open TCP port. The grandparent post was asking
"why can't lambda just use stdout?"

The magic of TCP or domain sockets is that calling accept() gives you a unique
file descriptor that corresponds with a single client.

Just using stdout doesn't work that way, unless your entire process was
launched to handle one request, hence CGI being very inefficient.

------
KirinDave
I'm tired of Golang proponents suggesting that benefits every compiled, native
language gets are unique to go in comparison to non-compiled or VM-targeted
languages.

I wish lambda would do more to support more executable contexts instead of
blessing individual languages.

~~~
wmf
If your career consists of Ruby->Node.js->Go then static binaries are radical
new technology.

~~~
mercantile
> _If your career consists of Ruby- >Node.js->Go then static binaries are
> radical new technology._

Rather than offer what amounts to nothing more than an ad hominem, perhaps
you’d care to explain why whatever you’re using is superior.

Your preference may be containers or bytecode bundles, but you’d be hard
pressed to justify a major advantage and add a lot of complexity overhead for
the privilege of using them.

~~~
KirinDave
> Rather than offer what amounts to nothing more than an ad hominem

Is this ad hominem? Why?

We can both agree to this and agree that Go's compile speed makes very large
code bases possible (and that its cross-compile story is very compelling when
your entire world revolves around writing on OSX machines and running in a
linux docker container.

Lots of languages can deliver big static binaries. Hell, once upon a time this
was considered a _disadvantage_ and languages like Common Lisp and Smalltalk
were totally raked over the coals for having an image & executable model!

> but you’d be hard pressed to justify a major advantage and add a lot of
> complexity overhead for the privilege of using them.

Why then is Go given this?

Because I could name 3 languages equally deserving right off the bat and one
of them would be way less work: Kotlin, Swift and Haskell.

------
nzoschke
I'm also finding Lambda is a very nice runtime for using the Golang toolchain.

There's a packaging challenge, but Go is also easier to install than most
languages (unzip the linux build and set GOROOT correctly).

Then running `go lint && go vet && go build && go test` inside Lambda is fast
and cheap compared to other options. No more time spent in a queue,
downloading Docker images, initializing containers, etc.

I made a GitHub app if anyone wants to quickly try out building their Golang
project on every build in a Lambda function.

[https://www.mixable.net/docs/bios/](https://www.mixable.net/docs/bios/)

------
ealexhudson
6 years is "unheard of" for language stability??

I struggle to think of a language where that hasn't been true..

~~~
brandur
There's a pretty similar discussion happening downthread:

[https://news.ycombinator.com/item?id=16169803](https://news.ycombinator.com/item?id=16169803)

The short version is that I don't think it's fair to compare Go's stability to
that of a "barebones" language with an infinitesimal stdlib (e.g., C), because
that's where most of the change is likely to happen. Go's already proven to be
far more stable than most "batteries included" languages: think Python 2 vs.
3, Ruby's fairly considerable churn over the years, PHP 4 to 5, etc.

Obviously my designation on Go here is subjective, but it seems to me that it
compares favorable to any other platform that comes to mind.

~~~
slaymaker1907
I think the point people are trying to make is that even your supposed
unstable examples such as python don't do big, breaking changes on small time
scales. python was originally created in 1991, python 2 came out in 2000, and
python 3 came out in 2008. Even if we assume that python was only stable after
the release of python3, it has been pretty stable for about a decade whereas
Golang has NOT EVEN EXISTED for that long.

~~~
inlined
Eh. Close enough. Go is 9yrs old. I don't think the difference in 9 and 10
years deserves all caps.

------
Xeoncross
Can someone provide good examples of why you would use a Lambda to respond to
traffic instead of regular VPS? Is it just to save transfer costs inside the
AWS network? Go can serve hundreds of thousands of connections a second on a
$20/m DigialOcean/Linode/Scaleway/Volr server.

~~~
tnolet
Say you have a pet project that does 500 to a 1000 API requests per day. If
you stick that into a Lambda, you have 0 running cost are close to zero due to
AWS free tier.

~~~
Xeoncross
Are you talking about a project that does 1000 AWS API requests a day?

~~~
seangrogg
Not the parent, but I assume he means an application that itself takes in 1000
requests per day.

At the free tier your options include, on a monthly basis:

* a single EC2 t2.nano box with 0.5gb memory and 1vcpu

* 1,000,000 requests (or 40,000 gb-seconds worth of requests) to your lambdas

Assuming you're taking in less than 1m requests monthly the lambdas will
handle traffic spikes much more gracefully than the small VM will; as well,
since it's not an either-or proposition this also means you still have a free
VM you can do something with.

------
skybrian
You might want to compare how Go works on App Engine. You deploy your Go app
to App Engine by uploading its source code, and the minor version is specified
in app.yaml.

It has taken some time for Google to support new versions of Go on App Engine,
though they're promising to get better at it.

~~~
spyspy
Despite their claimed improvements I believe they're still only supporting
every other Go version. Fingers crossed for go1.10

------
dominotw
Can somone tell me how are we supposed to deal with things like "postgres
connection pools" in a serverless world.

Do we need to open a new connection to the DB on every call?

~~~
Xeoncross
I found this reply from Amazon helpful.

 _TL;DR don 't use a connection pool in Lambda_

> Now I see that you mentioned in one of the github posts: "The Lambda does
> seem to use the pool, but if I make maybe 3 / 4 quick requests to the
> lambda, new connections are opened up despite there being all of those
> connections already."

> 3/4 quick requests would mean concurrent invocations, and concurrent
> invocations will execute in new containers. Containers are not aware of each
> other and cannot share state information so if an execution occurs in new
> container, it will open a new connection if there are none in this new
> container, despite there being existing connections from different
> containers. This can also cause your RDS instance to have too many open
> connections, because another container may have all the connections
> saturated, and if a new container is required to handle the load it may not
> be able to create the new RDS connection it needs to do it's work.

\-
[https://github.com/sequelize/sequelize/issues/4938#issuecomm...](https://github.com/sequelize/sequelize/issues/4938#issuecomment-245211042)

~~~
mjb
For most uses, database connection pooling (or, more accurately, keeping a
connection per container), is the right practice to follow in Lambda. For many
databases, creating a connection is expensive and slow, and amortizing those
costs over multiple container uses (see
[https://aws.amazon.com/blogs/compute/container-reuse-in-
lamb...](https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/))
helps reduce latency and cost. Lambda allows you to configure a per-function
concurrency limit, which you can use to make sure that the number of
containers doesn't exceed your maximum database connections.

------
fnl
Java and C++ are virtually fully backwards compatible. Java 1.6, 10 years old,
is still commercially supported. C++ compiler support depends, but can go back
far, too. Golang only seems to support the latest two [minor] versions,
however (so, 2 years, maybe). I don't see how Go's long-term support is in any
way outstanding. And, we still need to see what happens when Go moves to
version 2...

~~~
cratermoon
See The Go 1 Compatibility Promise[1]. Something programmed in Go 1.5 still
works in Go 1.10

1\. [https://golang.org/doc/go1compat](https://golang.org/doc/go1compat)

~~~
fnl
What part of my statement makes you think I said anything to the contrary of
that?

~~~
cratermoon
> Golang only seems to support the latest two [minor] versions, however (so, 2
> years, maybe).

Go 1.0 was released 28 Mar 28 2012.

Go is on a six-month release cycle[1]. Focusing on the number of minor
versions is not only the wrong thing to do (because different languages
release updates at different rates) but it's also factually incorrect, because
by schedule, 2 minor releases happen in a year.

1 [https://github.com/golang/go/wiki/Go-Release-
Cycle](https://github.com/golang/go/wiki/Go-Release-Cycle)

------
sidlls
6 years stability is all but unheard of? Maybe in the webdev world. Even
there, really?

------
acangiano
This is the stack Apex uses:
[https://apex.sh/#products](https://apex.sh/#products)

------
pcwalton
Six years for language stability is nothing compared to, for example, C89.

~~~
skybrian
But for a fair comparison you'd need to include a lot of C libraries to match
Go's standard library. How much have those libraries changed over the years?

~~~
brandur
(I wrote TFA.)

> _But for a fair comparison you 'd need to include a lot of C libraries to
> match Go's standard library. How much have those libraries changed over the
> years?_

Yes, exactly. Something like C isn't a fair comparison because there's so
little included in the core language. Think about everything that you get with
Go's standard library: concurrency, HTTP (even HTTP2), TLS, JSON, XML, Zip
encoding, Regexp, etc., and then consider C's APIs along with all the APIs of
all the libraries you'd need in C to accomplish those tasks — all of that
together would be a more fair comparison.

That might seem like apples and oranges, but for most real-world programs,
people generally need a considerable subset of that functionality. C's
stability is nothing to scoff at, but it's nowhere near as impressive of an
achievement.

~~~
slaymaker1907
A fairer comparison might be python3. Even if you assume python has only been
stable since python3, it is still older than Golang since Go came out in 2009
and python 3.0 was released in 2008.

------
tuananh
it seems the same things can be said about .net core as well.

~~~
jrs95
Wouldn’t that suffer from most of the issues Java currently does in a
serverless environment? Or do you mean .NET Core with native compilation? I
thought that wasn’t really used outside of UWP stuff.

~~~
sudhirj
Native compilation would count as a binary, and Lambda doesn't support
arbitrary binaries yet.

There are arguments on this thread that they should have supported arbitrary
binaries by using a generic RPC format, instead of th Go specific RPC format.

