
Comparing AWS Lambda performance of Node.js, Python, Java, C# and Go - dfirment
https://read.acloud.guru/comparing-aws-lambda-performance-of-node-js-python-java-c-and-go-29c1163c2581
======
petethepig
This is a good example of an absolutely pointless comparison. The author is
making some bold statements:

> Go performance is comparable to Java

> .Net Core 2.0 can execute up to 3x faster than Go.

> Consistent performance of compiled vs dynamic

But what are they measuring? How fast can a lambda function return "200 OK
hello"? This is just sad.

~~~
hugelgupf
They are effectively measuring how fast AWS infrastructure can reach whatever
it needs (VM?) for each language.

But these are not even cold start times. I'm assuming after the first request
this is using the same VM (/backend job, whatever) to reply, so now it just
depends on latency to get to that VM. (I'm assuming the runtimes' ability to
return "200 OK" is similar and the latency of that is negligible compared to
AWS infra that is involved.) Not completely meaningless, but not super useful
either, esp without any details while making those bold statements. This is
not "Go is equal to Java". This is "AWS Lambda infrastructure for Go is
comparable to AWS infra for Java". Article is unclear on that.

Maybe this tells us AWS lambda backend infra is way different for .NET Core
than Go and Java? ¯\\_(ツ)_/¯

~~~
petethepig
> This is "AWS Lambda infrastructure for Go is comparable to AWS infra for
> Java"

I wouldn't even go that far. They are not measuring anything that any real-
world application would do. Heck, it's not even a todo app, it's just a "200
OK hello".

It's more like "AWS lambda function invocation overhead for Go is comparable
to AWS lambda function invocation overhead for Java".

~~~
weego
Sure but if you start adding in language specific Web frameworks / json
handling / db work you're no longer testing AWS.

I agree there's questionable value here but at least it's bring constrained to
as few variables as possible.

~~~
lsaferite
Benchmarking the lambda-add latency is something that is useful as long as
it's called out as such a benchmark.

------
tofflos
Why the hate? The methodology is sound, the results are interesting and they
show that there is some room for improvement in language support. Also the
source code has been provided in case anyone wants to alter the benchmark to
be more comparable to their workloads.

Hopefully this sparks a race between the different teams within Amazon. Great
job!

Oh! These test scenarios seem to have a pretty good acceptance in case you
decide to do another benchmark in the future:
[https://www.techempower.com/benchmarks/#section=code&hw=ph&t...](https://www.techempower.com/benchmarks/#section=code&hw=ph&test=fortune).

------
sjellis
To the extent that this benchmark is valid, this is interesting:

"In the original 2017 benchmark results, the compiled languages of Java and
.Net Core 1.0 were clearly more consistent compared to the dynamic languages —
Python and Node.js.

Based on the latest 2018 performance testing results — we are no longer
observing any significant variances between Java and the newer compiled
languages (.Net Core 2.0 and Go), and the original dynamic languages."

TLDR: You may be able to use a "slower" language for Lambda without
necessarily losing much.

~~~
singularity2001
>> You may be able to use a "slower" language for Lambda

at least for small projects

------
thepumpkin1979
Ok, now being serious, they need to include a real example here. Whenever I
run a Lambda function is to process something and make a call to at least one
external service. I was a .NET developer for a long time and I seriously doubt
.NET Core has been improved to the point where blocking http calls are handled
better than in Go runtime. I may believe .NET Core may have some performance
advantages in some CPU bound scenarios, but nothing more. Until then, this
benchmark proves nothing.

~~~
jinder
In which scenario would you use a blocking http call?

~~~
avita1
Not necessarily HTTP, but I imagine hitting some sort of database from a
lambda is not uncommon.

~~~
Jtsummers
Isn't the intent with a system like Lambda that you'd not do any kind of
blocking action. Instead, you'd trigger some event that the AWS database
solutions would receive. Once the DB has finished the query it sends the data
back and triggers another lambda function.

EDIT: I write this as someone who does no web programming and has only toyed
with AWS. This was my understanding from what I read, but not something I
actually made.

~~~
lostcolony
No. "Once the DB has finished the query it sends the data back and triggers
another lambda function"

What mechanism is the DB 'sending' data via? Is it just the response part of
the initial query? Then it's executing in the context of the same lambda
function. You -could- then call another lambda function from the running
lambda function (but why?), but more, what was the function that executed the
query doing while it waited?

If it had something it -could- be doing, it should be doing it, so non-
blocking is important. If it had nothing it could be doing, it literally does
not matter if it was a blocking call or not; it doesn't effect anything.

Now, if you meant you have a trigger in the DB that calls another lambda
function, such that the original lambda could complete, yes, of course you
could do that. However, if this was a GET style call, not a PUT, your options
to get the data back to the user are limited if you don't just keep the REST
request open and respond that way. Certainly, any other solution is more
complex, probably more expensive, and may not be possible.

------
chrissnell
Lambdas bill at a 100ms minimum. What's the point of a few ms of difference
here, unless you're running some computationally-intensive code? If that's
what you're doing, the efficiency of the language itself is going to be what
matters, not Lambda's speed of executing the runtime.

------
dmitriid
> we are no longer observing any significant variances between Java and the
> newer compiled languages

<proceeds to show averages instead of mean and/or 90th percentile even though
maximum spikes are easily orders of magnitude higher than average>

------
QuinnyPig
Go on par with Java? .NET 2.0 beating everything else out?!

I feel like my world has just flipped upside down.

------
nzoschke
I have been doing a lot of experiments with Go and Lambda and have an open
source boilerplate app here:
[https://github.com/nzoschke/gofaas](https://github.com/nzoschke/gofaas)

This benchmark leaves some room for improvement. With Lambda and X-ray you
could really deep dive into where time is spent in a request. Here’s an
example trace for a Golang function that does some actual work against KMS and
DynamoDB:

[https://twitter.com/nzoschke/status/969598630033162240?s=20](https://twitter.com/nzoschke/status/969598630033162240?s=20)

One thing I found surprising about the Go support is that the ‘lambda.Start’
helper uses reflection to invoke your go func. This could contribute to some
of the cold start time.

~~~
dilap
Yeah that certainly seems...non ideal.

------
equalunique
I am glad that the person who created this intended for the results to be
reproducible. I will try my hand at what's in the git repo, and perhaps
compare with some other languages too.

------
stunt
Comparing their open-source ecosystem, I still prefer to build my lambdas with
NodeJS (or even Python) over C#. (if my only concern is performance)

------
igotsideas
What would be the right way to test this?

