
Serverless Benchmark – An independent benchmark of serverless providers - gitgud
https://serverless-benchmark.com/
======
Normal_gaussian
When graphs are shown next to each other like this it is important to use the
same scale across them. The Percentile graphs make IBM cloud functions look
significantly better than the competition, despite being middle of the pack or
worse.

Aside from that nit (which I see everywhere), this is awesome.

------
aloknnikhil
AWS' p99 is very impressive for hot functions. Pleasantly surprised by IBM's
performance here. Google seems to perform the worst of the lot. It's also odd
that Google's median cold start is faster than a hot function. Maybe the
concurrency is triggering some sort of back-off?

PS: Adding cost per request as a comparison would be nice.

------
sdan
Interestingly I’ve gotten much better results with workers than I have with
lambda

~~~
tyingq
It doesn't say where the metrics are collected from, and whether they include
the network latency from whatever point that is.

~~~
sdan
I did some testing a while back from servers globally (I used Postman w/
global monitoring) to test latency.

Pretty sure Postman uses some big cloud service (AWS or GCP) so you'd think
Lambda would've had an advantage.

Interestingly, that didn't seem to be the case (workers had 3-4x less latency
than Lambda).

~~~
tyingq
That is interesting. I was trying to roughly say that Cloudflare stats might
be artificially high/slow if measured from the center vs the "edge". Sounds
like perhaps not?

~~~
sdan
Yeah I kinda understood what you were saying, but as I also surprisingly found
out, it seems like they're fast in general regardless of where you're located.

I'll try to do more testing soon.

~~~
tyingq
I made a url shortener with Cloudflare workers and their edge KV store. I did
notice that it worked very fast in the US, but not so fast in Europe. The KV
store seems to require a US->EU trip if it's not a "hot key".

Or basically, it seems "single master" with the center in the US. No such
issue with the worker itself though...just the KV store. If I hardcoded the
data, or used the regular page cache, it was fast everywhere.

~~~
steveklabnik
Hi, PM of KV here. You should be seeing speeds improve in Europe within the
next few weeks. Your intuition about the single master is currently correct,
we're adding a second master in Europe soon, for exactly this reason.

