
Azure Functions vs. AWS Lambda – Scaling Face Off - deegles
https://www.azurefromthetrenches.com/azure-functions-vs-aws-lambda-scaling-face-off/
======
tnolet
Wow, that is pretty substantial difference between AWS and Azure. I’m pretty
confident Microsoft is hard at work getting these numbers better, but it shows
just how lonely AWS is at the top. I’m pretty deep into AWS-land but I’m
rooting for the boys & girls in Redmond; more competition == better service.

~~~
QuinnyPig
Concur; despite being so deep in AWS I've drowned here, competition makes all
of the offerings in this space better. It'd be nice to have a migration
strategy in my back pocket if I needed it; today that's a bit of a pipe dream.

------
taspeotis
“On Azure I’m using .NET 4.6 on the v1 runtime while on AWS I’m using the same
code running under .NET Core 1.0.“

.NET Core has way less baggage than .NET 4.6 [1] so you’d be expecting AWS to
come out ahead.

It looks like Azure functions supports .NET Core now [2]. (Edit: just noticed
the author is aware of this [3].)

Of course it might also be Blob storage having higher latency than S3.

Having used Azure Functions before I agree with the article’s criticisms about
scaling.

[1] [https://www.ageofascent.com/2016/02/18/asp-net-core-
exeeds-1...](https://www.ageofascent.com/2016/02/18/asp-net-core-
exeeds-1-15-million-requests-12-6-gbps/) Mind this article is talking about
comparing ASP.NET with ASP.NET Core, and .NET with .NET Core, not just .NET
Core’s performance.

[2] [https://github.com/Azure/Azure-
Functions/issues/98](https://github.com/Azure/Azure-Functions/issues/98)

[3] [https://www.azurefromthetrenches.com/azure-
functions-v2-prev...](https://www.azurefromthetrenches.com/azure-
functions-v2-preview-performance-issues-net-core-standard/)

~~~
karavelov
He also compares AWS Lambda with Azure blob storage backend and it still lot
faster - just 25ms added latency to cross internet from AWS to Azure DC.

------
lvh
My experience so far on Google Cloud has been just as bad. I want a simple app
that ingests DMARC reports.

\- Google Cloud Functions can't ingest e-mail. Also, JS lockin if you care
about which platforms are supported.

\- Google App Engine can only ingest e-mail in the Standard environment. OK,
fine. But the standard environment regularly takes almost a full minute to
load and boot my app, after which the request is killed for taking too long.
Additionally, the sample code (!) for opening new Stackdriver contexts breaks
with a totally inscrutable error message.

You can say "well, maybe your app sucks", but it's been running on AWS (SES +
Lambda) for months and every aspect about it has been painless.

~~~
mayank
Use Mailgun to trigger a POST to your Cloud Function on receiving an email.
Bonus: all the parsing is done for you.

~~~
teej
I don’t know how magic is in Cloud Functions, but I personally set up Mailgun
postbacks in AWS and it was a huge pain in the ass to configure correctly. I’d
love I know if google does something differently that would have been easier.

------
Axiverse
Do you know how Google Cloud Functions compares with AWS and Azure?

~~~
spyspy
If you're going to be handling http requests it's almost a given you'll be
using AppEngine rather than Cloud Functions. Still "serverless" but more tools
and languages to choose from.

~~~
the_common_man
AppEngine is more like a PaaS and not like 'serverless'

~~~
seangrogg
This. AppEngine assumes an "always on" mentality and is far from "pay for use"
unless your "use" is 24/7.

~~~
octalmage
AppEngine standard can actually scale down to 0 instances when not in use to
save money. Still very different from a serverless architecture though.

~~~
nothrabannosir
Not the flexible environment iirc. Which is basically the reasonable env to
expect, now. Standard env has some aging issues that keep getting worse. most
notably jvm 7, if that’s your poison, but just all round crippled env lacking
standard libs and with lock in substitutions.

As far as I remember, that is..

~~~
iangudger
Java 8 is now supported with the full standard library:
[https://cloud.google.com/appengine/docs/standard/java/runtim...](https://cloud.google.com/appengine/docs/standard/java/runtime-
java8)

------
intrasight
This face-off will be a lot more meaningful in a couple months when we can
have both serverless platforms running on .Net Core 2.0. I do look forward to
reading the results of that updated face off.

~~~
intrasight
I'm personally leaning towards using AWS Lambda for .Net because unlike Azure
it will also support ASP.NET.

~~~
bonesss
In general, from the perspective of a dev in a MS-shop, AWS gives you a bit
more 'responsibility' for infrastructure than Azure, but that level of control
lets you get further in practice sooner.

Azure has a lot of great things going for it, but I've had a lot of issues
getting newer tech out on it without considerable delay. I hope both Azure and
GCE give AWS a run for their money in coming years :)

------
appdrag
The biggest issue with Azure functions is the pricing model where you have to
provision and pay for servers to execute your functions... How is that
serverless?

~~~
intrasight
You don't have to provision servers with Azure Functions - but you optionally
can.

------
krisives
Does anyone know if any of these services provide the ability to cycle between
IPs? One of the use cases for Lambda (and now Azure Functions) is in web
scraping where companies get mad if you access them too many times from a
single address. Lambda made it clear while they do use a lot of IPs you're not
guaranteed to get distribution among them.

Seems like a long shot but I figured this is a good place to bring it up.

~~~
intrasight
I have the same question. Not for scraping but for APIs that are rate-limited
by IP address. Like some Google APIs. I certainly wouldn't count on IP
distribution from getting around limits for long. Anyway, I'm more interested
in understanding how, if at all, we will have to tweak our client code to be
well-behaved.

------
dronechip
Interesting article, but you don't mention what tier of AppService plan you
tested against in Azure (or what memory limit your Lambdas had)? I would also
be curious to see the same comparisons, but with Azure Functions in
consumption mode which, despite the perceived drawbacks you mention, would
really be the closer apples-to-apples test, wouldn't it?

------
mcintyre1994
Does AWS have first class fully production Python yet? In Azure a single
import (pydocumentdb) took a function up to 20-30s on Python, I guess they're
serious about it being experimental still sadly. Rewriting in Node it takes
50ms for the entire run, crazy difference.

------
daxfohl
I do like the "durable functions" concept in Azure preview. AWS has step
functions, but coding in flowcharts isn't scalable.

------
carlsborg
My money is going to the innovators and first movers, regardless.

~~~
p0rkbelly
That mindset doesn't change these numbers then seeing that AWS released Lambda
in 2014...years ahead.

~~~
carlsborg
Have to agree with that part, and am in no way taking away from the
commendable engineering on the Azure side. But as a developer, it makes more
sense for me to invest time and effort getting good at the platform that
innovates faster - and the pace at which AWS has been getting out new
functionality in the platform has been breathtaking.

------
berns
The fact that Azure provides no SLA for functions apps running under
consumption plans is clear evidence that it's not ready for production use.

~~~
QuinnyPig
To be fair, there's no formal Lambda SLA either.

------
abol3z
Did anyone try Bluemix Functions?

------
pwaai
so does this mean that the cold run startup time thing has really been
improved enough to build web applications? To me that is the biggest killer in
moving forward, not knowing what the end users experience is going to be like
because of cold startups that forces a user to wait a long time.

From the article's data, it's really tempting to use AWS Lambda now given this
benchmark.

------
dmitriid
Never ever use averages in your stats. Use medians and percentiles.

Averages are a meaningless stat especially if you benchmark.

~~~
wilgertvelinga
I would say leave the average in but also show standard deviation and 95th
percentile average. Of course it is still very unlikely that Azure Functions
would come out on top in this case.

~~~
currysoup_t
What leads you to believe performance follows a normal distribution?

For a given especially slow request would you expect a corresponding
especially fast request?

~~~
DenisM
Standard deviation applies to all distributions, not just the normal
distribution.

------
viggity
this is absolutely not a function of Azure vs AWS, the perf diff is completely
between .Net Framework (which was used on Azure) and .Net Core (which was used
on AWS). This compares Apples and Watermelons.

[https://www.ageofascent.com/2016/02/18/asp-net-core-
exeeds-1...](https://www.ageofascent.com/2016/02/18/asp-net-core-
exeeds-1-15-million-requests-12-6-gbps/)

[http://codinggorilla.domemtech.com/?p=1499](http://codinggorilla.domemtech.com/?p=1499)

~~~
mnglkhn2
Is .Net Core available on Azure functions? If not, then the comparison is an
accurate snapshot of the current status.

~~~
starik36
In preview. [https://azure.microsoft.com/en-us/roadmap/azure-functions-
su...](https://azure.microsoft.com/en-us/roadmap/azure-functions-support-for-
net-core/)

~~~
jamesdrandall
And the performance is dreadful: [https://www.azurefromthetrenches.com/azure-
functions-v2-prev...](https://www.azurefromthetrenches.com/azure-
functions-v2-preview-performance-issues-net-core-standard/)

