
What a typical serverless architecture looks like in AWS - prostoalex
https://medium.com/serverless-transformation/what-a-typical-100-serverless-architecture-looks-like-in-aws-40f252cd0ecb
======
zimbatm
Please don't do this.

I have seen this in practice. A simple CRUD app split across literally 100 of
"re-usable" repositories. The business logic is all over the place and
impossible to reason about. Especially with step-functions, now the logic is
both in the code and on the cloud-level. Each developer gets siloed into their
little part and not able to run the whole app locally.

The whole thing could easily fit in a single VM as a Rails or Django
application.

The only one that will be happy about this is AWS and contractors because it's
a guaranteed lock-in.

~~~
api
The whole "serverless" fad is about lock-in.

I also really wish they would not have squatted that term. It should refer to
decentralized P2P systems which are truly serverless.

~~~
minitoar
Even without serverless you are pretty locked in. Things like Terraform help
but the level of cloud integration required for a complex system is pretty
overwhelming.

~~~
api
If its "required" you need to think more.

We use GCP and GKS for some things, but we could leave pretty easily. Hardest
parts would be DIY Postgres HA and DIY K8S. CockroachDB is almost ready to get
rid of the former headache. Haven't looked into the latter headache yet. Of
course we really only use K8S for load balancing and HA and there are other
options there like Consul and Nomad.

~~~
minitoar
I’m glad you are able to move your use case easily!

------
davnicwil
The immediate thought looking at that diagram is that while on first glance
serverless runtimes seem to promise that you don't have to care about the
details, turns out you really do have to care a lot about the details!

You're not really relieving yourself of concerns by using serverless runtimes,
just shifting them around. Sure, you don't have to care about managing state
at the leaves, but you very definitely do now have to come up with a way to
hold state globally with ephemeral/stateless leaves, and it turns out
that's... a really hard problem with a really complex solution space.

Not to be overly dismissive or say that one way is better than the other, it
definitely depends on the usecase and probably the underlying business model
of the usecase. Just to point out that when it comes to engineering
abstractions, there really isn't any free lunch. It's all tradeoffs.

~~~
scarface74
How is that any different than a typical web server? Hopefully you don’t
maintain state on your web server either. This isn’t a new concept. Even in
the early 2000s when I was maintaining two web servers behind a physical load
balancer we knew to maintain session state separately from the web server.

~~~
voz_
I had the same question - if this guy has each web request adding to local
state on his web server, I have some bad news...

~~~
scarface74
I hate to be “that guy”. But “the cloud” is no more than someone else managing
servers and services at varying levels of abstraction.

I have no idea why some people hear AWS/Azure and they act as if the same
architectural principles that have always applied still apply, just on a more
granular scale in the case of “Serverless”.

~~~
voz_
Not sure I understand the point you are making?

> and they act as if the same architectural principles that have always
> applied still apply,

They do still apply...

~~~
scarface74
I was agreeing with you.

More of a rant about the person who complains about not being able to maintain
state at a tier that you never maintain state at in even a traditional
architecture.

~~~
voz_
ah, gotcha! Very good :)

------
jpswade
I'd say this is what any distributed system looks like, it's not exclusive to
AWS and serverless architecture.

It's not even exclusive to distributed systems, most computer systems look
like this when you actually think about it.

Consider what an application would look like if you mapped out each service
and object, it would easily look as complex.

This isn't a new concept, we've had the idea that everything in computing
could be represented by little computers communicating with each other since
the 60s, Alan Kay called it "object-oriented programming".

The only difference is between the 60s and now, is that instead of mainframes
we have platforms, so now we can actually have "little computers communicating
with each other".

~~~
physicles
True, if you squint then an application will look like this too. That doesn’t
mean this paradigm is not different in other important ways.

For example, in an application you can use things like types and functions to
make explicit the coupling between different parts. You can iterate and verify
changes on the time scale of a few seconds (instead of waiting for a lambda
deployment). You can use all the tools for exploring source code we’ve
developed as an industry over the decades. And you can atomically change the
entire application simply by deploying it.

Of course serverless has its use cases. But evolving a system like this
article shows, without most of the tools and guarantees we enjoy when dealing
with code, seems like an exercise in pain.

~~~
ledauphin
in our serverless application(s) we still use types, and yes, that requires
runtime verification at more boundaries, and therefore technically more
compute cycles. we share those types across all our Lambdas. Yes, technically
each Lambda must be deployed separately; in our case any changes to message-
passing types would require equivalent "overhead" as a no-downtime database
migration - then again, the best practice here is the same as with databases -
always try to make your types backwards-compatible, only adding new fields,
etc.

there's certainly _a_ cost; I just haven't seen it be as high as people seem
to imagine.

------
nojito
All I see is a whole bunch of dollar signs floating away.

It’s quite sad how effective cloud marketing penetrated the dev community.

~~~
smitty1e
Do you think that the cost calculator does a poor job of estimating the
financial hit?

The claim is that getting out of the regular instance-and-EBS-volume
architecture avoids cost.

The question being whether "avoids cost" meant Amazon, or your business.

~~~
m0xte
In my case I see people skipping over things they don’t know when estimating
like traffic and actual resource deployment and crap like “yeah Jenkins will
be fine in a container”.

Always assumptions, missing info and costs and poor understanding of it
because it’s so damn complicated understanding cost. Consider traffic costing
across several regions and you can regularly just piss $20k away from a small
architectural decision made wrong.

Estimating costs is Actually so difficult that it has almost entirely replaced
the old role of license management under the lie that renting your shit
monthly makes that concern go away.

~~~
smitty1e
In theory, the presence of other cloud providers should help drive costs down.

"The difference between theory and practice is greater in practice than in
theory."\--Herb Sutter in C++ Users Journal some decades ago.

------
ledauphin
I continue to be astonished at how many people seem violently opposed to
serverless architectures.

Having been working in one for two years and seen very few "hard" problems
come up (and having solved the ones that did), I guess I'm just wondering how
it is that our product, with its 5 different GraphQL endpoints, roughly 20
queues, and the equivalent of 100 REST endpoints, has somehow managed to avoid
all these supposedly inevitable disasters.

~~~
musingsole
It's the same dogmatic fervor seen in defense of this or that language, this
or that pattern, this or that library. I'm sure most of the detractors have
spent a long time learning their favorite cup of tea. Serverless models are
like demanding they switch to espresso.

------
wrnr
Boy look at that architecture diagram, I'm going in the opposite direction. A
mono repo that compiles to one single binary that does everything. Sort of
like how an fertilised egg divides and each cell finds its place in the
organism. A self organising architecture, you hear'd it here first.

~~~
josho
Your approach is great for small teams that can work closely together.

As the team size scales the additional complexity of this architecture let’s
teams work More independently.

At least that’s my take on why you’d want to replace a monolith with micro
service & serverless.

~~~
verdagon
Would love to hear more about this! Are there benefits to splitting binaries
which regular architecture techniques in a single monolith cant achieve?

~~~
josho
Yes. Given schedule pressure a dev, in a monolith, can avoid good practices
and create a web of dependencies. Eg. To get this feature shipped in time I’m
not going to refactor the billing interface to expose new attributes that I
need. Instead I’ll just query the DB directly from the user profile
controller. Repeat this over a few years and a monolith can be a nightmare.

Whereas shipping as independent services it is harder to cheat because the
user profile team doesn’t have access to the billing tables and must go
through the billing API.

Of course it is possible for monolith teams to enforce the separation but no
team has because it would slow things down in the short term. Then in the long
term it’s too complex to unwrap and teams refactor using serverless because
that’s what is in style. Of course however nobody is writing about the new
problems this approach creates.

~~~
dehrmann
> Instead I’ll just query the DB directly from the user profile controller.
> Repeat this over a few years and a monolith can be a nightmare.

I've seen this, a rat's nest of module dependencies, poorly thought-out
module/library use, and a lack of ownership and direction around shared bits.

Monorepos also make dangerous things like breaking api changes look easy, or
maybe hide them altogether.

Hard problems around ownership, dependency management, and api versions are
hard in polyrepos, and I think I'm OK with this. It makes you be more
thoughtful.

------
collyw
I started to have a look but that first diagram arrows everywhere, does no one
do simple any more?

It does look like what a normal web app would do in say PHP or Django. Is
there really an advantage to splitting it into microservices and putting it in
AWS Lambda over a load balancer and extra instances when needed? A normal app
would have those services split into classes / modules and would run on the
same machine (the aysnc tasks being an exception). I imagine my approach would
need a lot less ops and coordination between the parts.

~~~
josephorjoe
In my experience, when a single engineering team owns the entire service, a
single web app >>> cluster of microservices.

The advantage to splitting up systems is when multiple teams own part of the
service and do not want to be dependent on each other for testing and
deploying and signing off on features.

Although given the amount of time I spent this week to be able to send a
string from my microservice to a partner teams' microservice and get a
different string back, I'm not feeling microservice love today...

~~~
combatentropy
Conway's Law?

------
trashburger
I read the entire article, and my first thought is... why? Why go through all
that complexity? How much money does this actually save over writing a regular
server to prompt favoring this approach? Wouldn't this have a similar cost as
a regular server if you get regular activity, minus all the complexity? What
happens when your lambdas get DDoSed, do you get overcharged?

~~~
empthought
[https://aws.amazon.com/api-gateway/faqs/](https://aws.amazon.com/api-
gateway/faqs/)

“ Q: How can I address or prevent API threats or abuse?

API Gateway supports throttling settings for each method or route in your
APIs. You can set a standard rate limit and a burst rate limit per second for
each method in your REST APIs and each route in WebSocket APIs. Further, API
Gateway automatically protects your backend systems from distributed denial-
of-service (DDoS) attacks, whether attacked with counterfeit requests (Layer
7) or SYN floods (Layer 3).”

~~~
alpha_squared
It's being attributed to API Gateway, but I think this is really just
Cloudfront doing all that.

~~~
scarface74
CloudFront is just a CDN. APIGW let’s you create usage plans based on API Keys

------
agrippanux
I wish articles like these would include sections about how their developers
dev / test / promote through different environments. Only seeing how a
production system is structured is less than half the battle and can mask true
complications of going serverless.

~~~
calpaterson
Typically they simply don't test!

If you do test, you really can only test each unit in isolation - testing the
kind of setup in the article is just too hard and the tooling for doing
automated tests on that kind of thing is not very mature. There is stuff like
localstack/samstack/etc but I haven't seen anyone really use it as part of a
workable test strategy for serverless AWS.

Consider what happens outside of serverlessland: when people have 15+
microservices each of which needs a different docker container and different
data store. Do they tend to do whole-system testing of it? No, in my
experience people tend mumble something about "contracts" and test individual
bits in isolation.

This is on top of the open secret which is: it's very, very hard to predict
what the ongoing costs of a serverless solution will be. Most of the services
are priced by usage but the units are small (seconds, minutes, etc) and the
sums are too (eg $0.000452 per hour of xyz). Post-project bill shock is a
considerable problem.

Source: worked at an AWS partner consultancy on many cloud "architectures"
similar to the article.

~~~
corporateslave5
Actually using cloud formation, it’s really easy to do integration testing.
Often times server less can be cheaper because the systems only pay for what
they use. High tps is expensive on server less, but low tps is dirt cheap.
Source is I work at aws

------
imedadel
This looks so expensive! I've been going in the same path (with centige.com,
WIP) but then I stopped and refactored some stuff:

\- I had lots of lambdas written in TS. Things were going out of hands. So I
refactored them into three lambdas only. A large one that has a switch
statement to execute the correct function, and two other ones for some quite
long-running processes.

\- The big lambda was executed too many times. Moving it to an actual server
would cost me way less. So I did. Plus I could add more security features that
would need AWS API Gateway if I chose to stick with lambdas, which is too
expensive.

\- Initially I used Amplify. And while recently its bundle size has been
reduced substantially, it's still big (see
[https://bundlephobia.com/result?p=aws-
amplify@3.0.12](https://bundlephobia.com/result?p=aws-amplify@3.0.12)). In my
case, I only needed the Auth package, which is 50kb
([https://bundlephobia.com/result?p=@aws-
amplify/auth@3.2.7](https://bundlephobia.com/result?p=@aws-
amplify/auth@3.2.7)). However, my options were quite limited and I found
myself changing things around to make my app compatible with the way AWS
Cognito worked[1] (used by Amplify Auth). I ended up removing it and handling
authentication on my own.

A better stack for a serverless app would be Vercel for deploying the app
(it's cheap, it has SSL, and it has automatic deployments) and either lambdas
or a server, depending on your needs, for handling backend stuff.

Also, use something like aws4fetch instead of the actual SDK if you can. It
will save your visitors some kilobytes :)

\---

[1]: Email/password auth with Cognito is not the best thing out there. So I
wanted to add Google Sign In. At the same time, I wanted to use DynamoDB's
row-level access (similar to Firebase's write rules, but more limited). My
plan didn't go too well, as I needed the user's Id (provided by Google) when
writing to DDB. But Cognito wasn't returning the Id... I spent a couple of
days on it, then I figured that I'm just wasting my time.

------
kmclean
That looks really complicated to me. What's the advantage of setting up a
system like that over a typical server, i.e. with most of that stuff handled
by one server, writing code to coordinate between a few external things (like
maybe auth0 and a DB)?

~~~
carlsborg
For the core functionality the only part you touch is the code in the lambdas.
Everything comes out of the box if you use a framework like Serverless or SAM.
(The step functions, batches, identity, and async pieces you have to configure
separately).

The number of OSs you patch or upgrade over the course of running this for a
few years is... zero. And the time and effort to scale up, run failover tests
on your servers, migrate to new hardware ..also zero.

But the servers are just a small part of it. The application services around
it make it possible to build enterprise grade scalable services with low ops
overhead really fast.

~~~
kmclean
That sounds really appealing. I really love the sales pitch for serverless
stuff. I think I've probably just been exposed to a lot of really advanced
ways of setting it up for a big project. It seems complex.

------
momokoko
This is what vendor lock-in looks like.

~~~
scarface74
Have you seen how many “vendors” the average decently sized company is “locked
in” to?

------
arpinum
Too much marketing, not enough content. Glossing over the dumpster fire of
Cognito is enough for me to not trust the rest of the content.

There is so much custom code to essentially fling data between services. I
imagine only a small percent of the code and dev time is spent on business
logic. These Rube Goldberg machines are my least favourite part of AWS,
especially when dealing with at-least-once delivery.

~~~
k__
I only used Cognito via Amplify and it was a rather pleasant experience.

What's wrong with it?

~~~
mcculley
I recently built a new app on AWS and Lambda. I really wanted to use Cognito
to keep everything in AWS. I fought with getting it working for a day or so
before giving up and trying out Auth0. I was up and running with Auth0 in a
couple hours. I found the Cognito documentation to be insufficient for my
needs. Others may have better experiences.

~~~
imedadel
Are you familiar with Firebase's write rules? DynamoDB has something similar
with Cognito, but it's _very_ limited.

That's when I figured that I'm not gaining anything by using Cognito.

~~~
mcculley
No, I was not aware of that Firebase functionality as I have never used it.
Interesting. Yes, it would be ideal if I could just let another layer do all
of the authentication and authorization. That would have worked for the bits
of code that use the database. I still would need a way to use the
authentication token for things like the Stripe integration.

------
jacobra2
There's a lot of negative comments, so I'll just put out - I enjoyed reading
this and appreciate the time put into explaining this.

What does your dev/sandbox environment look like? Are you using localstack?

~~~
xavierl
I just fell on the post today. I am the one who wrote the article... I'm a bit
terrified by the comments haha. But that's a way to learn and improve stuff.

Thanks for your support in the middle of this.

To answer your question: our dev environment is iso to staging and production
with AWS accounts for each developer. The cost is close to 0. Deploying and
testing a code change takes seconds. Deploying and testing a config change
however is still a little bit longer for sure...

------
mrits
If you think serverless is scalable and cost efficient wait until you try
servers.

------
deanCommie
Did any of the people commenting how complicated (and/or expensive) this looks
actually READ the box titles before they ran to proclaim how this should all
be replaced with a simple monolith on a single VM?

There is a TON Of stuff here that is not related to your application business
logic, but still has to exist SOMEWHERE for a complete production application.

Let's go through what's actually here that your single VM/Rails application
does not/cannot encapsulate:

* CDN

* Domain name and certificate management

* OAuth/identity handling

* File upload

* Workflow orchestration

* Facebook messenger/email integration

And for those complaining that this is untestable, it's...exactly the
opposite. ALL Of this is infrastructure as code declarable in a configuration
file. Which means you can spin up a beta environment as quickly as production,
or hell every developer can spin up this entire environment in their account (
_FOR FREE_ i might add - because that's the whole value proposition with
serverless which is that you don't pay for something if you don't use it)

What's inside these Lambda function is still standard code you can apply good
software engineering and SOLID principles and unit test thoroughly and in
isolation.

And if you think workflows, and asynchronous queues, and event buses are
overkill - well maybe they are for your use case. If you have 1 API invocation
per minute you don't need ANY of this.

But if you're handling any serious scale, this will scale elastically with
minimal operational intervention.

Remember - your company is not just your devs but your ops people too. And
this makes their life way easier.

I understand the frustration with the 'serverless' buzzword, but let's not
miss the forest for the trees. This shit is REVOLUTIONARY. And all people can
do is try to one-up how much they could replace it with a simple shell script.
The lack of humility in this crowd is mindboggling.

Look I agree - at both extremes (very little usage, A LOT of usage) serverless
architectures like this are either overkill or to expensive. But there is
avery fat middle (I'd venture >90%) of software projects out there for whom
this would be both the easiest and the cheapest architecture (in terms of
total cost of ownership) to build and maintain.

------
planxty
> What a typical 100% Serverless Architecture looks like in AWS

Does it look like an unmaintanable untestable mess?

~~~
nostrebored
SAM provides a pretty painless way to test locally and is core to most CI/CD
with Lambda.

~~~
Dunedan
Unless you use custom API Gateway authorizers, want to interact with other AWS
services or ...

Don't get me wrong, I absolutely love AWS SAM, but I believe testing
serverless applications locally is still an unsolved problem.

------
meritt
This article is a blueprint for not only creating a massive amount of
technical debt, but ensuring your startup's entire opex is spent on AWS.

------
thrower123
These reference architectures always look less like anything a sane person
would come up with than an opportunity to cram every possible cloud service
into a solution and jack the price of hosting a simple web app up into the
thousands of dollars a month.

~~~
dane-pgp
Actually when I saw the diagram at the top of the article I thought "Yup, that
looks almost exactly like the architecture of the last project I worked on",
as it uses all those AWS services in roughly the same arrangement.

That's not to say I disagree with your comment though...

------
storafrid
Lots of bashing here. Please provide clear examples, based on facts. I'm not
very interested in how complex you think serverless architecture seems to be.
Instead, let us know: What did you try, what was the aim, why did you choose
serverless, what was your experience, what did you learn? Make an addition to
the discussion.

I'm a solution architect in an environment where 20 teams of five developers
each are deploying API- and web-oriented code daily, and have been for a year,
on a 100 % serverless architecture. The teams are able to deliver MVPs within
days, from scratch. We don't pay for any allocated capacity, only usage. Aside
from bonuses such as being able to spin up a complete set of services as a
temporary environment, in minutes and for free, the production environment is
also incredibly cheap.

I can't say whether this stack would have been as effective if we weren't at
the size of benefitting from a microservice way of working, with independent
team responsibilities and explicitly defined ownerships. But what I can say is
that in my 12+ years in IT, I've never seen developers be this productive.
Especially considering the complete development lifecycle.

I'm sure plenty of this would have been possible using a traditional
architecture, but having 500 lambdas in production and knowing that you can go
home and not get a phone call... pretty nice. The reason being that they are
small and thus easily testable and securable, in addition to the obvious
(fully managed, auto-scaling etc).

We've had to solve issues of, course. Some silly, some unfortunate. But I
wanted to provide a counterweight to the apparently popular opinion that
serverless is complicated. To me, that's like looking at a car and complaining
that it's more complicated than a train. Both have their use cases.

~~~
ledauphin
thanks for this detailed and concrete counterweight opinion. I hear horror
stories but my experience jibes with yours.

------
k__
It's not typical (because most people don't know about it), but API Gateway
can integrate directly with many AWS services without the need to put AWS
Lambda in between.

If you find yourself writing many Lambdas just to do simple transforms and
piping around data, it's always a good idea to check if the two services can
directly talk to each other first.

~~~
0xFF0123
The mapping template language(?) is frustrating to do anything in though. Its
documentation is somewhat hidden / distributed, especially any special
variables / util functions, and suffers from limitations like the foreach
operator being limited to 1000 iterations. Unless you're literally returning
entire database objects (unprocessed, including all the type info) or
something similar, it's pretty limited.

~~~
k__
Yes, the usual AWS problems.

The docs for most services are really bad. I wasn't fond of learning VTL for
AGW either :/

------
PragmaticPulp
At a past job, we hired some people out of Amazon to work on our cloud
services. They rewrote large portions of it using AWS serverless tools.

Cost was a big concern for us. Strangely enough, the AWS experts insisted it
would be impossible to estimate the cost of the serverless architecture until
we deployed it at scale and tried it out. Is this still the case, or has it
become easier to estimate costs during development?

In our case, costs did go up significantly after the serverless rewrite, but
they also added additional functionality and complexity that made a 1:1 cost
comparison impossible.

Serverless was interesting, but if I was involved in another backend project I
would want to understand the cost better before diving into the deep end with
all of these intertwined services.

~~~
tracer4201
How long ago was this?

I used AWS for a recent personal project. Lambda was not the right approach
for me and primarily due to cost. I had an idea of how long a request takes to
process (ball park estimate) and the expected throughout (requests per
second). From that I knew that at minimum, I would be paying $x a month, and
as the execution time of my code goes up or I get a spike in my traffic
volume, it would only increase from there.

If you’re at scale, AWS Lambda blows up real quick.

~~~
scarface74
That’s why you use Lambda+APIGW with proxy integration where you can use your
standard frameworks like Express, Django, Flask, ASP.NET MVC and your can
easily move from lambda without any code changes.

~~~
tracer4201
I’m not sure what specific problem is being solved with the solution you’ve
written. Also, I wasn’t building a web application.

~~~
scarface74
You’re not “locked in” to Lambda. When you see that lambda isn’t the answer
you deploy the code to a server or Docker just by making a few changes to your
deployment pipeline.

If you use the standard APIGW solution where it routes to your various
lambdas, you would have to rewrite code.

~~~
tracer4201
Sorry but maybe you misunderstood my post. I never said anything about “lock
in”. I could easily estimate ballpark minimum costs in front. There’s
absolutely no reason to continue using Lambda at that point.

Lock in or no lock in, there’s also a cost of writing the deployment scripts
and setting up your development vs. production Lambda stack. And if you are
using Lambda, you’re going to end up making various decisions to keep your
costs low. For example, I was working with a JVM stock. Spring is too heavy,
and even Guice has some execution cost. So on Lambda, I would use Dagger... a
decision made purely because I’m operating on Lambda.

Building an architecture on Lambda requires certain decisions to be made up
front. Saying that you can just lift and shift later on is very simplistic and
will be costly depending on what you’ve already done on Lambda.

I would always say estimate your costs first and think about the long term
picture about your request rates, patterns (spikes), and growth rates.

~~~
scarface74
Setting up your lambda deployment scripts is setting your CodeUri value in CF
to match the output directory of your build and running CloudFormation package
and deploy.

Changing your deployment to use Docker/Fargate is creating your Docker
container by copying from the same output directory to your container in your
Docker file, pushing your container to ECR, and running a separate
CloudFormation deploy to deploy to Docker.

I’ve deployed to both simultaneously.

There are no decisions to be made up front. You create your standard
Node/Express (for example) service and add four lines of code. Lambda uses one
entry point and Docker or your VM uses the other.

It’s simplistic because it is simple. Java is never a good choice for Lambda.

~~~
tracer4201
>Setting up your lambda deployment scripts is setting your CodeUri value in CF
to match the output directory of your build and running CloudFormation package
and deploy.

What about writing CF to begin with for all your Lambda functions and setting
up all their permissions to work alongside whatever other resources you have?

>Changing your deployment to use Docker/Fargate is creating your Docker
container by copying from the same output directory to your container in your
Docker file, pushing your container to ECR, and running a separate
CloudFormation deploy to deploy to Docker.

Nobody said anything about Docker or Fargate. This has nothing to do with the
topic. My problems had no need for using Docker. Do you randomly pick a tool
kit and just hope it works out? I suspect you're not really working at scale
and costs aren't a concern.

>I’ve deployed to both simultaneously.

Great job. We are all proud of you I guess, but this is not how engineering
works, especially when you're building at scale.

>It’s simplistic because it is simple. Java is never a good choice for Lambda.

It's actually not simplistic. Software engineering is about being able to make
tradeoffs. Not everything is a CRUD/web application. It's funny you keep going
back to Node/Express as your examples. Have you looked into Express and what
dependencies it has, or do you just randomly pick the new framework of the
week? I don't mean this as an attack, but I'm more than slightly annoyed that
you keep referring to Node and Express, when that has nothing to do with the
problems I'm trying to solve.

The funny thing is you've written out all these lengthy posts but never
stopped to ask about whatever constraints the system has. You started with the
solution and are basically now looking at one of my constraints (JVM
ecosystem) and are saying it's not a good choice.

In the real world, we often don't have the ability to randomly swap out a
language. There's enough properly tested code that already exists or a massive
code base that entire teams are supporting. They're not going to drop that
just so they can go pick up a shiny new framework or product offering. LOL.

~~~
scarface74
_What about writing CF to begin with for all your Lambda functions and setting
up all their permissions to work alongside whatever other resources you have?_

There are no “all of your lambda functions” with the proxy integration. You
write your standard Node/Express, C#/WebAPI, Python/Flask API as you normally
would and add two or three lines of code that translates the lambda event to
the form your framework expects. I’ve seen similar proxy handlers for PHP, and
Ruby. I am sure there is one for Java.

As far as setting up your permissions, you would have to create the same roles
regardless and attach them to your EC2 instance or Fargate definition.

But as far the CF template. Here you go:

[https://github.com/awslabs/aws-serverless-
express/blob/maste...](https://github.com/awslabs/aws-serverless-
express/blob/master/examples/basic-starter/cloudformation.yaml)

Just make sure your build artifacts end up in the path specified by the
CodeUri and change your runtime to match your language. I’ve used this same
template for JS, C#, and Python.

 _Nobody said anything about Docker or Fargate. This has nothing to do with
the topic. My problems had no need for using Docker. Do you randomly pick a
tool kit and just hope it works out? I suspect you 're not really working at
scale and costs aren't a concern._

You’re using your standard framework that you would usually use. You can use
whatever deployment pipeline just as easily to deploy to EC2 with no code
changes.

 _Great job. We are all proud of you I guess, but this is not how engineering
works, especially when you 're building at scale._

Actually it is. Since you are using a standard framework choose whatever
deployment target you want....

 _Have you looked into Express and what dependencies it has, or do you just
randomly pick the new framework of the week? I don 't mean this as an attack,
but I'm more than slightly annoyed that you keep referring to Node and
Express, when that has nothing to do with the problems I'm trying to solve._

I also mentioned C# and Python. But as far as Java/Spring. Here you go.

[https://github.com/awslabs/aws-serverless-java-
container/wik...](https://github.com/awslabs/aws-serverless-java-
container/wiki/Quick-start---Spring-Boot)

 _The funny thing is you 've written out all these lengthy posts but never
stopped to ask about whatever constraints the system has. You started with the
solution and are basically now looking at one of my constraints (JVM
ecosystem) and are saying it's not a good choice._

I’ve also just posted a Java solution. The constraints of Lambda are well
known and separate from Java - 15 minute runtime, limited CPU/Memory options
and a 512MB limit of local TMP storage.

 _In the real world, we often don 't have the ability to randomly swap out a
language. There's enough properly tested code that already exists or a massive
code base that entire teams are supporting. They're not going to drop that
just so they can go pick up a shiny new framework or product offering. LOL._

My suggestion didn’t require “switching out the language”. Proxy integration
works with every supported language - including Java. It also works with
languages like PHP that are not supported via custom runtimes and third party
open source solutions.

~~~
tracer4201
This solution is entirely wrong and doesn’t work for the problem I was trying
to solve. You seem to believe every problem is a web application or a web API
of some kind. The proxy doesn’t make sense in my use case, and for the simple
reason that it’s cost prohibitive alone, I would pass.

Feel free to keep writing more paragraphs. You’re trying to solve problems
that don’t exist.

It seems like you start from solutions and hope that the problem will fit.
Lambda seems to be your hammer and you write as if everything is a nail.

Good luck!

~~~
scarface74
_It seems like you start from solutions and hope that the problem will fit.
Lambda seems to be your hammer and you write as if everything is a nail._

So lambda seems to be the solution and my “hammer” even though I mentioned
both EC2 and ECS? Is there another method of running custom software on AWS
that I’m not aware of besides those three - Docker (ECS/EKS), Lambda, and EC2?

------
nicesave
Lambda is only good in non mission critical services where you don't care
about failures or scaling.

I've re-platformed so many projects these past few years because of these
issues.

I wish more would take that to heart instead of marketing for Amazon.

~~~
musingsole
I used Lambdas as our core logic processor for 4 years. We cared about every
failure and the core motivation was it's scaling ability as our traffic was
bursty and would scale 1000x on random days.

Lambda -- Serverless in general -- is fantastic when your application has any
amount of downtime (on the order of 10 seconds). If there is even the briefest
moment your system can be turned off, you'll save dramatically over a
traditional server model.

~~~
nicesave
This is actually a decent use case for it. I should have been more nuanced in
my response.

I have seen more then one project fail when they use lamdas to serve apis.
This is because of the cold start problem, but also because lambda does have
scaling issues unless you work around them. By scaling I mean greater then
1000 tps.

All of the services performed within lambdas SLA but failed to meet the
requirements that the project had.

The solution was always to wrap whatever function it called in a traditional
app and deploy it using a contanerized solution where response times
dramatically improved and services became more reliable.

The idea behind lambda is good, but it can't beat more traditional stacks at
the moment. One could argue that most projects don't have these requirements,
and I would agree, but the marketing behind lambda doesn't make that clear.

------
larrydag
Not a system architecture guy. When the industry says "serverless" they really
mean hardware-less? It means you are renting on other people's servers?

~~~
bmm6o
Serverless means you don't manage servers. For instance, you create containers
but don't deal with the hosts.

~~~
XCSme
How do you mange cost, then? If you host your container on a DigitalOcean VPS,
you can just spin one up in a few clicks and you have a consistent,
predictable pricing. Who would be the target audience using "serverless" over
spinning up a few virtual or dedicated servers? I haven't heard of any company
that needed to instantly scale from 1k to 1m users overnight.

------
throwphoton
Any diagram like this should include an estimate of usage, and how much that
usage will cost, for every "block of architecture" displayed. Absent that,
it's impossible to tell whether it's a good idea or a bad idea.

Moreover the cloud providers should make it harder for an uncontrolled "block
of architecture" to accidentally spend too much money. The focus seems to be
on "always available", but depending on how fast it's spending your money, it
might be better if it crashed.

Expanding further, perhaps each "block of architecture" should have a separate
LLC dedicated to it to control billing liability. Incorporate your "lambda
fanout" to keep it from bankrupting your "certificate manager" when it goes
haywire, and let it go out of business separately.

~~~
coredog64
AWS includes a boatload of free calls in both Lambda and API Gateway per
account. So set your API Gateway call limit at/below the free limit and Bob’s
your uncle.

What’s harder is when your app hits an _Amazon_ limit and you have to figure
out if it’s hard or soft, how quickly your TAM can do something about it, etc.

------
somehnguy
And it only costs $8000/month to host your website, yay!

The over use of microservices and cloud computing is terrible.

------
maallooc
I'd hate to be the maintainer of that mess, good luck!

~~~
carlsborg
The entire system is described by a few cloudformation templates that you
maintain in source control. Plus the actual application code running in the
lambdas, these frameworks are super-thin.

~~~
echelon
Maintainable and reproducible, just like Gulpfiles.

~~~
XCSme
Unless something changes in gulp, gulp packages, OS, your codebase, disk write
speed or network latency...

Or was your comment sarcastic and I didn't get it? Whenever I tried to run the
same gulp tasks on a different machine, it almost never worked first-try.

------
yingw787
I wonder what the difference between managed serverless and having an EC2
instance w/ Firecracker loaded up on it would be.

I've used Lambda and I fear maintenance burdens due to the underlying
environments migrating; they'll keep the runtime for a running function for
you but if it's old I don't think you can update that without having to
upgrade runtimes.

Also you get billed in 50ms increments at minimum, and so beyond that there's
little incentive to make a function faster / a lot of incentive to stuff more
logic into a function and make it less composable.

I think I'd rather define an AMI w/ Firecracker using Packer, and stand up a
box with Terraform, and write logs to CloudWatch, and have a much more
predictable billing cycle / greater control.

~~~
asguy
Rolling your own firecracker solution would put the control in your hands, but
then aren't you limited to .metal EC2 instances (with all the cost increases
that come with it)?

~~~
yingw787
I don't get why you have to go with .metal EC2 instances only, can you
elaborate?

~~~
asguy
Firecracker uses KVM to implement virtualization to launch containers inside
"micro" VMs. As far as I know, Amazon only gives you virtualization extension
access on bare metal instances.

------
nihil75
I have serverless functions that I've written a few years ago and forgot
about. They are still working when needed, costing me nothing when not in use.
How much does your server cost? Is it patched? secured? Does it have a name?

------
joshuanapoli
In my startup, we began with a monolithic back-end where we could refine our
product and prove market fit. At first, this was indeed fast to develop and
simple.

The company grew and the product saw heavier use. The original monolithic
server became slow and problematic. Hosting cost became very high. Separate
teams ran into conflicts while working on the server.

So we have incrementally switched to an FaaS-centric architecture, similar to
the one in the article. The new design makes it easier to get excellent
stability, performance, cost and observability. Now that the development team
is a little bigger, FaaS/microservice design makes it easier for the teams to
work without blocking each other. I’m sure that same benefits could be
achieved in a traditional monolithic server with careful design and control of
the development process, but it seems easier and quicker to accomplish with
FaaS.

Indeed, a simple REST API sometimes requires dozens of handlers. One might
worry that this is complicated because each handler is somehow running in a
separate container... but that’s really the cloud provider’s problem.
Execution is very finely segmented, while logical division of the code-base is
generally free to follow natural domain divisions. It’s pure benefit to us:
endpoints have excellent isolation. We have excellent control for behavior in
overload scenarios.

FaaS handlers tend to be simpler than each action in the former monolithic
server. With simplicity, it’s easier for us to ensure handlers follow a common
set of best practices. With simplicity, consistency, and fine-grained
isolation it is easier to reason about the performance and capacity of the
system. While there are scenarios where FaaS is prohibitively expensive, our
use stays within its sweet spot and is significantly less expensive than
running the previous monolithic server.

A downside of FaaS is that it quickly became impossible to run the entire
system on a developer laptop. We really need to run things on the cloud
infrastructure for any kind of integration test. So far, it has been hard for
us to provide developers or test infrastructure with independent Integration
test sandboxes. Vendor lock-in is a problem: it will be hard for us to change
cloud providers if ours becomes problematic.

~~~
staticassertion
I build a FaaS based project and it runs locally using docker-compose. When
running locally it just switches to a persistent mode. It's not nearly as
large as yours I suspect, but I'm wondering what the problem you're running
into is?

~~~
joshuanapoli
Maybe we have too much reliance on proprietary technologies of our cloud host.
More likely our problem has to do with the order in which we learned to build
certain things relative to the growth of the organization. Or maybe we just
haven’t tried hard enough to set up a local build of our system.

~~~
staticassertion
Ah, fair enough I could see that. We do use Dynamo, S3, and SQS, but there are
local versions of all of those.

------
jorblumesea
At what point do you just run a "normal" application instead of this giant
spaghetti mess? Is this really cheaper compared to ecs or eks? What do you get
out of this?

This can easily be 1-2 very simple services.

------
cs02rm0
When your business API is one get data and one post data I find 100%
serverless passable.

I'm starting to accrue a list of projects that started out looking like that,
with product owners swearing blind that was the sum total of all they could
want, but ended up becoming much more complex. Then you get into the pain of
sequential lambda cold starts, etc and before you know it everyone thinks
you're in too deep to turn around and the pain just won't go away.

Just one more AWS service and all our problems will be solved...

------
gfodor
At best, our children will look back at this and laugh. Most likely they won’t
though because this whole pile of garbage won’t exist anymore and will be long
forgotten.

------
bcrosby95
> and their pay-per-use pricing model

Sure, if you want to inflate developer costs. Isn't a huge portion of the
cloud argument that people are expensive, infra is cheap so who cares if you
spend a lot on AWS as long as you have fewer sysadmins? If you suddenly care
how much you're spending on infrastructure but, apparently, don't care how
much you spend on developers to work within such a convoluted system, why not
exit the cloud at that point?

~~~
scarface74
Everything is convoluted if you don’t know what you are doing.For instance, I
find the entire modern front end development ecosystem a royal clusterf*&$ but
people are using it effectively everyday.

------
black_puppydog
I have to say I saw that diagram first and started reading with the
understanding that this was satire. Quite shocked this seems to be for real.

At this point, it might be feasible to reclaim "serverless" as the domain of
p2p systems, as some others here have suggested. The non-p2p folks have
clearly given up on the "it's too complex to reason about" line of argument,
so now might be the time. :)

------
corporateslave5
This article is click bait for everyone used to mono repos and traditional
software architecture. It gives them a place to rip on new technologies, much
like react and the trope about how many js frameworks there are. The truth is
software like this is the future, no amount of get off my lawn complaining
will change that

~~~
XCSme
Can you elaborate why you consider this to be the future? For almost all
software projects which are not at Google-scale you can always get away with
just hosting a simple app on a few VPS or dedicated servers and be able to
handle millions of monthly users.

------
Ididntdothis
I always wonder how well these systems will do once they become legacy. It
seems to that if it's difficult to maintain old COBOL apps then maintaining
such a complex architecture a few decades later will be a lot of fun.

------
aabbcc1241
How does the bills looks like for typical / best practice serverless
application ?

------
davewritescode
One of the biggest problems with serverless on AWS with cloud formation is how
ridiculously slow your deployments get. The other problem is long tail
latencies on lambda invocations, it gets worse if you want to run services
inside a VPC

Serverless isn’t going anywhere until we get something like kubernetes for
functions.

------
meddlepal
All I see is lots of brittleness and potential failure points.

~~~
scarface74
Details?

~~~
XCSme
It is known that the more moving parts a system has, the more likely it is for
one of them to fail, thus for the entire system to malfunction.

~~~
scarface74
So do you have any practical examples from actually implementing anything or
is this just all theory?

~~~
XCSme
There were several articles regarding microservices recently, for example:
[https://www.infoq.com/news/2020/04/microservices-back-
again/](https://www.infoq.com/news/2020/04/microservices-back-again/)

Also, this "simple is better" theory has been proven to be true again and
again in a vast range of disciplines and use cases.

~~~
scarface74
Because it’s an anecdote on the internet it must be true.

I can give you just an opposite anecdote - and from personal experience - not
from what I found on the internet.

We have a lot of microservices that are used by our relatively low volume
website, but also used by our sporadic large batch jobs that ingest files from
our clients and our APIs are sold to our large B2B clients for their high
traffic websites and mobile apps.

When we get a new client wanting to use one of our microservices, usage can
spike noticeably. We have deployed our APIs to both Lambda for our batch jobs
so we can “scale down to 0” and scale up like crazy but latency is not a
concern and we host them individually on Fargate (Serverless Docker). Would
you suggest that it would be architecturally better to have a monolith where
we couldn’t scale and release changes granularly per API?

~~~
meddlepal
You constructed a straw man and then attacked it. There are some places where
an architecture as you described makes a lot of sense and is probably the best
solution. However, it does not change the reality that this type of
architecture is often brittle and hard to debug because the experience needed
to actually operate it is not present within the company that adopts it.
Similarly the lego-block nature of these cloud components leads to further
quickly glomming on of additional pieces without understanding the overall
impact those pieces will have on the existing system.

I've seen it time and time again. Anecdotal of course, but we're all just
shooting off anecdotes on HN anyways.

~~~
scarface74
So I constructed a straw man when the original post was a very hand wavy _It
is known that the more moving parts a system has, the more likely it is for
one of them to fail, thus for the entire system to malfunction._

But how pray tail did I “construct a strawman with a one word reply” -
“details?”

And then when I asked for _personal_ experience a random article on the
internet was found. At least I was able to speak from personal experience. I
can also go into great detail about best practices as far as deployments,
logging, security, troubleshooting, testing (functional and scaling) and
tracing without citing random articles....

------
jessaustin
TFA needs more exclamation points!

------
yourapostasy
The premise cloud has been sold to business stakeholders under has been
dramatically reduced cost mainly through not having to staff up for a number
of technical positions, and it works for certain use cases. But I'm not seeing
real payoffs for many less-tactical, broad-scale use cases.

What appears to happen is the "get going coding" part is dramatically
accelerated, that saves on the order of days of waiting; a few weeks/months if
new tech stacks the organization hasn't worked with before are in play. But
man, once in production, I'm still puzzling over how to manage the operational
costs.

There is a huge disconnect in the business community in their perception of
cloud that I'm collaborating with. They're taking the delivered "get coding
fast" benefits which do save time and money. They see we're only paying for
infrastructure the moment devs start hammering fingers to keyboards so to
speak, and not waiting around for teams to stand up that infrastructure and
paying for all of that while it's not delivering benefit. They take that
experience and are applying the same expectations to the operational,
production support side.

I don't know how others are doing it delivering on the operational and
production support benefits, but nowhere near the same scale of savings are
happening so far for teams I work with. What I'm experiencing is while there
are marginal savings on the nodes of services (the actual service itself),
we're adding a lot to staff payroll at high-end pay scales and spending a lot
of their time on all the connections and complexity between them; between
troubleshooting and adding new capabilities (more troubleshooting), it is
eating up a lot of expensive staff time.

I'm pushing for greater automation through DevOps, but that's encountering
lots of resistance now with the economic outlook because DevOps generates a
net increase in absolute payroll even with a marginal decrease in operational
staffing once the automation is burned-in. In fact, because automation takes
away the mundane operational aspects, we end up having to cost-justify far
more capable and expensive operational staff that can handle the corner-cases
that emerge as business-as-usual activity, after automation ate away all the
usual cases. While headcount goes down, total payroll goes up, and budget-
watchers resist the idea that they're saving overall by delivering more
stable, consistent, quality service with more feature points per dollar, and
if we continue to carry on the old way our costs would be easily triple to
order of magnitude more. This is a marketing/education perception problem to
work upon nearly daily.

I love working with the cloud tech stack, but at the end of the day I have to
show my business stakeholders it really did save net budget. For judiciously-
targeted and considered projects where I've had a lot of input tactically it
has worked really well for them and me. For strategic-level "Cloud All The
Things!!@!%!" initiatives, the cost savings get way murkier to discern; this
is the perception problem on steroids. I'd love links to detailed readings
from people who have been in the trenches of successful strategic-scale "we
went to the cloud whole hog" efforts, describing the traps and pitfalls to
avoid and how they showed a _net_ benefit.

