
Going Serverless: From Common Lisp and CGI to AWS Lambda and API Gateway - lispm
https://medium.com/@glikson/going-serverless-from-common-lisp-and-cgi-to-aws-lambda-and-api-gateway-9fba46c84fb8
======
shaunpersad
I don't get why Lambda gets all the love while Google Cloud Functions doesn't.

I've tried both, and GCF is vastly superior in many ways.

1) Developer experience is pretty terrible in Lambda, whereas GCF is
compatible with Express.js to the point where you can simply pass in an
Express app instance to it (which makes vendor lock-in a moot point as well).
Develop your app more or less like you normally would, and just export it to
GCF. Job done.

2) Deploying is a simple cli command away, even with native binaries.

3) No API Gateway necessary for HTTP functions.

4) It's now compatible with Cloud SQL so you don't have to give up your
traditional database.

5) No crazy IAM permissions hoops to jump through to get anything to work.

Disclaimer: I'm not affiliated with Google.

~~~
giancarlostoro
I feel the same about Azure Functions but I think its all dependant upon which
provider you're in bed with at the time. If some one has plenty on AWS and is
happy with those resources on AWS they will likely try or use Lambda

~~~
ryanmarsh
I'll respond to you and GP. Lambda is more challenging to use than Azure
Functions and GCF. I should know, I use Lambda every day. To be honest the
overall developer experience with almost any tool on AWS is more painful
compared to Google Cloud or Azure.

I think the reason AWS gets all the love is, GCF only came out of beta very
recently. Azure doesn't generally get as much press as AWS or Google Cloud.

As a secondary issue, "why use something with a poorer developer experience"?
My answer: AWS's tools are quite complicated but it's also clear to me that
Amazon takes AWS very seriously. My interactions with Google Cloud's products,
support, and account management have left me with concerns about their long
term commitment. This is a personal gut feel. I trust Amazon and Bezos'
commitment to AWS. Focus and commitment are things we're all aware Google has
had issues with in the past. As far as Microsoft is concerned, I see the great
strides they're making and I would consider Azure for future projects. I'm
growing to trust Nadella's Microsoft. It's clear they're betting the farm on
multi-platform and cloud. Time will tell. Again this is just a personal
viewpoint.

Lastly, once you climb the hill of learning IAM, CloudFormation, etc... you
can be quite productive.

------
threeseed
I will never understand why people are attracted to the Lambda SAM style of
programming. Sure there are scalability benefits but it comes at the cost of
complexity, development time, vendor lock in, wildly variable p95/p99 numbers,
a nightmare debugging experience and costs that are expensive and fluctuating.

You could just build a Docker web app eg Rails/Play, deployed in ECS or
Elastic Beanstalk with the same scalability in easily 1/10 the time and
complexity. And if you get sick of AWS trivially move it to GCP, Azure or on-
premise.

~~~
Touche
I don't use SAM but I do use Lambda and I disagree with everything you say
here. FaaS allows me to completely eliminate web server junk from my brain. I
only have to think about individual functions (routes) that takes some input
and produce an output. Essentially I can focus completely on my business
domain.

~~~
tuyguntn
without fine-tuning. How do you handle database connection pooling? yes,
lambda can scale as a function, but not as a wrapper around database (yet)

~~~
djstein
I think the answer to this right now is either: \- DynamoDB \- Aurora In my
case I am going to go with Aurora as I am deploying Django applications with
Lambda

------
bsbechtel
Has anyone heard of implementing an architecture where you have normal server
backend architecture, while using serverless to handle traffic spikes and any
failover? If you properly modularize your backend codebase, it shouldn't be
excessively costly to duplicate it across a serverless architecture and in
some sort of node instance.

My biggest issue with serverless is the high cost at scale, which could be
mitigated with this type of architecture. Are there any other benefits or
concerns regarding something like this?

~~~
toomuchtodo
Sounds more expensive than scaling up more servers on demand.

~~~
brazzledazzle
What I hear a lot is that the scaling can take too long. I wonder if a hybrid
where you use lambda as a temporary buffer would be the most cost effective.

~~~
pageald
In my experience using AWS, you can spin up new autoscaled instances pretty
quickly. I don't even have a custom AMI, I use a generic image and run startup
scripts to kick off my server software. Takes < 3 minutes to spin up a new one
and start accepting requests.

If that's not fast enough, you can bake an AMI and scale more proactively
(e.g. scale when your server is 70% utilized rather than 95%)

------
vrodic
Most of these articles I read about AWS Lambda (they rarely mention API
Gatweway because it's expensive) sound like paid marketing.

I've seen just API Gateway costing more than entire infrastructure costs of
similarly sized websites.

If you can properly saturate EC2 it will be significantly less expensive than
Lambda but with lambda you have to pay API GW and the vendor lock-in price.

I've started flagging these submissions.

~~~
nothrabannosir
That calculation doesn’t include the salary of the person managing those EC2
instances. Lambda is easier to manage and costs less time than an EC2 box. SSH
keys, backups, AMI base images, system updates , it all adds up. Ansible and
terraform are so far removed from the core competency of any company that you
really need to ask yourself, for any second you spend working on them: is this
worth it? At almost no company’s scale* is it so.

Salary is a hidden cost people often forget. Or time spent by your
programmer’s debugging infra; time not spent creating value. Frustration from
having to deal with TF or ansible in the first place.

These are all benefits of a serverless solution which cost aware criticisms
should at least try to quantify and take into account.

Signed, not-a-shill.

* I’ll rephrase: a significant amount of companies don’t have the luxury of being so swamped by requests they can saturate EC2 boxes and make the devops overhead (which, I’m arguing, is big but _relatively_ constant/“sub linear”) worth the price difference (which scales linearly). Significant enough for these articles to have a raison d’être.

~~~
Zelphyr
In my experience, the developer time it takes to get a serverless application
running (on AWS anyway) far exceeds the benefits that serverless brings. We
spent over six months on a project written in serverless that we could never
really get working properly. We dumped all that code and rewrote it on a
standard LAMP stack in less than half that time. Factoring in that the problem
was better understood the second time around, it would still have been cheaper
to build it the way we are from the beginning.

~~~
scarface74
Was that due more to lack of training? I am first and foremost a developer,
but I’ve spent weeks understanding AWS from a netops, devops and development
standpoint.

I’m also working on certifications mostly as a method to force me to learn in
a structure manner, my company will pay for them, and they still have some
market value.

~~~
romaniv
Training also takes time.

~~~
scarface74
Are you proposing that developers shouldn’t study and learn about whatever
infrastructure they are using?

~~~
romaniv
I'm not proposing anything. I'm saying that training should be factored in
when considering development costs. The notion that training is always a one-
time investment that can be pretty much ignored if you plan to stick with some
tech long-term is a fallacy.

~~~
scarface74
Things get added to AWS all of the time, but if are using the same set of
services, training is a one time thing. Amazon doesn’t just pull the rug out
from under you.

------
sheeshkebab
Serverless is a latest heavily marketed fad.

There are some edge cases where it’s useful - education, low use throw away
bots - other than that it’s all marketing bs.

~~~
pavolt
> Serverless is a latest heavily marketed fad.

Serverless is the future of the Industry.

Lambda is not Serverless , it's just a FaaS which is a component of an entire
Serverless Architecture.

Most specifically the majority of services provided by AWS and in the Cloud
already are Serverless :

\- Pay per usage ( GB , Execution , API Call )

\- No Scaling or Provisioning ( SQS , CloudFront , Firebase )

\- High Avaibility by default ( SQS , CloudFront , Firebase )

\- Auto Scaling by default ( Azure CosmosDB , Firebase )

\- Little to no technical limits for the service ( GCloud DataStore , S3
etc... )

If you have very little knowledge about something you shouldn't criticize it ,
that make you sounds even more ignorant.

Most of the stuff that people actually use on AWS (S3 , CloudFront , SQS , SNS
) are delivered with a serverless architecture in mind.

Again , if you are thinking "Lambda === Serverless" you really need to read
more about this topic.

~~~
zdragnar
Wait, we're calling S3, Firebase and any sort of metered charge scheme (aka
pay per usage) "serverless" now? That's news to me, I've apparently been using
serverless since I started programming as a full-time career 10 years ago!

Here I thought most of those were just "cloud" services or "third party APIs".

~~~
pavolt
> I've apparently been using serverless since I started programming as a full-
> time career 10 years ago!

You have indeed.

As I mentioned above , S3 is a serverless service.

Last time I checked , you don't provisioned , partition or scale S3 this is
done by AWS behind the scene.

Same for Firebase , same for many services.

Now the move is into the "compute" aspect of thing and the architecture.

Lambda basically replace, in a limited way , the compute aspect of your
architecture (EC2) but obviously it has it's limits.

There is a confusion between "Serverless Framework" which is an automation of
Lambda + API Gateway + CloudFormation , and "Serverless Computing" which is
basically an "event driven" , "pay per execution" , "auto scaling / self
healing" architecture.

~~~
zdragnar
I probably should have added a `/s` somewhere in my post; my internal sense of
"this doesn't sound right" is coming from a desire to enforce a distinction
between "cloud", "API" and "serverless".

To my mind, serverless is exclusively limited to FaaS or similar means of
_deploying code without visibility of the underlying server it runs on_. S3
isn't about code deployment, it's a third-party API that stores data. EC2,
docker et. al. at least have instances of servers you see, even if you
automate away the management portion- that makes them cloudy, but not
serverless.

Maybe my understanding of serverless is completely at odds with what the
industry is, and if so I have learned something new and am chuffed.

~~~
yebyen
> deploying code without visibility of the underlying server it runs on

Well I take issue with this, I think Kubeless and Riff are definitely
serverless, even though I need a Kubernetes cluster provisioned (and it
obviously has servers in it). I can get visibility into those servers, or I
can run them in a managed way where there will be no visibility.

Both ways are serverless, I think this because they are FaaS solutions and
because of the way the runtime environment is dynamically provisioned. I don't
have to know or care what server it's on. I'm not sure that's what makes it
"serverless" but there is definitely room for questions between "serverless"
and "Serverless" with a capital S. And in at least one of those (Riff) it's
actually cold/pods all go offline at rest, so not warm processes running on
any server until it is called (scale to zero.) You just have an API gateway
that is always online, and it starts the workers in response to events. It's
hopefully lightweight compared to your app / suite of microservices / the
kubeless version of what I'm describing, where each function has a memory
footprint that doesn't evaporate when it's not in use.

Regardless of whether you're using FaaS or not, the pricing model for servers
as scale grows seems to be quite favorable compared to same size "all-lambda"
solutions, and based on the feedback in this thread and prior ones, I think
there's a compelling benefit to price predictability if you take the time to
estimate how big your computers need to be and, say, order enough reserved
instances to cover something like the capacity you will need for the year.

You can always scale past that to grow when needed, but reserved instances are
cheaper than ordinary instances, and ordinary instances are cheaper than an
equivalent amount of CPU time on Lambda. I get how this kind of negates the
benefits, but not really. It depends entirely on the pricing model and how
much you use. If you can fit in the free tier, it's much less compelling.

But you can start with a really small cluster if you can scale your worker
deployments' pods all the way to zero without causing a downtime event. (But
if you're now groaning because I've got you managing a cluster of nodes again,
then your priorities might be different than mine too...)

------
ForHackernews
> Moreover, given the minimal traffic this web tutorial is targeting — the
> backend is likely to easily fits the free tier of cloud providers. So, the
> infrastructure cost will be essentially zero. And you can’t beat zero cost,
> really.

Right up until it hits the frontpage of HN...

~~~
tazjin
HN frontpage traffic is not as high as you might think ... the websites that
go down from it are probably just poorly built.

~~~
ForHackernews
I don't think this setup will go down, but it could certainly end up costing
much more than $0.

~~~
viraptor
The free tier is 1M requests. If they're very lightweight, it should easily
handle the spike without extra cost. I've been on HN front-page a few times
and the traffic is never that high.

------
acejam
At my company we have a few small basic API's now running on Lambda and API
Gateway. We're a big AWS shop and I can't envision us ever moving to Google or
Azure, so vendor lock-in isn't much of a concern. At first local development
was a bit challenging, but frameworks like Serverless help with that.

Also, many don't know this, but you can run and debug AWS Lambda locally with
AWS' own tooling. Using this to build out and test Lambda's before creating
them with Terraform has been priceless: [https://github.com/awslabs/aws-sam-
cli](https://github.com/awslabs/aws-sam-cli)

------
cubano
So this is question I've wanted to ask for awhile.

Is AWS Lambda _really_ "serverless", or is the server part just buried so deep
in the stack that the app devs can disregard it?

And if it is truly "serverless", what mechanism is being used to "serve" the
function calls? How is this radically different from say a REST-api that auto-
routes function names from url POSTs and GETs?

I suppose I could look all this stuff up, but to be honest having a quick
rundown of this tech stack here on HN sure would save me the time.

Thanks in advance.

~~~
pjc50
Obviously it does have servers and is quite similar to the router you
describe. It's also similar to the old CGI and FastCGI systems.

It's only serverless in the sense that you don't have to use servers as a
_unit of account_ ; at no point do you have to say "how many servers are we
paying for?" or "what sort of servers should we get/lease/instantiate on EC2".

~~~
elboru
Rarely the meaning of a technology name is "obvious", this particular name is
pretty misleading (every single tutorial about the topic starts explaining
that serverless doesn't really mean serverless). The first time I heard about
serverless, I imagined some kind of decentralized technology, but as we all
know, it turned out to be a marketing thing.

------
pmoriarty
It's a pity that Lambda doesn't directly support Lisp or Scheme.

~~~
mark_l_watson
I think that you can create standalone executables and load those with lambda.
I think they need to be statically linked.

------
code_sloth
Can someone remind me what serverless does better than existing tech?

