I went to a webdev convention, and it ended up being a serverless hype train. Industry experts with a financial incentive to promote serverless went on stage and told me they can't debug their code, or run it on their machine. They showed me comically large system diagrams for very simple use cases, then spent an hour explaining how to do not-quite-ACID transactions. Oh yeah and you can only use 3-5 languages, and each import statement has a dollar amount tied to it.
More importantly, all those skills you develop are tied to Amazon, or some other giant. All the code you write is at their mercy. Any problem you have depends on their support.
Am I supposed to bet hundreds or thousands of man hours on that?
AWS in particular seem to have a carefully refined technical sales/certification/advocacy channel whose main product is those fucking stupid architecture diagrams. Hello world service with $4000/mo. worth of geo-replicated backing databases, CloudWatch alarms, API Gateway instances, WAF etc.
But don't let it encourage you to think serverless has no value, or it can't be done portably or cheaply. It has its sweet spots just like everything else.
Reminds of something that was on the HN frontpage some month ago, where readers are not sure if it's a parody or not, because of the architecture you're required to deploy yourself to use this new "Perspective" product. Direct link to the architecture, that in the end serves the use case of generating a diagram of your AWS resources: https://d1.awsstatic.com/Solutions/Solutions%20Category%20Te...
Reading this made me remember that back in the day the AWS selling point was "here you can create virtual machines with few clicks and have it instantly instead of waiting 30 min for you colocated server to be ready" but now it seems to be "here is a bunch of random expensive tools, please, produce as much stuff as possible and share the word that having servers is bad™".
This field used to be inspiring, but now I see the ideia of having a server being sold as the plague and lots of negativity towards people who are good at servers. They are not seen as another human being but the "other".
Also I can't understand why one would prefer to pay that much for such complexity.
It seems unsustainable for me, not to mention the new generation being spoon fed that that is way to go makes me concerned about the future of open computing.
"Also I can't understand why one would prefer to pay that much for such complexity."
If there is one thing that dealing with AWS reps has taught me, it's that this was 100% by customers. I swear to god, AWS doesn't do anything without customers asking for it.
If you are wondering why products are build in AWS, it's because people wanted to give them money for this. Say what you want, but this isn't something are pushing on us. This is something "we" push on them to provide.
Your response made me smile. Glad to see there are still reasonable folks out there.
Working for a startup without a "resume-driven-development" CTO gave me the freedom to go "servermore" architecture with max flexibility.
For all the talk about resume driven development, I am hiring now, and will say that people who have solved their problems by learning more about Linux and such sound way more impressive than those who list out passing familiarity with a bunch of high level services. The first style of resume really stands out. The second is a dime a dozen.
To make an analogy, it's like hiring a carpenter based on the number of tools they have. NO. Show me your skill with a hammer and chisel, and I'll assume you can figure out the rest.
> not to mention the new generation being spoon fed that that is way to go
The new generation is always joining one cargo cult or another, that's why competent technical leadership is important. Remember when noSQL was the best thing since sliced bread?
Serverless can be a good option if you have large and unpredictable transient loads.
Like any architectural choice, you need to consider the tradeoffs and suitability for your use case. A TODO app probably doesn't need to be built with a serverless SOA.
It reminds me of https://www.frankmcsherry.org/graph/scalability/cost/2015/01... - you pay an extremely high cost upfront without even noticing - for the promise that at some point, you'll be able to scale out to any scale, with no extra human effort and for costs proportional to the scale.
I have not observed enough uses of serverless I can draw conclusions from, but if it's anywhere like hadoop style scaling, 95% of the users will pay 10x-100x on every metric without ever actually deriving any benefit compared to a non-distributed reasonably defined system, and 5% will actually benefit -- but everyone would want to put it on their resume and buzzword-bingo cards.
It depends. Give you an example: We have an expensive reporting workflow that needs considerable resources, but usually only once a month. Two ways to do this cost effectively: Scale down containers to zero, meaning several minutes of response time if you do need it ad-hoc sometime. At least as complex to configure. OR you fit it to AWS SAM. The latter has proven to be a good match for us. I solved the debugging story by keeping lambda functions AWS agnostic and wrapping them i a Flask debug server - will respond exactly the same but it is all local in a single process.
I’m the head of product for Temporal. You should check us out, have to self host (MIT) right now but we are working on a cloud. You can safely write code that blocks for years. No more job queues.
I suppose you mean temporal.io, but just FYI, the first google result for me was https://github.com/RTradeLtd/Temporal which is far but not far enough to be “obviously unrelated”.
You might want to give a link next time to avoid ambiguity.
Every server less design I've seen turns into a complex workflow of distributed functions chained together by a pile of xml.
Each example that I've seen could have been replaced with a simple single process job. You should not need to go over the network because you need a process to run for more than 10 minutes. You should be able to check the status of resource creation via for loop.
An argument can be made for not needing a server, but I can easily fire off a container in fargate/gks/ecs and get similar benefit.
Our approach is to start with a mildly-distributed (among threads within the process) monolith based on future passing, and my impression is that any latency-sensitive operation in a session cluster (that is, all of the sessions who are interacting directly with the same working set of processes and aggregates) can and should be supported by a single node. Rebalancing involves moving the cached aggregate (any blob, like a sqlite database or a JSON file) and/or† catching up the log (sequence of blobs) on the target node.
Futures are basically trivial to serialize, so the cost of involving a second node is as little as it could be; and node consolidation can be generalized because each node's dispatcher knows how many times it has accessed a remote process, which means that consensus on who should lead its log can be reached by simply sending it to the node that has the most use for it (with load balancing achieved by nodes simply not bidding for logs when their load factor or total service time is over some threshold, or say the 60th percentile of all the nodes, or whichever of those two is higher).
† In some cases it's faster to just catch up the log, rather than sending the aggregate.
Thank you, serverless is the new structure that everyone loves and wants to work in, but no one talks about the downside. I just took over a project that has a serverless architecture.
We can't run it locally because the emulator won't run 7/10 of the code, there are memory limit errors that are totally opaque to us since we can't step through and watch it break. We are getting unexplained timing issues that were being worked on via logs in dev. The costs are insane.
If we want to say serverless is the future, the future isn't here yet. There are many tools that would need to be built to make serverless viable, and the tools that are built are immature and bad. Not to mention, you're tied to the companies support, so Google was outdated from the LTS for Node for a while. The reality is going serverless means you don't have control over what happens to your code.
I believe one of the main reasons Serverless will never be more than a low-tier niche is the combination of the following:
In the end, you are just renting well maintained server farms (well, a specific percentage of operational time of some of the servers in them). There is absolutely no appeal for large technology-based companies do this once they can (the following is the lowest scale example) afford to maintain their own servers, while potentially renting a few offsite backups in other areas of the world (again, this is just the lowest-scale architecture starting at which Serverless is always the worse option).
Existing solutions are extremely over-engineered. It can be excused with "officially planning for the use-case where maximum scalability is required", but it's almost certainly a pretense to sell "certifications", aka "explaining our own convoluted badly documented mess".
What this actually means is that many SWE's who are good enough to learn to use Serverless effectively, can learn any other framework that allows building distributed systems across server nodes with equal effort. Why would I base my whole business on your vendor locked, sub-optimized dumpster when I can do the same on an infinitely scalable VM networks, that can be ported to literally any vendor who supports $5/mo VMs (or, you know, self hosted if my company is large enough)?
the flip side to your statement:smaller companies (like where I am) are not interested in maintaining our own servers and do all the un-differentiated heavy lifting related to maintaining servers. I want my small team to work on things that make us stand out. Running our own servers does not.
Vendor lock-in is not a big consideration for us. I increasingly think about cloud vendors like Operating systems. I really dont care which Linux distro we are on. Pick a vendor and run 'natively' on it to go as fast as possible.
I'm happy paying AWS for maintaining the servers and getting out of that messy business. There are other concerns about serverless around observability and managability, but vendor lockin and cost is not part of my equation
I view the serverless, specifically aws lambda, as nice way to hook into the aws system and add some code to handle certain events. Basically, to customize the behavior of "the cloud" where aws falls short (for whatever reason).
But developing with "serverless" as base stack? Nope.
I bought into hosting stuff on AWS, specifically on EBS which I assume is meant by a 'serverless' infrastructure, but I don't know crap about it.
Still think it is mostly awesome and the services are solid. But such infrastructure comes with its own caveats.
EBS now bugs me about some 'HealthCheckAuthEnabled'. I don't know what it wants, just that I have limited time to react. Cannot understand the clearly auto-translated text. Still didn't get what it wants from me when I read it in English. I banned the health check from my logs and said where it can get a response from the service I run. I hoped our friendship ended there. Maybe it did and this is some form of retaliation.
The load balancer AWS set up for me now suddenly costs money. And I thought I could just use it to grab the free TLS cert that comes with it... well, the clients pay for the additional costs anyway and I would be surprised if anyone even noticed the price spike. The traffic is minimal but I am surprised it is actually that expensive. I could probably cut the costs in half if I actually had motivation to look through all the settings...
Microsoft just cancelled some features of some online service my colleague works with because they want to market an alternative for their BI solution. Did cost him 3 weeks of work at the minimum. The stuff cannot be ported to an on-premise version.
Hosting your own server is a lot of work and isn't fun. There is still a lot of work to do if you host on something on such an infrastructure.
Somehow the amount of work you have to put into software hasn't decreased. Honestly the main argument for it is that you can give away responsibilities.
edit: Just noticed that EBS probably counts as PaaS instead of serverless. I run "stuff" (very simple functions) on Lambda too, but to think it would replace apps like the article suggested seems a bit much. It has its purpose, but nearly everything I have on Lambda is AWS cloud specific. It comes in handy for Alexa integrations for example. I think the container won this fight to be honest.
This could have been written by me. And for who ever doesn't have time to maintain roles, there is Ansible Galaxy or tons of roles in GitHub, including mine: https://github.com/liv-io
I've looked an Ansible a couple of times, but I always think it looks too verbose, but more than that the way files for the same thing are spread out across multiple folders just makes it seem like there is a steep learning curve. Something like Ansible sounds great, but it feels like learning it would be a chore.
Indeed.
Kind of reminds me of that sayin' about how "the greatest trick the devil ever pulled was to convince the world he didn't exist ..."
Sometimes I think it is a great trick some big players are pulling off, in convincing the world to subsidize their infrastructure costs (and more than that) by using their resources because it is "simpler" or more effective or alternatives are hard, etc ...
... to where the ability to competently setup and maintain those services will vanish, in due time.
(Which, of course would only help to consolidate those -already- running such systems ...)
It's really not that bad. Most of the investment is early on, after that servers just run. I use unattended updates to keep patches coming in while I'm doing other stuff. Updating from a Debian stable repo won't break your stuff.
Now, whether or not you, personally, do, is entirely up to you and what you want your career trajectory to go. (Both are equally valid because internally, AWS will always needs ops people that understand computers.) However, a cottage industry of AWS Certified Solutions Architects has already sprung up, and some have been getting lucrative consulting gigs based on knowing the mismash of Amazon names and how best to mash them up.
Serverless (specifically AWS Lambda) plays a part in the cloud, but most of your objections are true of the cloud and/or AWS more broadly, and have little to do with, specifically, serverless. (Despite whatever the webdev convention was trying to sell you.) There's a whole world of EC2 instances which are mostly VPSes, to know about that definitively aren't serverless (though there are some really neat cattle-level features that serverless learns from)
If the computer is a bicycle for the mind, then the cloud is a car for the programmer's mind. You can definitely build your own stack with your own team that manages to avoids the cloud, but at some point it becomes more than one person can handle. As you scale, there's a points where the cloud does, and does not make sense, but the cloud gets you access to far more resources as a tiny team than would otherwise be available to you.
Serverless (as far as I can understand) is what we can consider as the early stages of the commoditization of compute.
Meaning, similar to electricity or tap water, at some point in our lives we will subscribe to a provider and then just plug and play whatever we want to. It is just we are in the early stages.
Well, only because you had bad teachers doesn't mean the topic is bad. You had the misfortune to interact with people that don't understand the topic they try to explain. I try to give a simple explanation here: https://consulting.0x4447.com/articles/knowledge/should-i-go... - just as a starting point to at least get a frame of reference when to use and not to use Serverless. But hey I'm just a random person in a random location on this planet :)
Serverless is meant to make bar lower for front-end developer so they don't have to learn the basics about web back-end development and be able to develop apps. I'm very glad if failed!
People who don't put time into learning development should not do it.
Your second point broadly makes sense. In fact, it's almost a truism: if you don't put time into learning, you won't have the skills necessary to be able to execute.
However, it seems detached from your first point.
If one's role is a front-end developer, is it necessary that they know about back-end development? If it is outside their intended job function, why would they need to know about it, if it doesn't get in the way of performing their job? If you are a backend developer, do you need to know about how to host your own infrastructure? Handle your own networking? Chip design? Your logic could be applied to any job function. Each level of the stack benefits from the levels below it being abstracted. We all stand on the shoulders of giants, and we're all much better for it.
Overall, I do think it's better that one has a good understanding about the various components one interacts with. Having a grasp of the overall system will come in handy. A curiosity into other parts of the system is beneficial, and likely is one of many indicators of success. However, if job functions can be simplified and superfluous context removed, why should we fault those for taking advantage of that?
This issue with "lowering the bar" and being glad that a simplification has "failed" (which, is yet to be determined), reeks to me like gatekeeping. The same logic could be applied to any job role which benefits from simplification. In an extreme example, this logic could be extrapolated to support the notion that anyone who cannot build their own machine from the ground up should never work in a programming position. What height is appropriate for the "bar"?
Kind of surprised the article didn't mention lack of reasonable development environment.
At least on AWS, the "SAM" experience has been probably the worst development experience I've ever had in ~20 years of web development.
It's so slow (iteration speed) and you need to jump through a billion hoops of complexity all over the place. Even dealing with something as simple as loading environment variables for both local and "real" function invokes required way too much effort.
Note: I'm not working with this tech by choice. It's for a bit of client work. I think their use case for Serverless makes sense (calling something very infrequently that glues together a few AWS resources).
> It's so slow (iteration speed) and you need to jump through a billion hoops of complexity all over the place. Even dealing with something as simple as loading environment variables for both local and "real" function invokes required way too much effort.
Honestly, it reminds me of PHP development years ago: running it locally sucked, so you need to upload it to the server and test your work. It. Sucked.
It was actually pretty good if you had an IDE with sftp/scp support because you could save a file, refresh your browser, and have immediate new results.
Yeah this wasn't too bad and it was what I used to do back in the day with Notepad++. By the time you hit save in your editor, your changes were ready to be reloaded in the browser.
With SAM we're talking ~6-7 seconds with an SSD to build + LOCALLY invoke a new copy of a fairly simple function where you're changing 1 line of code, and you need to do this every time you change your code.
That's even with creating a custom Makefile command to roll up the build + invoke into 1 human action. The wait time is purely waiting for SAM to do what it needs to do.
With a more traditional non-Serverless set up, with or without Docker (using volumes) the turn around time is effectively instant. You can make your code change and by the time you reload your browser it's all good to go. This is speaking from a Python, Ruby, Elixir and Node POV.
The workaround my team uses is to make two entries to start the webapp: one for SAM, one for local. For fast iteration we just `npm start`, and when we're ready to do more elaborated testing we run with SAM. This works pretty well so far.
I'm not sure why that's PHP's fault? I never had problems running it locally... and to "get my code to the servers" was as easy as a git pull from the server which is probably the 2nd laziest way of accomplishing that.
Out of genuine interest... is there a modern solution to this problem with PHP/MySQL?
(I'm still doing the "upload to server to test" thing.... I've tried MAMP and Vagrant/VirtualBox for local dev but both of them seem horribly complex compared to what we can do with local dev with node.js/mongo and so on.)
"docker-compose up" and your OS, codebase, dependecies and data is up and running locally in the exposed local port of your preference. You can even split parts in different layers to mimic a services/cross-region logic.
Of course this won't fix the fact that you have a lambda behind api gateway that does some heic->jpg conversion and can't be hit outside the DMZ, or some esoteric SQS queue that you can't mimmic locally - but it should get you almost there.
this doesnt solve the OPs problems though, if Vagrant is complex, so would be a docker image. the problem is the user doesn't know how to manage/configure the underlying complexity of the OS and needed services, which would still be a problem if using docker. unless you find that perfect docker image with every dep you need... but that would also be true with vagrant.
FWIW I haven't hit any scenario out of the basic services that localstack couldn't run locally. I even have it executing Terraform on localstack as if it was AWS (without IAM which is indeed a problem when I forget to create/update policies)!
Just run PHP and MySQL locally? Native PHP on Windows is horrible to set up, but with WSL/WSL2 you suddenly can get a normal Linux environment without much hassle.
sudo apt install nginx php mysql, point the www directory of nginx to something on your computer (/mnt/c/projects/xyz) and you've got a running setup. Or run Linux in general, that's what most people I've seen work on backends seem to do. You can run the same flavour of software that your deployment server runs so it'll save you time testing and comparing API changes or version incompatibilities as well.
I don't know any solution for macOS but you can probably get the necessary software via Brew if you're so inclined. Then run the built-in PHP web server (php -s) and you get the same effect.
What's horrible about it? I just download it, unzip to Program Files, add the folder to my %PATH% and that's about it. I didn't find myself in a situation where I would need an Apache or other webserver, the built-in one is good enough. It also makes using different versions easy, no need to deal with Apache and CGI/FPM. You just use other PHP executable.
I find it easier to handle multiple PHP versions on Windows than on Linux. As you say just download zip,unpack somewhere copy php.ini-development to php.ini, and you can do this for every minor PHP-version.
Apache is almost as easy, download zip, unpack & configure apache conf to use your php.
MySQL is somewhat more complicated because you need to run an setup script after unpacking the zip.
I used to be complain about the same thing and even asked someone who was head of BD for Serverless at AWS what they recommended, and didn't get an answer to my satisfaction. After working with more and more serverless applications (despite the development pains, the business value was still justified) I realized that local development was difficult because I was coupling my code to the delivery. This is similar to the way you shouldn't couple your code to your database implementation. Instead, you can write a function that takes parameters from elsewhere and call your business logic there. It definitely adds a bit more work, but it alleviates quite a bit of pain that comes with Lambda local development.
Disclaimer: I work at AWS, however not for any service or marketing team. Opinions are my own.
> Instead, you can write a function that takes parameters from elsewhere and call your business logic there.
This is what I tried to do initially after experiencing the dev pain for only a few minutes.
But unfortunately this doesn't work very well in anything but the most trivial case because as soon as your lambda has a 3rd party package dependency you need to install that dependency somehow.
For example, let's say you have a Python lambda that does some stuff, writes the result to postgres and then sends a webhook out using the requests library.
That means your code needs access to a postgres database library and the requests library to send a webhook response.
Suddenly you need to pollute your dev environment with these dependencies to even run the thing outside of lambda and every dev needs to follow a 100 step README file to get these dependencies installed and now we're back to pre-Docker days.
Or you spin up your own Docker container with a volume mount and manage all of the complexity on your own. It seems criminal to create your own Dockerfile just to develop the business logic of a lambda where you only use that Dockerfile for development.
Then there's the whole problem of running your split out business logic without it being triggered from a lambda. Do you just write boiler plate scripts that read the same JSON files, and set up command line parsing code in the same way as sam local invoke does to pass in params?
Then there's also the problem of wanting one of your non-Serverless services to invoke a lambda in development so you can actually test what happens when you call it in your main web app but instead of calling sam local invoke, you really want that service's code to be more like how it would run in production where it's triggered by an SNS publish message. Now you need to somehow figure out how to mock out SNS in development.
Unless I’be misunderstood, every knock against serverless above has actually been a knock against the complexity of having tiny, de-coupled cloud native services and how difficult it can be to mock... to which the answer is often “don’t mock, start by using real services” and then when that is less reliable or you need unit tests, then mock the data you expect. In the case of SNS, mock a message with the correct SNS signature, or go one layer deeper, stub out SNS validation logic and just unit test the function assuming the response is valid or invalid? In the case of Postgres, you could use an ORM that supports SQLite for dependency-free development but at a compatibility cost... worst case you might need to have your local machine talk to AWS and host it’s own LetsEncrypt certificate and open NAT port... but one can hope it doesn’t come to that...? Even so... that’s not exactly a knock against serverless itself, is it?
> In the case of SNS, mock a message with the correct SNS signature, or go one layer deeper, stub out SNS validation logic.
SAM already provides a way to mock out what SNS would send to your function so that the function can use the same code path in both cases. Basically mocking the signature. This is good to make sure your function is running the same code in both dev and prod and lets you trigger a function in development without needing SNS.
But the problem is locally invoking the function with the SAM CLI tool is the trigger mechanism where you pass in that mocked out SNS event, but in reality that only works for running that function in complete isolation in development.
In practice, what you'd really likely want to do is call it from another local service so you can test how your web app works (the thing really calling your lambda at the end of the day). This involves calling SNS publish in your service's code base to trigger the lambda. That means really setting up an SNS topic and deploying your lambda to AWS or calling some API compatible mock of SNS because if you execute a different code path then you have no means to test the most important part of your code in dev.
> In the case of Postgres, you could use an ORM that supports SQLite for dependency-free development but at a compatibility cos
The DB is mostly easy. You can throw it into a docker-compose.yml file and use the same version as you run on RDS with like 5 lines of yaml and little system requirements. Then use the same code in both dev and prod while changing the connection string with an environment variable.
> That’s not exactly a knock against serverless itself, is it?
It is for everything surrounding how lambdas are triggered and run. But yes, you'd run into the DB, S3, etc. issues with any tech choice.
So there’s an argument that the future deployment model is actually Kubernetes Operators, which means you could have test code that deploys and sets up AWS APIs... thus if your code responds to the trigger, it’s up to another bit of code to make sure the trigger is installed and works as expected against AWS APIs?
And yes, I think the problem here are APIs you use in multiple places but can’t easily run yourself in a production-friendly way. Until AWS builds and supports Docker containers to run their APIs locally, I don’t see how this improves... end to end testing of AWS requires AWS? ;-)
> I realized that local development was difficult because I was coupling my code to the delivery.
Of interest, I've spent some free time crunching on CNCF survey data over the past few months. Some of the strongest correlations are between particular serverless offerings and particular delivery offerings. If you use Azure Functions then I know you are more likely to use Azure Devops than anything else. Same for Lambda + CodePipeline and Google Cloud Functions + Cloud Build.
I think his point was that you should be able to run and test the Lambda code independently of Lambda. After all the entry point is just a method with some parameters, you can replicate that locally.
Yes, this is a great way of doing things - I have no problems TDD'ing business logic hosted in Lambda, because the business logic is fully decoupled from the execution environment. SAM should be for high-fidelity E2E integration testing.
This principle works with front end development too. Crazy build times on large applications can be alleviated if your approach is to build logic in isolation, then do final testing and fitting in the destination.
It’s hard to do this when surrounding code doesn’t accommodate the approach, but it’s great way to design an application if you have the choice. I really love sandboxing features before moving them into the application. Everything from design to testing can be so much faster and fun without the distractions of the build system and the rest of the application.
I felt your pain immediately and decided to write my own mini-framework to accomplish this.
What I have now is a loosely coupled, serverless, frontend+backend monorepo that wraps AWS SAM and CloudFormation. At the end of the day it is just a handful of scripts and some foundational conventions.
I just (this morning!) started to put together notes and docs for myself on how I can generalize and open source this to make it available for others.
stack is vue/python/s3/lambda/dynamodb/stripe but the tooling I developed is generic enough to directly support any lambda runtime for any sub-namespace of your project so it would also support a react/rails application just as well.
As a systems developer, comments like yours make me amazed at the state of web development. From the outside looking in, it seems like 10% code and 90% monkeying around with tooling and frameworks and stacks.
I believe it's the moment when there's a solution that just makes sense and works well for most people. A gold standard that other solutions will try to develop more and spice up, instead of reinventing it.
A lot of these DX (developer experience) concerns are, imo, rooted in what the article describes as "Vendor Lock".
Sure, you can write a bunch of tools to work around the crufty, terrible development environment's shortcomings. But ultimately, you are just locking yourself further & further & further in to the hostile, hard to work with environment, bending yourself into the bizarre abnormal geometry the serverless environment has demanded of you.
To me, as a developer who values being able to understand & comprehend & try, I would prefer staying far far far away from any serverless environment that is vendor locked. I would be willing & interested to try serverless environments that give me, the developer, the traditional great & vast powers of running as root locally that I expect. Short of a local dev environment, one meets both vendor lock in, & faces ongoing difficulties trying to understand what is happening, with what performance profiles/costs. I'd rather not invest my creativity & effort in trying to eek more & more signals out of the vendor's black box. Especially if trouble is knocking, then I would very much like to be able to fall back on the amazing toolkits I know & love.
aws's whole pitch has been cutting out server huggers & engineers, relying on aws. since day 1. often to wonderful effect. with far far better software programmability than our old crufty ways.
but lambda gets to the point where there is no local parity, where it's detached, is no longer an easier managed (remotely operated) parallel to what we know & do, but is a system entirely into itself, playing by different rules. one must trust the cloud-native experience it brings entirely, versus the historical past where the cloud offered native local parallels.
I never got the hand of cloud formation. I suppose it is nice from a visual (drag and drop) point of view, but I couldn't use it in production and moved on to manage my architecture with terraform.
It sounds like you're describing the Cloudformation template visualiser/editor in the AWS Console, which I have never heard of anyone using as the primary interface for their Cloudformation templates.
Personally for simple projects I've had pretty good experiences writing a Yaml-based template directly, and for more complex projects I use Troposphere to generate Cloudformation template Yaml in Python.
This is a really funny thing, since for the last ~10 years I've been hearing how we're deliberately doing IaC/CM tools "not a programming languages because reasons" (and thus have to do horrible hacks to support trivial things like loops and conditions), and now suddenly we're building libraries in programming languages that convert the code into non-programming-language description, which is then interpreted by a program into several other intermediate representations and finally emits trivial API commands. I guess the next step would be write a declarative language on top of CDK or Pulumi that will compile it into python which will generate CF/TF files.
I manage a handful of projects with Terraform and it works well in many situations. It has improved a lot recently but for a long time I really hated the syntax. I still do to some extent but have learned to cope with it most of the time.
If you are working on a project where all of your infrastructure will live on AWS I would definitely urge you to give it a second look. The amount of infrastructure I manage right now with a single .yaml file is really killer.
Yes, it (Python) was chosen because we could leverage existing internal code that was written in Python and it happens to be my strongest language.
If I could do it all over, I would still choose Python. That being said, I have been working professionally (building apps like this) for almost 14 years so my willingness to bite off a homebrew Python framework endeavor as I did here is a lot different than someone just getting into the field.
Django: avoid unless you have a highly compelling (read: $$$$) reason to learn and use this tool. I cannot think of one, honestly.
Flask: fantastic, but be conscientious about your project structure early on and try to keep businees-logic out of your handler functions (the ones that you decorate with @app...)
Sophisticated or more sugary Node.js backends are not something I have ever explored, aside from the tried-n-true express.js. I tend to leverage Python for all of my backend tasks because I haven't found a compelling reason not to.
Django is decent for POCs that need some level of security since you get authentication out of the box with no external database configuration necessary due to sqlite. Sometimes you have an endpoint that needs that due to resource usage, but the number of users is so low that setting up a complicated auth system isn’t worth it.
Minimalist frameworks are great for either very small (since they don’t need much of anything) or very large projects (since they will need a bunch of customization regardless).
In that regard, I think Django is kind of like the Wordpress of Python.
That is such a tough question to answer carte blanche.
All-in-all, Django is not bad software. I have a bad taste in my mouth though because as I learned and developed new approaches to solving problems in my career I feel like Django got in the way of that.
For instance, there are some really killer ways you can model certain problems in a database by using things like single table inheritance or polymorphism. These are sorta possible in Django's ORM, but you are usually going against the grain and bending it to do things it wasn't really supposed to. Some might look at me and go: ok dude well don't do that! But there are plenty of times where it makes sense to deviate from convention.
That is just one example, but I feel like I hit those road blocks all the time with Django. The benefit of Django is it is pre-assembled and you can basically hit the ground running immediately. The alternative is to use a microframework like Flask which is very lightweight and requires you to make conscious choices about integrating your data layer and other components.
For some this is a real burden - because you are overwhelmed by choice as far as how you lay out your codebase as well as the specific libraries and tools you use.
After your 20th API or website backend you will start to have some strong preferences about how you want to build things and that is why I tend to go for the compose-tiny-pieces approach versus the ready-to-run Django appraoch.
It's really a trade-off. If you are content with the Django ORM and everything else that is presented, it is not so bad. If you know better, you know better. Only time and experience will get you there.
That's great, cheers for that. It's helpful to know that your concerns are mainly to do with taking an opinionated vs non-opinionated approach - that's a framework for thinking about the choice between Django and (e.g.) Flask that many people (including myself) can hang their hat on.
On the flip side, not being able to use Django is one of the reasons against serverless for me. There's immense value in having a library for anything you might think of, installable and integratable in minutes.
You have to roll your own way too often in Flask et al, so much so that I don't see any reason to use Flask for anything other than ad-hoc servers with only a few endpoints.
Django gets you a lot if you have a traditional app with a traditional RDBMS and a traditional set of web servers. It’s too opinionated to easily map into AWS serverless.
Take a look at the [CDK](https://aws.amazon.com/cdk/) if you haven't already. It lets you define your infrastructure using TypeScript, which then compiles to CloudFormation. You can easily mix infrastructure and Lambda code in the same project if all you're doing is writing some NodeJS glue Lambdas which sounds like what you're looking for.
There's a couple of sharp edges still but in general it just 'makes sense'. If you don't like TypeScript there are also bindings for Python and Java, among others, although TypeScript is really the preferred language.
CDK made IaC accessible to me. I hated raw CloudFormation and never bothered with it because of that reason. I had a crack at Terraform, but never got passed the learning curve before my enthusiasm died.
Currently using some CDK in a production app and finally I found a way of doing IaC I actually enjoy.
You might really like pulumi. I'm kind of on the opposite end, ops>swe so tons of IAC and i'm using pulumi now as I'm more swe focused https://www.pulumi.com/ (ive no relation to them)
Basically exact same as CDK. I really prefer this style over CloudFormation and Terraform. I think Pulumi emerging as another player in the space legitimizes the approach.
CDK is moving quite fast and not all parts are out of the experimental phase, so there are breaking changes shipped often. I think in a couple of years it will stabilize and mature and become a very productive way of working with infrastructure.
GCP has https://cloud.google.com/functions/docs/functions-framework but I will not use it. I have found the best solution is to abstract away the serverless interface and create a test harness that can test the business logic. This adds some extra complexity in the code, but iterations are fast and do not rely on the overly complex and bug prone "platforms" like SAM and Functions Framework.
This is precisely what I do when I write code destined to be an AWS Lambda Function. It really feels like the only sane way to do it. It also makes it easy to mock the incoming request/event for integration tests.
Developer experience for serverless is such a pain point, spot on. AWS SAM has tackled some of the IaC modeling problem (on top of CloudFormation which is a mature choice) and they've had a crack at the local iteration (invoke Lambda function or API Gateway locally with connectivity to cloud services).
It's a little incomplete, missing some of the AWS IAM automation that makes local development smooth, environment management for testing and promoting changes, and some sort of visualization to make architecture easier to design as a team.
I work for a platform company called Stackery which aims to provide an end-to-end workflow and DX for serverless & CloudFormation. Thanks for comments like these that help identify pain points that need attention.
Yeah, I took a look at using a serverless framework for a hobby project, and it was just a real pain to get started at all, let alone develop a whole application in.
I tried AWS, and then IBM's offering which is based on an open source (Apache OpenWhisk) project, thinking that it might be easier to work with, but that was also a pain.
I just lost interest as I was only checking it out. For something constantly marketed on the ease of not having to manage servers, it fell a long way short of "easy".
> Yeah, I took a look at using a serverless framework for a hobby project, and it was just a real pain to get started at all, let alone develop a whole application in.
Look into Firebase functions. Drop some JS in a folder, export them from an index.js file and you have yourself some endpoints.
The amount of work AWS has put in front of Lambdas confuses me. Firebase does it right. You can go from "never having written a REST endpoint" to "real code" in less than 20 minutes. New endpoints can be created as fast as you can export functions from an index.js file.
And if you need a dependency that has a sub dependency with a subdependency that uses a native module prepare for poorly defined -fun- hell getting it to work. A surprising amount of standard is libs do.
Being able to throw up a new REST endpoint in under 10 minutes with 0 config is really cool though.
And Firebase Functions are priced to work as daily drivers, they can front an entire application and not cost an insane amount of $, per single ms pricing. Lambda's are a lot more complicated.
> Kind of surprised the article didn't mention lack of reasonable development environment.
I've been pretty happy with Cloudflare Workers.
You can easily define environments with variables via a toml file. The DX is great and iteration speed is very fast. When using `wrangler dev` your new version is ready in a second or two after saving.
I can report that Azure Function App development is at least pretty decent, as long as you have the paid Visual Studio IDE and Azure Function Tools (haven't tried the free version yet).
I tried AWS Lambdas a few years back and it felt way more primitive.
Azure Function App development experience is indeed pretty nice at least when using .NET Core. There are some issues, like loading secrets to the local dev environment from Key Vault has to be done manually and easy auth (App Service Authentication) does not work locally.
I've used Azure's serverless offering "Functions" quite a bit. The dev experience is pretty good, actually - it "just works" - start it and point your browser at the URL. And certainly no problems setting up env vars or anything basic like that.
My only nitpick, and only specifically relating to dotnet, is that config files and env vars differ between Functions and regular ASP.NET Core web apps. I think there is some work going on to fix that, but it's taking forever.
Couldn’t agree more, the dev experience was awful. You basically have to develop against public AWS services, my dev machine became a glorified terminal. They do seem to be iterating on the tooling quickly, but I wouldn’t use it again if I had a choice.
Edit: CloudFormation was also painful for me, the docs were sparse and there were very few examples that helped me out.
SAM templates are a subset of CloudFormation templates; that PDF could be three times as long and still not have the content I needed.
Yes there are examples, but there wasn’t one at the time that mapped to what I was trying to accomplish. Because, again, SAM templates are not one-for-one CloudFormation templates.
I found the community around SAM to be very limited. One of the many reasons I’ve moved to the Kubernetes ecosystem.
It definitely doesn’t have to be that way. I work on Firebase and I’ve spent most of the last year specifically working on our local development experience for our serverless backend products:
https://firebase.google.com/docs/emulator-suite
Still very actively working on this but our goal is that nobody ever has to test via deploy again.
Love firebase, thanks for your work! the local emulator suite is so important a feature and keenly following your progress.
Slightly OT... perhaps cheeky... any idea why firebase doesn’t provide automated backups for firestore and storage out of the box? Seems like a no brainer and a valuable service people would pay for.
I'm currently working on a little project backed by Firebase. Really interesting. Good to hear you're doing this - at my day job one of our key factors in choosing a technology is whether we can spin it up in docker-compose in CI and do real-as-possible ephemeral integration tests.
My experience is with .NET Core and the development experience is awesome... Dropped a $250/m cost down to ~$9/m moving it from ec2 to lambda's. Environment variables are loaded no differently between development and prod. Nothing is all over the place as it's almost 0 difference between building a service to run in Linux/Windows vs a Lambda.
Keep in mind Firebase has a big caveat. Firebase is great... for what it does. However, there's no way to easily migrate the Firebase resources to the larger GCP ecosystem. Firebase does what it does, and if you need anything else, you're out of luck.
Firebase is magic... but I never recommend it for anyone, until there's some sort of migration path.
[Firebaser here] that’s not quite accurate. For cloud functions they’re literally the same. Your Firebase function is actually a GCP function that you can manage behind the scenes.
With Cloud Firestore (our serverless DB) that’s the case as well. And Firebase Auth can be seamlessly upgraded to Google Cloud Identity Platform with a click.
However you’re right that for many Firebase products (Real-time Database, Hosting) there’s no relation to Cloud Resources.
Deploying is super slow. Usually it takes a minute or two, which is already quite long, but sometimes something goes wrong and then you can't redeploy immediately. You have to wait a couple of minutes before being able to redeploy.
To be fair, Firebase recently released a local development tool which alleviates the need to deploy on every change, but I haven't used it yet.
I'm a big Firebase user with Firestore and has been great...no not perfect and the "cold start" is probably the worst issue. However, deployments are easy, the GUI tool keeps getting better, like the extension packages, and the authentication system is quick to implement.
I found Amplify excellent to get up and running quickly. I’d highly recommend it for anyone without a well-oiled CICD setup who wants to quickly get a website up to test out an idea.
Unfortunately, I quickly hit the limits of its configurability (particularly with Cloudfront) and had to move off it within a few months.
“Serverless models don’t require users to maintain their own operating systems, or even to build applications that are compatible with particular OSs. Instead, developers can produce generic code, and then upload it to the serverless framework, and watch it run.”
... is utterly compelling and is why serverless will not just win, but leave renting a server a tiny niche market that few developers will have an experience of post 2030.
Maintaining your own server is completely nuts. If that isn’t obvious now, it will be in another decade. It’s massively inefficient. Like running your own power plant to serve your factory, except you also have to worry about security and constant maintenance, along with all the moving parts that surround a server.
Almost all the objections in the article can be rephrased as “serverless is not mature enough yet”, and that’s accurate, but I suspect there’s also a bias against giving up control to the cloud companies, and some wishful thinking as a result.
The future of software development is going to be defined by cloud providers. They’re going to define the language ecosystem, the canonical architectures for apps etc... it’s just early days and cloud is really very primitive. Just clicking around Azure or GC or AWS illustrates how piecemeal everything is. But they have a lot to do, and just keeping pace with growth is probably hard enough. I’m not sure I’m super happy with this outcome, but I’m pretty certain the trend line is unmissable.
It's not clear to me how much experience with serverless architectures the author of the parent comment has, but speaking as someone with plenty, the operational costs of serverless are at least equal to managing stateful infrastructure, with much less control when things go wrong. Lambda was a major step up in long term predictability compared to for example App Engine, where there have been plenty of instances of overnight unannounced changes, or changes announced with incredibly short notice period, requiring developer time rather than ops time to bring an application back to service.
On the ops side even with a platform like Lambda, training an operations team to take over maintenance of a nested spaghetti of random interlinked services and bits of YAML trapped in random parts of the cloud is a total nightmare. The amount of documentation required and even the simple overhead of enumerating every dependency is a long term management burden in its own right. "The app is down" -> escalate to the developers every single time.
Compare that to "the app is down", "this app has basically no ops documentation", "try rebooting the instances", "ah wonderful, it came back"
I'm pro-cloud in many ways and even pro-serverless for certain problems, but let's not close our eyes and pretend dumping everything into these services is anything like a universal win.
This. 100 times this.
Also, in several places most of the services downtimes are due to, you know what? Application bugs, not infrastructure outages. Sure, they happen as well and being on a good cloud provider mitigate a lot of them (but not all of them!) but if you increase the application design complexity you will increase also those downtimes. Yeah sure, there are tens of really good engineering departments where everything is properly CI?CD, automated, they can scale to thousand of services without skipping a beat... but that's not the reality for thousands of other smaller/less talented shops. So, "moving to serverless" will not just automagically fix all of your problems.
Also - and I'm an infra guy so I'm probably biased - I don't really get all this developer anxiety to outsource infra needs. Yeah if you are 2 devs working on your startup it makes sense, but when you scale up the team/company, even with serverless, you WILL need to dedicate time to infra/operations and do not dedicate it to strictly-business-related code. Having somebody dedicated to this is good for both.
I haven't done anything with serverless, but surely the class of problems that would be fixed by an instance restart don't happen in the first place on serverless
It was intended more to invoke general ideas about management ease than being a specific remediation, however
elsewhere in the thread there is an example of a diagramming tool split out across 37 individual AWS services/service instances. In a traditional design, this is conceivably something where all state and execution could easily fit in one VM, or perhaps one container with the state hoisted off to a managed service. In this case we could conceivably fix some problems with an app like that literally just by kicking the VM
I don't think you're wrong, I just think you're not looking far enough ahead.
What we have now is very primitive compared to how app development might work in the future; serverless is laying the foundation for a completely different way of thinking about software development.
It's more back to the mainframe model of software development. I did this back in the 90s and I never had to think about scaling. Granted these were just simple crud / back-office apps.
But I can see how it would work for most modern software.
>training an operations team to take over maintenance of a nested spaghetti of random interlinked services and bits of YAML trapped in random parts of the cloud is a total nightmare. The amount of documentation required and even the simple overhead of enumerating every dependency is a long term management burden in its own right.
Upon learning about it some time ago, this was exactly my conception of what a Lambda-like serverless architecture would yield.
And it would seem difficult, if not impossible, for any dev to maintain a mental map of the architecture.
I've in this area professionally for some time now, and I've never "maintained" servers in any reasonable sense. There are kernel people who maintain the kernel and there are Debian devs who maintain the operating system. The server may be mine (but more often that not, it isn't) but only in very specific circumstances do I ever concern myself with maintaining any part of this stack.
A vanilla Linux VM is a platform to build on. Just like AWS or anything else. It is the environment in which my software runs.
Thus far, something like Debian has been more stable and much less of a moving target than any proprietary platform has been, cloud or non-cloud. Should a client wish to minimize maintenance costs of the software over the coming decade, it is most cost effective to not depend on specialized, proprietary, platform components.
That may change in the future, but right now there is no indication that is the case.
That's exactly the argument why you should go serverless. If all you do is keep a vanilla Linux distro running in a VM with occasional updates and some initial config-magic (webserver, certs, ip tables, ssh etc.) why even bother? The serverless isn't going to be any different, other than it just runs. No need to make a cron for updates, no iptables, no certs or webserver-stuff ... just put your app on it and let it go. On the other hand, if you actually need to tinker with your OS, roll your own. But what you described is the prime example of serverless target audience.
One reason serverless isn't the solution for most applications is that you're basically making yourself entirely dependent on the API of one specific cloud host. If Lambda decides to double its price, there's nothing you can do about it but pay. If you need to store your data in a specific place (for example, in Russia, because all Russian PII must be stored in Russian borders), then you're out of luck. And best of luck to you if you're about to catch a big customer but they demand your application to run on their premises.
There's also the long-time guarantee; if you write an application that runs on Ubuntu Server or Windows Server now, you can bet your ass that it will still run, unchanged, for another 10 years. The only maintenance you need to do is to fix your own bugs and maybe help out with some database stuff. If you deploy a Lambda app now, you have nothing to guarantee compatibility for such a long time other than "Amazon probably won't change their API, I think".
If you built everything in proprietary infrastructure, porting is a lot more work.
Using lambdas tie you to AWS because as soon as you use a few step functions, or you have a few lambdas interacting, changing to Azure or GCP becomes a huge pile of dev work and QA.
Having everything "just run" on linux instances let you be perfectly portable and now you can actually shop around.
Kubernetes, microservices, distributed systems, SPA apps. Getting a development environment to reproduce bugs takes up so much of my time these days (a fair bit is because of our crappy system, but there also a lot of inherent complexity in the system because of the architecture choices listed above). We get the promise of scaling, but most places making these choices don't actually need it.
The above comment was intended to answer exactly that: because it is much less maintenance in the long run.
Had you deployed an application to a proprietary cloud platform ten years ago, a handful of those services would have had their APIs changed or even been sunset by now.
Yeah but those "very specific circumstances" can come up at inconvenient moments. Certbot didn't run, disk got full, CPU hitting 100% for some reason, Debian needed to be upgraded, These are all things that need to be taken care of. Sometimes, right away and just when your family needs you.
I agree that it almost never happens, and that's why I run Debian as well. However, if you run production then things happen.
Just like when something unexpectedly breaks due to a component of the serverless environment that has been changed slightly causing your system to break, only then you have even less control when trying to debug the issue
I don't do any server admin. My code runs in docker on pretty much any server I can get my hands on. Some of my code runs on a ThinkPad stashed behind my desk, on DigitalOcean, on my Macbook. I could deploy to a Raspberry Pi and it would run just the same. It takes 10 minutes to deploy an exact copy to a new environment.
None of that requires OS maintenance. My house plants require far more maintenance than my software. I sometimes forget where some things run because I haven't touched them in years.
Serverless code runs on the cloud you built it for. I don't want that. I don't want to invest years of my life becoming an Amazon developer, writing Amazon code for Amazon servers.
That's without delving into the extra work serverless requires. There isn't a dollar amount tied to my import statements. I don't need software to help me graph and make sense of my infrastructure. I can run my code on my computer and debug it, even offline.
On hosts I manage professionally, I update/upgrade weekly after reading the notes - it takes a few minutes, I know I'm up to date and if there is anything I should be wary of.
On a personal debian server, I have an update/dist-upgrade -y nightly on a cron job, and I reboot if I read on HN/slashdot/reddit/lwn about an important kernel fix; Never had an issue, and I suspect it's about as secure and trouble free as whatever is underlying lambda -- with the exception that every 3-4 years I have to do an OS upgrade.
> None of that requires OS maintenance. My house plants require far more maintenance than my software. I sometimes forget where some things run because I haven't touched them in years.
Then how do you know they are still secure and even working?
Yes, deploying servers is very easy, maintaining and securing them is the hard part. Sure, you can automate the updates and it will work with a good OS-Distribution for some years. But no system is perfect, exploits are everywhere, even in your own configuration. And then it becomes tricky to protect your data.
The HDD died already. I opened it, moved the stuck head with my fingers and shoved it back in. I have good backups, and as someone else said, if it was important, it would not run on a recycled laptop behind my desk.
That hard drive event showed me how disposable the machine itself has become thanks to docker.
I didn't miss the point. You singled out my ThinkPad and I'm answering your questions.
My point was about having portable code on generic hardware, which in my opinion is a better bet than writing Amazon software for Amazon servers, and praying their prices don't change much.
actually You can not run Your code smoothly on Rasberry PI because it has different architecture (ARM) and needs different sets of dependency images (If You are lucky at all Your deps are available on ARM)
It's not nuts to run servers. If an application is operating at any scale such that there is a nonstop stream of requests, then it will be cheaper, faster, and more energy-efficient to run a hot server. This follows from thermodynamics. No matter how good the cloud vendor's serverless is, it's always going to be less efficient than a server, unless it doesn't do any setup and teardown (i.e. no longer serverless).
It is nuts to run one server. Then you're wasting money with a server/VM. That's what serverless is ideal for: stuff no one uses. That's a real niche. Who's going to use that? Not profitable companies.
Often I think for most cases where you reach for serverless, you should reconsider the choice of a client-server architecture. An AWS Lambda isn't a server anymore; it's not "listening" to anything. Why can't the "client" do whatever the Lambda/RPC is doing?
Maybe what you want is just a convenient way to upload code and have it "just work" without thinking about system administration. The types of problems where you don't care about the OS is once again a niche. You probably don't even need new software for these kinds of things. You can just use SaaS products like Wordpress, Shopify, etc.
Serverless won't be profitable because the people who need it don't make money.
> Serverless won't be profitable because the people who need it don't make money.
You seem to implying only applications with huge numbers of users can be profitable. This statement ignores a tremendous amount of (typically B2B) applications that provide enormous value for their users but don't see a lot of traffic.
I have worked on applications that are at the core of profitable businesses yet they can go days, in some cases weeks without any usage. Serverless architecture will be a real benefit there once it matures.
I don't think that's necessarily true. Google Cloud functions give you 2 million invocations free a month - that's almost 1 per second. You can keep adding another 2 million for $0.40 at a time. It's not terrible.
I agree with the suggestion when to use a server, but I think making it out to be an obvious physical law is a bit too far.
Serveless runtimes can be massively multi-tenant, and in cases like Cloudflare have very little overhead per tenant, so they can share excess capacity for spikes, which you have to factor into your server. This gives them the ability to beat the server in thermodynamics. Maybe they will, maybe they won't but I don't think that's the argument that matters.
> Maintaining your own server is completely nuts. If that isn’t obvious now, it will be in another decade. It’s massively inefficient. Like running your own power plant to serve your factory, except you also have to worry about security and constant maintenance, along with all the moving parts that surround a server.
Except that it is not. The security and constant maintenance is needed but it is worth in many cases. And large companies can not really off load all ownership of data and applications. The interest in servers actually went in reverse direction due to the cloud effect.
I would say having own server or application hosting capacity is very similar to why you produce your own solar power and store it in batteries - it is simple? No. But the technology is improving and thus it makes it easier for people to adopt this paradigm.
When I see what is happening with WebAssembly/WASI in particular, I see a great future for self-hosting again. Software written in any programming language (as long as it targets WASM) is a lot easier to host than existing models. Also there is interoperability of software coming from different languages at the WebAssemply level as I understand.
> Software written in any programming language (as long as it targets WASM) is a lot easier to host than existing models.
For the past 20 years you have been able to target x86 gnu/linux and have it running without modification on a readily available server, either your own hardware or rented/public cloud. How does switching from one binary format to another (x86 to WASM) change anything (except maybe slowing down your code)? As I understand it, the main draw of WASM is running non-JS code in a web browser.
I use Google Cloud Run to run my serverless code for exactly this reason. GCR is literally just a container that runs on demand (with scaling to 0). Literally the only GCR specific part is making sure the service listens on the PORT env. If I was so inclined, I could deploy the exact same container on any number of services, host it myself and/or run it on my laptop for development purposes. There's also Kubernetes Knative which is basically (afaik) self hosted GCR.
Cloud Run is excellent if I wrote my own application. My biggest issue is that most off the shelf open source software that has a docker container often use a complicated docker compose file so if they can be deployed, they might be waiting on each other's cold starts (which can be very long and expensive) and/or need more databasesish things than I want. So obvious mistakes and unrealistic expectations aside, I have several nodejs and crystal apps on cloud run which is running well and just the concept "lambda for docker containers" is pretty awesome. Cold starts are pretty harsh ATM, hopefully they improve.
It varies massively based on the tech stack. I have seen both simple Spring Boot and Quarkus apps take in excess of 10 seconds to start up in JVM mode. However, Quarkus compiled to native binary with GraalVM starts consistently under a second (in the ~500ms range). This is still brutal compared to running it on a vanilla MacBook which usually takes less than 20ms.
That depends a lot on the use case. The ephemeral nature of serverless environment generally require you to proprietary solutions by the cloud provider to have things like DB access and such. So you end up using DynamoDB instead of Postgres (as an example). You CAN make portable serverless code but it generally requires a fair amount of work to do so.
Most actual FaaS code is quite portable; the configuration is what can’t move easily. And things like OpenFaaS and kNative make self-hosted runtimes completely flexible- it’s just a short-lived container.
Is there anything stopping an organisation from defining some standard types of serverless environments?
Is there anything stopping someone from turning that standard into implementations to help cloud providers offer it, or even be a fallback option that could be deployed on any generic cloud infrastructure?
The point of serverless for vendors is lock-in. Everything else about it is an annoyance, from their side (having to manage lifecycle, controlling load in shared engines, measuring resource usage...). But it locks people in and can be slapped with basically-arbitrary prices. The incentives to set standards are exactly zero, because once all providers support them and easy portability is achieved, it becomes a race to the bottom on price. Unless vendors can come up with some extra value-added service on top and choose to commoditize away the serverless layer, there won’t be any standard.
> Is there anything stopping an organisation from defining some standard types of serverless environments?
Yes. Basic economics. There is nothing _technical_ stopping 'an organisation' from making a federated twitter or facebook. But there are (evidently) insurmountable non-technical reasons: It hasn't happened / there have been attempts which have all effectively failed (in the sense that they have made no significant dent in these services' user numbers).
Why would e.g. Amazon (AWS) attempt to form a consortium or otherwise work together or follow a standard, relegating their offerings to the ultimate in elasticity? Economically speaking, selling grain is a bad business: If someone else sells it for 1ct less per kilo then the vast majority of your customers will go buy from someone else, there's no product differentiation.
Serverless lockin (such as GAE or AWS Lambda) is the opposite. No matter how expensive you make the service, your users will stay for quite a while. But make a universal standard and you fly in one fell swoop to the other end of the spectrum. If I have a serverless deployment and the warts of serverless are fixed (which would, presumably, involve the ability to go to my source repo, run a single command, give it some credentials, and my service is now live after some compilation and uploading occurs) - then if someone else offers it 1ct cheaper tomorrow I'll probably just switch for the month. Why not?
This cycle can be broken; but you're going to have to paint me a picture on how this happens. Government intervention? A social movement amongst CEOs (After the war, there was a lot of this going around)? A social movement amongst users so aggressive they demand it? Possible, but that would require that we all NOT go to serverless until the services offering it come up with a workable standard and make commitments to it.
I think it can happen simply through one serverless offering becoming very popular and other services (or open source projects) trying to reimplement the API of that offering. To some extent, this happened with Google App Engine. (AppScale)
I think cloud customers are savvy to the lock-in. That we're having this conversation in evidence of that. Perhaps AWS can achieve adoption of Lambda without needing to cater to customers who are cautious about getting locked in, but any challenger might find that it's much easier to gain customers if they also provide some form of an escape hatch.
As Jeff Bezos would say about retail, "your margin is my opportunity."
I disagree. Every AWS Lambda function I've ever written can be ran as a regular node/python process. The lambda-specific part is miniscule. If I wanted to run these on Azure or Google only the most inconsequential parts of the function would need to be changed.
In my experience, having started and abandoned side projects in both aws lambda and google app engine, half your project becomes:
* Well, obviously we use a hosted database
* And obviously, AWS provides our logging and all our analytics.
* Obviously when people call our lambda functions, they do so either through an AWS-specific API, or one constrained to a very limited set of forms.
* Of course, we can't blindly let everything access everything, so naturally we have IAM roles and permissions for every lambda function.
* Well, the cloud provider will look after secrets and things like that for us, no need for us to worry about database passwords.
* Naturally, with all these functions and IAM roles to look after, and we need tagging for billing. We should define it all with CloudFormation scripting.
* Well, the nosql database the they provide comes with their specific library. And as it shards things like this, and doesn't let you index things like that, you've got to structure your data this specific way if you want to avoid performance problems.
* You don't want your function to take 200ms+ to respond, your users will notice how slow it is. So no installing things with apt-get or pip for you, let me get you a guide on how to repackage those into the vendor-specific bundle format.
* You want to test your functions locally, with access to an interactive debugger? You're living in the past, modern developers deploy to a beta environment and debug exclusively with print statements.
* And so on.
In this case, a lot of the 'complexity' one hoped to eliminate has just been moved into XML files and weird console GUIs.
This, but it was told by serverless experts, on stage, in front of hundreds of people. If that was their sales pitch, their reality was likely even less impressive.
This sounds a bit like the posh workshop I went to on WAP (wireless access protocol) years ago when I worked for BT.
It was a complete omnishambles - to the point I avoided the fancy lunch and went to the Pub for a ploughman's lunch, In case I suddenly blurted out "this is all S*&T" and caused a political row with the mobile side of the company I worked for.
Never tried something marketed as Serverless but Google App Engine. If you wanted any performance, you had to follow the Guidelines really closely. Which could be legit, if it's worth the effort but I think it isn't. I think people under-estimate the effort and that the code will require much more not so nice optimizations than expected. That includes very verbose logging. It's sold as carefree and elegant but it only works when using patterns that nobody enjoys. I really liked the log browser and Dashboard though ;) It's like a stripped down version of New Relic and Elastic Search combined.
Most of your points are not relevant to my original statement of the code being generic - they are talking more about the architectural decisions. If I want to use DynamoDB I can do so in EC2 or Lambda, serverless doesn't dictate that. You also seem to believe one chooses Lambda because of complexity reduction and that's not really the only reason. I can very easily port a node/express API backend that connect to RDS to any other cloud provider. What about serverless makes you think that's not the case?
> Every AWS Lambda function I've ever written can be ran as a regular node/python process. The lambda-specific part is miniscule.
Of the actual function code, sure.
Of course, if you aren't manually configuring everything and are doing IaC, that isn't generic. And if you are supporting the Lambda with any other AWS serverless services, the code interfacing with them is pretty AWS specific.
I was only talking about function code. It should be painfully obvious to anyone that if you opt to leverage other AWS services that those are things you would ultimately have to replace but those decisions have nothing to do with serverless.
> but those decisions have nothing to do with serverless.
Sure they do.
Because if you need a DB for persistence you are either setting up a server for it (and therefore not serverless, even if part of your system uses a Lambda) or consuming a serverless DB service. And so on for other parts of the stack. On AWS, for DB, that might be Dynamo (lock-in heavy) or, say, Aurora Serverless (no real lock-in if you don't use the RDS Data API but instead use the normal MySQL or Postgres API, but that's higher friction—more involved VPC setup—to use from Lambda than RDS Data API is, so the path of least resistance leads to lockin.)
Lambda or other FaaS is often part of a serverless solution, but is rarely a serverless solution by itself.
What are you going to do when the container solution / node.js instance / insert x component here crashes? Wait for support to do something about it when you can fix it instantly? Or when you want to deploy a gRPC / crypto daemon to communicate with your back-end?
As an experienced back-end developer and linux user I would pull my hair out if I was completely helpless in fixing an issue or implementing some side thing that requires shell access. I don't want to wait for some guy in Philippines who will be online 12 hours later to come try to fix it.
Well, when will it be mature and what will it take to get there?
My first experience with serverless architecture was back in 2007 or so when trying to port Google News to App Engine. That was a thoroughly painful experience, and things haven't exactly gotten much easier since. If you go back in time a decade, Google's strategy for selling compute capacity was App Engine. Amazon went the EC2 route. Reality suggests AWS made the better choice.
I can understand the superficial notion that having idling virtual machines is inefficient (because it is). But this reminds me a bit of tricks we did to increase disk throughput in systems with lots of SCSI disks back in the day. Our observation was that if we could keep the operation queues on every controller full as much of the time as possible, we'd get performance gains when controllers were given some leeway to re-order operations, resulting in slightly better throughput. Overall you got higher throughput, and thus higher efficiency, but from the perspective of each process in the system, the response was sluggish and unpredictable. Meaning that if you were to try to do any online transactions, it would perform very poorly.
For a solution to be desirable it has to be matched with a problem that needs solving.
As the article points out, there are some scenarios where serverless architectures might be a good design paradigm. But extrapolating this to the assumption that this paradigm is a universal solution requires not only a leap of faith, but it also requires us to ignore observed reality.
So you owe me some homework. Tell me what needs to happen for serverless architectures to reach "maturity".
The midway point between "maintain your own servers" and "wholly in the cloud" is "configuration as code", using chef and terraform, or similar tools.
You don't patch your OS, or your apps, you define their versions and configuration in code and it gets built for you. And typically snapshotted at that point and made into a restartable instance image that can be simply thrown away if it's misbehaving, and rerun from the known good image.
Sometimes I wonder if this is propaganda by cloud/serverless providers to get everyone to jump on it and get locked in. The serverless black box kind of sucks, apart from “auto scaling” stuff. Crap performance too.
Auto-scaling is usually a myth anyway. You have to understand your system deeply and where all the bottlenecks are to really scale. If you have a part that's a big black box, that's going to get in the way of that.
I've never really understood this argument for serverless. Everything you do in AWS is through an API. I've never quite understood how replacing one set of API calls to provision an EC2 (or ECS cluster) is so much more complicated than another set of API calls to create a severless stack. If anything, my experience has been the complete opposite. Provisioning a serverless stack is much more complicated and opaque.
> Serverless models don’t require users to maintain their own operating systems, or even to build applications that are compatible with particular OSs. Instead, developers can produce generic code, and then upload it to the serverless framework, and watch it run.
Instead, developers can build applications that are compatible with particular serverless frameworks.
>The future of software development is going to be defined by cloud providers. They’re going to define the language ecosystem, the canonical architectures for apps etc...
That's... quite depressing to consider, actually. I long for a return to the internet of yore when Native apps were still king and not everything was as-a-service.
I agree. But you can still do native apps today, if you’re willing to jump through all the hoops that OS vendors set in your path and maybe renounce a couple of platforms (like chromebooks). Unfortunately, now that basically OS vendors are also effectively cloud providers, their incentives are set to increase those hoops (“for secuyriteh”), nudging more people towards “easier” cloud deployments.
It's massively inefficient, but it's also still massively safe in comparison, regarding soveignerity.
Like, today, given the political uncertainty in the USA, any large company would be nuts to bet on hosting their critical services on US-dependant infrastructure without having a huge plan B already in the works.
Is there a serverless provider with a "self hosted node" option that could be used as a fallback? That pretty much defeats the purpose of serverless, but at least you could hang on while figuring out how to transition to a new solution if the provider fails.
That has always been my sticking point. With AWS in particular. I don't trust AWS to exist forever, and it's definitely not without it own ongoing maintenance issues. Locking my entire business into their ecosystem seems risky at best.
Something I see really often in startups is huge dependencies in the form of SaaS. It should be no secret to those in tech that many of these businesses will not be around in 3 years. Even the likelihood of their service staying the same for 3 years is pretty low. I have been bitten by enough deprecated services, APIs and Incredible Journeys that I am wary.
In my experience, maybe it just isn't there yet, serverless has much more points of failure than a conventional infrastructure. I am sure there a lot of software that really needs the scale the cloud can provide. Funny thing is that we tend to use it for tiny apps, some IOT voice interfaces or BTB tools that we don't want in our corporate network.
I doubt cloud providers can dictate environments, other providers would quickly fill the gap to meet developer preferences. I also think that more developers care about lock in these days.
> The future of software development is going to be defined by cloud providers
That's probably true of web development, but "software development" writ large is much more than just cloud providers and webapps. "Software" encompasses everything from embedded microcontrollers to big iron mainframes that drive (yes, even today, even in 2030) much of the world's energy, transportation, financial and governmental infrastructure.
I think you've made a fair point, which I did have in mind but didn't write down - that there is a dichotomy between embedded and cloud systems, and the definition of software development will be to a lesser extent defined by the embedded side. Apple, for example, will have clout.
But long-term I think the cloud providers will assimilate so much power that the embedded side will follow its diktats.
Big iron mainframes have longevity, but will absolutely die out close to complete extinction - I've worked on those systems, I understand their strengths and the legacy issues, and there's no way that cloud isn't going to gobble up that market, it's just going to take a long time (as you say, beyond 2030).
Serverless is already here, it's just unevenly distributed. Instead it's called (managed) Kubernetes and yeah, you need a devops team and there's still a bunch of overhead but like you say - the writing's on the wall.
>> Like running your own power plant to serve your factory
With newer power technologies becoming more affordable and effective, solar, wind, & storage are increasingly being used to power factories and other businesses.
It's all about control of your product and operations. If it is economically feasible, it's always better to control your own stack all the way down.
Does serverleds actually deliver better control over your development, portability, reliability, security, etc. for your application & situation, or not?
This sounds a bit like the "When will they turn off the last mainframe?" arguments a while back - I wouldn't expect servers to disappear either...
Anyone moving to Hitrust or SOC2 loves serverless for that exact reason. When asked how I maintain my infrastructure, I point to RDS, API gateway, and Lambda. This leaves my security mostly free to focus on application level security.
Those are the same arguments as the ones put forth in the article under "the promise of serverless computing". It remains to be seen if they can be realized without the downsides.
The difference between time-sharing and serverless is that the former solved the issue of expensive personal computing, until cheap personal computers took over that market. The latter solves perceived expensive computing on the "server" side.
But what does it serverless solve exactly? It doesn't solve a technical problem, rather addresses concerns on the business side. Serverless solves a cost problem.
First, computing needs aren't linear, they fluctuate. And so, there's a problem of under- and over-utilization vs availability of resources. Serverless approaches computing power like tapwater: you're essentially paying for the CPU time you end up using.
Second, elasticity. Instead of having staff struggle - losing time - with the fine intricacies of autobalancers, sharding and what not; you outsource that entirely to a cloud provider. Just like a tap, if you need more power, you just turn the tap open a bit more.
Finally, serverless services abstract any and all low level concepts away. Developers just throw functions in an abstraction. The actual processing is entirely black box. No need to worry about the inner details of the box.
Sounds like a good deal, right?
> Like running your own power plant to serve your factory, except you also have to worry about security and constant maintenance, along with all the moving parts that surround a server.
Well... no. Outsourcing all of that to a third party cloud computing vendor doesn't dismiss you from your responsibility. All it does is shift accountability to the cloud provider who agreed to take you on as their customer. Securing your factory still very much includes deploying a secure digital solution to manage your machinery and process lines.
Plenty of industries wouldn't even remotely consider outsourcing critical parts of their operations, and this would include digital infrastructure. And this is regardless of the maturity of serverless technology. Risk management is a vast field in that regard.
Then there's legal compliance. There are plenty of industry specific regulations that simply don't even allow data to be processed by third party cloud services unless stringent conditions are adhered to. Medicine, banking and insurance come to mind.
Finally, when it comes to business critical processes, businesses aren't interested in upgrading to the latest technology for the sake of it being cutting edge. They want a solution that solves their problem and keeps solving that problem for many years to come. Without having to re-invest year after year in upgrades, migrations and changes because API's and services keep shifting.
Does that mean that there isn't a market for serverless computing? Of course there is. Serverless computing is a JIT solution. It's an excellent solution for businesses in a particular stage of their growth. And it closes the gap for plenty of fields where there really is a good match. I just feel that "maintaining your own server is completely nuts" is a bit overconfident here;
We had that utterly compelling framework in the early 2000s. Write generic code, upload to any provider you want, watch it run - that's exactly how shared-hosting PHP worked in ye olden days and to this day no one has made a developer experience as nice and It Just Works as that.
This on the assumption that there isn't a balkanization of the serverless frameworks between providers. History doesn't bear this. In 2020 we still need tools like BrowserStack even after decades now of web developers complaining about the fragmented ecosystem.
Instead of maintaining software compatible with the right operating system, you're maintaining software compatible with the right flavor of serverless by Cloud Provider. Now we're back to square one on at least one front.
On the control aspect, the bias against giving up control is not an unwarranted one. Maintaining control of critical infrastructure is extremely important. And in fact outsourcing your critical infrastructure is an existential one, and not just in an academic sense. When you give up control you give up the ability to prevent your infrastructure being hostile, but incompetent. In these cases it reduces the quality of your product for your customers.
I won't even go into the anti-competitive tactics Amazon themselves get into that make them not a good choice for your infrastructure. Instead I'll draw upon a recent experience that illustrates why outsourcing infrastructure, even at a higher level, is a bad idea.
My girlfriend recently was taking her NLN exam remotely. They weren't allowed to use calculators of their own, they had to use a virtual calculator provided by the company administering the test. Like most of these companies they are doing remote proctoring of the exams. During her exam this virtual calculator flat out wasn't available. The proctor told her to simply click through the exam and that once she submitted it she'd be able to call into customer service to get the exam re-scheduled due to the technical difficulties.
Well, that wasn't the case. After doing some deep digging for her, here is what I found. The testmaker NLN contracted out the test administration to a third party, Questionmark. Questionmark in turn contracted out yet another 3rd party, Examity to handle the proctoring. Examity proctors don't have access to Questionmark's systems. Questionmark doesn't have access to NLN's systems, etc. So how did we get this resolved? I had to track down the CEO of Questionmark, the CEO of Examity, and the head of testing for NLN. I had to reach out through LinkedIn inmail to get this on their radar. And then it was handled(quickly and efficiently I might add!). However, frontline support for each of these companies could do nothing. They just had to offload blame onto the support staff of each other. Another aspect of this is that each handoff between 3rd parties creates a communication barrier. In this case the communication barrier seems to have kept Questionmark from configuring this specific test correctly. I wouldn't blame any of these companies for this specific failure mode because it's just the nature of what happens when you offload your work to 3rd parties.
When you say, "Oh it's great we don't have to worry about X because Y can do it." The implication is that you lose all of the power of a vertically integrated company by essentially spinning off tons of subsidiaries and creating a communication overhead both before and when problems DO arise.
What is the future of software development going to look like when it reaches consumers and you have to say "Sorry, we can't fix that issue because Cloud Provider has to get back to me, and then in the background Cloud Provider has to say sorry we can't get back to you because we have to wait for our spunoff hardware division to get back to us?"
Maybe this type of business is fine for fun apps, but it's not fine for a lot of businesses. Even SLAs and disclaiming responsibility in your own contracts won't save your reputation. All it does is protect you financially!
Maintaining your own server is completely nuts. If that isn’t obvious now, it will be in another decade. It’s massively inefficient. Like running your own power plant to serve your factory, except you also have to worry about security and constant maintenance, along with all the moving parts that surround a server.
TANSTAAFL. Let’s say serverless becomes commoditised the same way as electricity. What are the margins in that business? What are AWS etc margins now?
There are very strong reasons to believe that serverless will offer convenience at a premium price, forever.
As someone on a two-man-team who runs a lot of little "utility" functions in AWS Lambda with the Serverless Framework[1] to support our DevOps / Build processes, it's been one of the most productive tools in my toolkit (after the initial learning curve, of course).
It allows me to stand up a practically maintenance-free endpoint in a matter of hours (usually to glue separate services together):
* Want to run a quick ETL process to refresh some tables or pull some data down from a third-party endpoint?
* Want to expose a quick usage report for the project manager?
* Want to send a slack message when a build finishes from TeamCity so QA can jump on it?
* Want to monitor the rate of Crashlytics logs and send a warning to slack if it breeches a threshold?
There are tons of "one-off" little processes that now just automatically run and don't require server upkeep. Most of them are functionally independent, so having them as separate functions makes sense (when we do need to coordinate, that's done through Step Functions).
Note: I still don't think that they are a great fit for production services (and more fits one-off and cron usage patterns)
Sounds like its working for you, which is great, I guess.
However in general what you're doing isn't devops, its 'devs creating a big old mess for someone else to cleanup later'.
All you're doing is trading long term maintainability and quality for 'getting it done right now'.
Sure; it works. I get it. ...but I've also been on the receiving end when that 2-person team has 1 person leave and gets scaled up to a 5-person team, and you have to a) bin everything, and b) write the entire thing from scratch.
I personally consider it a very selfish way to build things. Me first. Me now. ...someone else's problem later.
There are scalable solutions in this space (eg. pipelines, airflow, azure data factory, glue); doing it using lambdas is... not. ideal.
That's not a problem with severless in general; but what you've described isn't any of the things which is good about serverless either: what you've described is the problem with the low barrier to entry with serverless, resulting in production serverless code that skips past 'quality control' to 'let's deploy straight to production!'.
As someone in charge of cleaning up the mess in a company that has 800+ employees and 300+ developers, I can only agree.
My company's infrastructure was never tended to and was apparently a free for all for a few years. A year or two ago, it was finally decided the free for all needed to end and all the developers saw their SSH root access removed.
My team of 5 SREs is now facing a constant backlash from developers frustrated at the rotten infrastructure, constant bugs, and impossibility to ssh into machines. We're slowly fixing the mess, migrating to Kubernetes and things are starting to get better, but god, how painful it is.
So yea, in its infancy a startup's infrastructure can't be perfect, but every single shortcut you're taking now, you'll pay double down the line. Choose your path wisely.
This assumes that survival is guaranteed if you do it the right way. Long term thinking can work in firms that are larger. For smaller firms with limited resources, motivation and technical expertise getting velocity is crucial. This is one of the reasons startups run into issues when they have to scale. I think it has to be a well thought out balance of speed and quality that is required in such environments.
That's such a straw man, it pains me to even respond to it.
Doing it 'quick and dirty right now with the tools I happen to know' instead of 'doing some research, then doing it in a way that is maintainable using standard tools and taking a bit longer' isn't 'endangering the survival of the company', it's professional level practice.
Failing to do so is, a) negligent, and b) incompetent.
I'm certainly not suggesting the OP isn't competent; they seem on the ball, and maybe serverless is the best choice for them because (insert reasons here)...but I think its fair to say, if you're using your one tool (ie. serverless) for all your tasks, you probably haven't done your due diligence.
Specifically in this case, using a standard ETL tool to do ETL type tasks... would probably be suitable.
Anyhow; no. Long term thinking isn't what this is about; it's about behaving like a professional when you do a professional job.
You pick the label you want to wear as an engineer, because, your practices define you.
You are on such a high horse, it pains me to even respond to you.
You can call it devops, a mess, whatever you want... but nothing about serverless frameworks is creating that mess. A service is a service, whether it's running Lambda, Spring Boot, or whatever the hottest new framework is... Nothing prevents you from documenting a service. Nothing prevents you from creating basic alarms (it's literally easier for me to create these alarms because of CDK than it is to alarm on the standard service framework I use).
We get it, you don't like serverless. But you're going to rag on OP and question their competence, and then drop some "you pick the label you want to wear"? Come on dude, seriously?
You're strawmanning too here... Can you show me where OP said they are using one tool for all of their tasks? they literally said "for one-off, little processes."
I cant imagine what it's like to be this arrogant.
If you read my comment, you’ll realise that I’m not saying serveless is bad.
I’m saying; if that’s your only tool, you’re not doing your job.
“Velocity” is not an excuse for not doing your job to a professional degree. Its a strawman people use to justify doing substandard work (or, often, pressuring engineers who know better, into doing substandard work).
While I think that there is definitely a worry of it turning into a big old mess (especially if they start depending on and talking to each other), I don't really know that I'd consider cron-job style independent one-off tasks any more maintainable in a monolith. I also don't know that I'd consider that specific to serverless (it's easy to create a ball of mud in anything).
All of our serverless projects are located in a single repository, and each little service gets its own little google doc that explains what it does, what functions it has, and any resources that it uses.
As mentioned below, as soon as an ETL gets multiple steps I agree that pure-lambda is not the right tool for the job (Airflow would be my preference, though we've also successfully used Step Functions)
How is owning an AWS account easier than a cheap dedicates server? You should have a code repo and backups in either case.
At least some people mention "but I can scale this lambda x1000" and that's one advantage... But you can do all those tasks on a hetzner server in same amount of type, just python scripts.
Because that "cheap dedicated server" isn't all that cheap. Just the documentation for backups on your Hetzner server is ridiculous. Why, in the 21st century, should I as an application developer ever have to worry about that?
Let's say that Hetzner server goes completely toast at 2am and you get a pingdom message. You log in, realize you have to restore from backup. You fire up a new instance, and go through your restore procedure, and everything works great. You're lucky to be offline for an hour.
I cannot even tell you what would happen if an underlying instance running a Lambda goes offline, because it never happens. But let's say that an underlying machine dies in us-east-1. Lambda just starts running more instances of your code on another underlying group of cpus, and you aren't offline.
Yes, this exactly. The fact that I can guarantee that my code is running, and have entire massive teams at Amazon whose literal job title is to keep my code running, it is such a freeing feeling. I might be able to delete rows from Dynamo from my code messing up, but Dynamo will never go offline.
The psychological effects of technology seem often overlooked—and they’re hard to quantify.
I wish there was some easier way to quantify the “soft” aspects of everything that surrounds us, e.g. the long-term impact of beautiful and usable UI design, the reduction in “ambient” psychological stress, the impacts of chaos/consistency on how we feel, any robust measures of happiness, excitement, relaxation, etc.
They seem to be such important attributes, but not as easily measured as a latency. Which in turn makes reaching consensus harder—and probably why design by committee doesn't work?
My joke around the office is some people optimize for latency, some for throughput, some for memory usage. I, however, optimize for sleep.
My entire goal is for my services to never wake me up. And if they do it had better because I clearly messed up and am the sole human alive able to fix it.
I optimize for escape hatches. Because if everything is on fire I want the ability to be able to bail(fix it).
There's no point in waking up when everything is blowing up if you don't have the access to fix the underlying issue. AWS doesn't actually guarantee 99.9% uptime. What they guarantee is they'll give you some company currency[1] if they can't meet or exceed 99.9% uptime.
So the next time the ship is on fire, don't worry, stay asleep and your Amazon account manager will be by shortly with a thimble of water to throw on you :)
Having been on call for some truly monstrous amount of things and seen things break in all kinds of ways... I can't agree more with this principle.
While the cloud isn't cheap, if your org can afford it, there is no question about it: go with managed services. Focus on building things, leave the OPS to AWS/GCP/Azure.
Tracking exactly what individual db entries ran and didn't run when there's one of these "serverless" "painless" interruptions to an ETL, ensures that you have an endless number of ever-changing status columns and re-run processes.
Are you really sure that it all re-ran? No. You have to query your latest batch - which means you have to have arbitrary boundaries for when failures might have occurred like say, last 24 hours. If was a weekend, just query the whole database looking for any entry that's missing the "final" status success. Try not to run anything while you're doing this, unless your system is tolerant to the DB being locked for a full-table scan.
This is all very fragile and change-averse. Who would choose to run an ETL like this? I have worked for large companies who all end up with this rickety system. Every one, every time. God forbid you want to run tests with a mock set of serverless AWS services.
Here's the beautiful thing about thinking serverless. "Are you really sure that it all re-ran?" Yes I am, because Dynamo streams or Kinesis streams guarantee message delivery. If my Lambda code has a bug in it, which yes sure that happens, I can as part of my exception catching, put that message into a dead letter queue and process it later or report on it. No polling, no querying, I know exactly what went wrong and when.
Meanwhile, I don't have to do things like /make sure the ETL process is running/ or /clear out a bunch of logs when a disk inadvertently fills up/ or any of those other stupid last-century compute problems. It's just a far, far easier way of thinking about application development.
I can appreciate that. A lot more people than you might think, use Kafka or SQS because of various requirements (like that's all their infrastructure supports) and you end up with a massive "source" DB which is a duplication of all-time messages. Such is the reality of a larger byzantine organization. If it can't be audited by someone else, at their own pace, it's not approved.
I generally agree that if you're setting up multi-step or "enterprise grade" ETL processes that it's a job better handled by something like Airflow (we've done a few smaller ones with Step Functions which hasn't been terrible, but Airflow is still better).
Our use of AWS Lambda for ETL is usually just one-off small processes that don't have dependencies.
As soon as you start getting into "This ETL depends on X, Y, and Z being run first" I think you're out of "small utility" territory.
This is so naive. Unless you’re just creating utilities all day long, at some point you’ll actually have a production service that people are using that needs to be monitored, even if it is build out of lambda functions. And unless you’ve run AWS vs colo at scale, you have no concept of the orders of magnitude cost savings on bandwidth (at 95th percentile billing vs per gig) and compute.
I have run AWS vs colo at scale. For the cost of precisely 1 engineer fully loaded, I can have 500TB of bandwidth per month, every month. Guess what, it isn't going to take just 1 engineer to maintain your homegrown worldwide CDN.
Does it make sense for FAANG to run their own datacenters? Yes of course. Does it make sense for you? No. This is what I don't get about arguments for running a bunch of VPSs-- for literally any environment that isn't thousands of engineers, it is more cost effective to let someone manage your underlying infrastructure for you, and any workload at that scale, you aren't contracting with some random bare metal hosting shop.
You seem to forget there is are a ton of people who grew up on bare metal and know it quite well. And they can squeeze far more performance out of it then you could ever hope to see with outsourcing business-critical infra to a supplier who literally doesn't give a shit about you.
Have fun with an army of one-trick pony hires. One old-school diehard computer nerd knows enough to supplant 10 of those fools, but only costs 2x.
And many of them already work for AWS, who I can pay a small premium to to take advantage without having to hire one myself and then worry about replacing them when they decide to leave.
I think the point is sometimes that AWS small premium is really large. It varies a lot. For some the AWS costs are free or tiny. For others, AWS costs are larger than salaries, and it creeps up over time.
About staff, maybe consider finding a good consulting/support firm that can look after your servers for <10% the price of hiring one person, are friendly and on-call, and will likely stick around even as their people change.
That solves the worry about replacing people, as well as the cost.
But as with people, it can take some luck to find a good one :-)
(I used to provide that kind of consulting/support service. Not as much now because there isn't much demand, and development & research work is more satisfying. But still a little, ticking along in the background.)
Maybe it's hubris, but I'm 99% sure I could run a homegrown worldwide CDN for the cost of one engineer salary. I'd be willing to take that bet from anyone who thinks it can't be done.
You're forgetting that a huge part of that is maintaining accounts at a bunch of different colos and maintaining relationships with account managers and shipping hardware and all of the non-engineering work involved in running a CDN.
You'd be spending more than 1/2 your time just on the phone talking to people.
I definitely have not forgotten this aspect of it. Let's just say I've spent a lot of time on the phone for fun
If we want to talk about running a CDN company in terms of lots of additional organizational complexity in terms of customer service it can't be done on a single engineer's salary. My perception of a homegrown CDN is one that rivals uptime and deliverability metrics of existing commercial CDN providers. Within an existing org it can be done. If we're talking outside of an existing org think http://www.nosupportlinuxhosting.com/ to shave costs.
If we're willing to define parameters on what level of service this CDN needs to meet and ramp up time I'd be willing to take it on.
If you are not in the CDN business, why on earth would you want to do this?
This is my point-- I am not in the hosting business. I am in the application creation business. Every minute that I even have to think about modifying /etc/localtime in order to make sure that the instance is running in UTC is a minute I'm not being productive. Multiply that by those million little things that you have to do in order to keep even an ec2 instance running, and it's such a colossal waste of time and effort. Similarly, I am not in the database tuning business. I plan out my data relationships, create the required Dynamo table or tables, and I never need to worry about replication or backups or any of those ancillary tasks that keep me from being productive.
This is a slightly different conversation than the original but I'll give it a shot.
I'm not a woodworker, machinist, farmer, or doing a PhD in literature. Nevertheless, I engage in all of these things. I enjoy crafting things and learning new techniques. I enjoy learning how to optimize plant growth. I enjoy reading old literature and gathering what I can from it about the historical context, linguistics, etc.
All of these types of things, while not "productive" give me additional knowledge that I can pull from in "unrelated" tasks.
In the tech world, I'd say that realistically all abstractions are leaky. So engaging in these tasks(you mentioned) are productive. But even more than that because abstractions are leaky this is knowledge that is useful. If you don't have the knowledge of the systems underlying the abstractions you're using it's a footgun waiting to happen.
I can give an example. I had a boss that was doing some serverless work and he made an assumption about JSON. I told him in review that JSON does not respect ordering of keys. Well, he assumed it would not be a problem. It was pushed to production anyways. And a few weeks later it came back to bite him.
Now what happens when you don't dig into your abstractions? You lack the knowledge to fix problems that happen at a lower level. When you're deploying these "relationships" to your database provider. What happens when the machinery breaks down for replication, backups, etc? You have to sit around and wait for an expert to fix your app. Meanwhile you're staring at your users and stonewalling.
No, you're not in the hosting business, you're not in the database tuning business, you're not in the sysadmin business, and you're not in the Ecmascript working group. Until you are. Then when you're foisted into this role you're a fish out of water. Specialization is for ants, but we're humans, not ants.
Yep, something like terraform + your favorite config management tool and monitoring system would do nicely for a mostly automated setup once done. I am guessing that a really nice setup could be created in under a month.
Depends on scale of course. There will always be some company that isn't of sufficient size to make it worth the cost.
This is how we actually end up having centralized services. The larger companies who have all this infrastructure, and have the size to justify building their own.. well they inevitably ask themselves "Hey! we have all this infra, we have to pay these people anyways, why aren't we selling excess capacity?"
Then you get AWS. Then as time progresses people at even large institutions that could benefit from running their own infra just go and use another company's infra. Where the other company has evolved into being an infra provider instead of a car parts store, book store(amazon), or search engine.
To answer your question directly. Yes I think this can be done within one engineer salary even if I'm including my own salary. There would be a startup cost that eats into that salary in year 1. But it would be manageable. And then iterative upgrades/replacement parts would be a negligible cost ongoing after year 1. With some financial tricks that startup cost can be spread out over a long enough period that it wouldn't even impact year 1.
I wouldn't say I am. But I think the perceived complexity(fear) of managing hardware is overblown. However as time goes on and people become more afraid of it due to the loss of institutional knowledge it won't be an overblown fear anymore. At that point institutions will have no choice but to contract out their infrastructure to a centralized institution or an oligopoly of institutions.
And we're approaching that because as these types of skills are outsourced to these institutions the amount of people with the skills necessary to carry them out are diminished in number from attrition. Amazon doesn't need 10 company's worth of engineers to have the requisite skills. If everyone buys into this model then eventually the market will only bear the number of engineers having that knowledge as Amazon needs.
Nobody’s saying serverless makes monitoring unnecessary. But a serverless architecture gets rid of an entire category of bureaucracy and issues that can arise from dedicated servers.
A very good point. Exchange the risk of a problem taking you offline, losing sales, for the risk of a problem not taking you offline, costing you a fortune.
That's a big selling point for DigitalOcean: you sleep easy knowing what your bill will be. If you have an unexpected spike in traffic--whether a great opportunity, a mistaken test run wild, or a DDoS--it doesn't increase your bill. AWS offers the opposite: no matter what happens, we'll keep you online and just send you the bill.
A middle ground between both would be ideal. Does AWS offer settings to cap your spendings? (I googled and found AWS Budgets but it seems to be only an alerting system, not a cap.)
AFAIK at the moment there is no cap. It's pretty much like going long on a crashing market. However, you can later have a bunch of emails and tweets basically begging to be reimbursed and it works most of the time.
Yes, going with AWS is a short. You're shorting your own popularity. If your popularity explodes upward, there is no cap on your losses to AWS. The theory is that you are also hedged by your business so the limitless losses to AWS are more than compensated for by even larger gains in your business. But if you're caught with an unhedged short....yikes.
The recommended solution is, ironically for this topic, to have your billing alerts trigger Lambda functions you've written to shut down your infrastructure.
My psychological stress is knowing our company is hemorrhaging 50k €/month on AWS fees (mainly Lambda, step functions, and CloudWatch) to run a production app that could easily run on a couple of VMs (for failover) and a managed DB instance.
Edit: add RDS to the parenthetical list of AWS services above.
I like how people here pretend like this is some hard task that should only be done by experts when I know fifteen-year olds who figured this out all by themselfes for backing up their game servers
> Dealing with server infrastructure isn't business differentiating
I agree, for most businesses no it isn't.
However, saving on cloud hosting costs to replace them with a handful of dedicated servers in different data centres can make a large difference to costs, depending on what the cloud usage is. For a funded rocketship startup with lots of free AWS credits it makes sense to use AWS; for a steady state or long-term low income business, not so much.
Those costs can make all the difference to a business if the cloud rental is creeping up to look comparable with salaries.
I've used a combination of dedicated and cloud hosting for a long time. In general the dedicated systems are so much cheaper for the same amount of bandwidth and compute, and you can run VMs on them with great performance, so you can run everything interesting inside VMs and a lot of the arguments around upgrade downtime have completely gone away.
With some cloud to assist, you can even use cloud temporarily to cover the downtime for major upgrades such as replacing hosts or reconfiguring the cluster network.
99.9% uptime with excellent bandwidth is easily achievable this way, and you can completely avoid user-facing downtime due to planned upgrades, with a bit of planning around DNS and IP transitions, VM migration, overlay networks, things like that.
Sometimes at much lower opex cost (100x) than the modern cloud-recommended equivalents, although it takes knowledge and time to ensure it.
If you have enough redundancy and occasional infrastructure stress testing, a similar level of "optimise for sleep" is possible as with cloud deployments. If you care about this (not every business does), it does take some work and knowledge to produce and verify hot redundancy, and of course there are costs.
Things will go wrong, but they'll tend to be at the level of things running inside VMs and containers, rather than hosts failing. The same things which go wrong with cloud deployments anyway.
The cloud can be used alongside dedicated to provide some of that redundancy and extra scaling. Cloud is rented in small units of time, so you can use cloud as "free most of the time" cold-backup service, and if there's a burst in traffic beyond what the dedicated systems can provide, or if you want to temporarily improve geographic locality for a service.
It might seem wasteful to rent larger dedicated services rather than cloud-as-needed, but the cost differential has changed in the last 10 years to favour the former more than it used to.
Are you taking into account the salaries needed to do this?
Even assuming that you save 100% of your hosting costs, you have to be spending quite a lot to even cover the salary of one full-time employee.
I can believe that larger businesses can save quite a bit, but I don't see how it makes sense for anyone spending less than seven figures a year on AWS.
>Just the documentation for backups on your Hetzner server is ridiculous. Why, in the 21st century, should I as an application developer ever have to worry about that?
I dunno, that's a problem for the ops team. It was in the 20th century too. The only reason you would have to worry about it is if you're pulling double duty as under-trained ops. If that were the case the right answer to every "build or buy" question is "buy" because you don't have anyone with the domain knowledge to bring more architectural options to the table than 'half-ass server management with internet tutorials.'
That's why you have three hetzner servers, so if one goes toast, you still have a redundant operation as you can easily failover while you recover the other host.
Mostly it's about the maintenance costs of owning and managing a server -- on Lambda most of these tools are "set and forget".
I don't want to have to manage security updates, disk usage, or any of the other multiple things that come with owning a server.
I also could probably debate on the friendly-ness of just dumping all of these one-off services onto a single server (seems like it could get messy pretty quick), but I think you could probably go either way on that.
We are also an AWS shop for our "production" stack already, so there is no added overhead for owning an AWS account.
One to run the application while the first is being upgraded.
Now you need three servers, one to run the application, one as backup and one as load balancer.
99.99% uptime means you can be down no more than an hour a year. Which is really easy to overshoot when you're dicking around with a dist upgrade on three linux boxes.
So that's why you end up with a server farm starting from your one small server.
Basically nothing I have ever used had 99.99% uptime except maybe the phone and emergency services. Why do you need this level of uptime unless life and limb depends on you?
My bank has regular scheduled downtime, Steam/Dota has regular downtime, entire Microsoft datacenters had issues and had around a day of downtime, Lime, Uber, my mobile carrier - all had downtime or bugs or glitches that were equivalent to downtime.
If you really need crazy uptime, then you must make sure you have no bugs, because bugs cause more downtime than any hardware failures.
That means an approach to software development that isn't relying on flavour of the month js library.
Once you get to 99.9% uptime you’re still less than 1 hour per month. I used to run apps for companies at work and it was standard operating procedure to have two app servers + load balancer for upgrades.
Drain a server, upgrade it, boot it, check that it works, let it accept traffic. Repeat once more. Done.
It may be true that most businesses don’t need the uptime, but I’d say it’s usually easier to give them high availability rather than field complaints when some random user tries to look something up during a maintenance window.
A pair of database servers (leader and hot spare) behind it, although some data was in-app and shared between the two app instances (or more; two was the minimum).
I don’t think of this as particularly fancy or expensive, considering how much you have to pay for engineers and technicians. It was critical enough for >3 nines, and that means having a technician on-call.
I saw this setup at two different companies I worked at, even though the tech stacks were completely different. Whenever I price out setups for similar requirements in cloud I almost always end up with basically the same setup, just on cloud VMs instead of dedicated servers.
Notice that grandparent comment explicitly carved out production servers. If you can't tolerate downtime while you're being upgraded, you're production.
You don't need two servers rented, even for 99.99% uptime.
That 99.99% uptime is achievable with a single dedicated server, plus cloud as a cold backup and some monitoring to trigger spinning it up.
You will spin up the cloud server when the dedicate server is down. It will cost, but only for brief periods of time.
That's sometimes cheaper than two dedicated servers.
I prefer more than 1 dedicated server, but it's definitely possible to get the 99.99% uptime with just 1.
And in some ways the extra diversity of having a different kind of backup is useful, e.g. for burst scaling as well, and for peace of mind not relying on a single provider.
Even better, you can (and should) run the applications in VMs, or in containers inside VMs. They perform well these days, and even a severe application crash won't take down the host.
Then regular application upgrades don't need a separate host server. Instead you're spinning up and shutting down VMs.
You can also migrate VMs between hosts, and this can be done live if you want. It takes some setting up to have working distributed storage and movable networking, but it's possible.
However, you might decide not to bother with that, as starting and stopping instances and having fleets of instances managed by something like K8S is modern practice for other comforting reasons anyway.
Then the only "big" upgrades are limited to host server upgrades and major network reconfigurations, both of which you will do rarely. You might do host server security upgrades (kernel etc) more often, but that costs the time of a reboot unless it goes wrong.
If those are planned or automated, those "big" upgrades can be done with zero user-facing downtime except in the event of failure, in which case it's limited to the cold-spare startup time
So provided you have the backups in place, maybe on the cloud as a cold-spare so you're not paying for it except when it's needed, you can still have that 99.99% uptime for approximately the cost of one server.
You do not need a separate load balancer device for uptime specifically; only if load balancing is something you want anyway for other reasons. For seamless transitions between backend servers, if you have multiple servers running at a particular time, there are other methods.
On occasions you might instantiate a cloud load balancer temporarily to cover downtime during a planned host upgrade that can't be routed around another way. Like with a cloud cold-spare server, you don't pay for it except when it's temporarily instantiated.
It doesn't actually scale to 0. The scale to 0 argument is actually just accounting trickery. Servers are still waiting around burning cycles doing nothing waiting to run a serverless function. All that has changed is that Amazon has spread the cost of low utilization across all customers. Which gets baked into your cost as well.
But not good for production REST API endpoints in my experience. In the context of AWS Lambda, you are talking a 29 second timeout in API Gateway. API Gateway is fairly expensive. Also, is connection pooling a solved problem?
Many times you can do better with an EC2 instance running PM2 and Express, or whatever. Like, what does Serverless Framework do locally anyway? Express? Why not just run Express?
I have a production REST API endpoints backed by PHP on Lambda that are called 12k/sec on the high end. no problems since ~2017.
The best part is, I never have to worry about scaling the underlying resources, if sales brings in large clients. Worst case is I ask AWS for a greater Lambda concurrency limit.
Is anyone else worried about the proliferation of disjoint processes that people can whip up using Serverless? At least with a monolith, you've got a relatively coherent grouped together codebase. But with Serverless, you can have some lambda that is infinitely important, but not documented or well known to anyone else in the org.
If you deploy enough Lambdas that depend on each other you effectively end up with a monolith, but the call paths are done through HTTP instead of method dispatch. Arguably this is worse, since discoverability and modification can become extremely difficult.
I think it all depends on how searchable and generally well-maintained the codebase is. Having some obscure critical function hidden in a monolith is totally possible too.
Serverless and microservices aren’t the same thing. You can deploy a monolith to lambda, and make a thousand small services on VMs or orchestration platforms. It’s purely a question of how to organise your code.
I'd say that is misleading at best. Lambdas are inherently unsuited for monoliths due to payload, RAM and execution time limits and cold start times. Lambdas are inherently suited to "real" micro-services, because by nature can only do 1 thing and have strict limits.
Depends entirely on workload - there's 3GB RAM available, which is more than what we use for normal monoliths on Heroku. Payload limits are 6MB or something, which isn't hit for a majority of sites on the internet (each request, not total page size). And there's nothing in the nature of Lambdas that force them to do only one thing. HTTP events can be fed through the normal router (lots of adapters for many web frameworks out there). Non-HTTP events can either go through a switch-case-board. There's no inherent "nature" to lambda, it'll execute whatever code you give it. There are limitations, but those are constantly changing.
How does a build trigger the serverless function to notify all of you? I kind of assume serverless means it activates on a timer or when you visit a specific URL. So a build script executes it by visiting a URL when it's finished? And the script executes from any internet connected machine?
Why is it better than having a VPS? I currently use a VPS for a few one off scripts. Cron does the timer ones and the URL ones are entries in a nginx config file. Actually that's half true I actually config an app I wrote to do it cause I didn't want to bother finding out how to do shellexecute on nginx
The serverless framework is an open source tool that you can use to "deploy" "serverless" assets. Basically it covers most of those scripts you usually have to write and gives you a declarative format with some great variable capabilities that help you managed config and declare your components clearly. Running the tool deploys to your cloud of choice and results in functions, APIs, workflows, databases, or whatever you declare now existing and ready for use.
That framework is separate from the class of services regarded as serverless. The star that kicked it off was AWS Lambda. Serverless colloquially means you are dealing with an abstract service contract rather than a server (e.g. the oldest is S3). This removes patching and other maintenance that usually does not directly support business value. More formally, serverless includes auto-scaling to zero, paying only for what you use, high availability, and other design patterns most outside the large tech houses cannot use at low to no cost.
Your VPS is always on and always paid for. It can crash or get in a bad state. It can go out of date and need patching. It is mutable and more vulnerable. It is limited in resources.
Nike reported AWS Lambda scaling in production at 20K RPS/S (0RPS@0s, 20K RPS@1s, 60K@2s, 120K@3s, ...).
serverless-artillery, with loosened account "safety" limits can scale from nothing to producing billions of requests per second on target systems.
Full disclosure: I contributed to the serverless framework and serverless-artillery. I'm a biased fanboy.
Serverless functions can be set on a timer exactly like cron jobs. But you can also set them to act on certain triggers, when a message comes into an SQS queue, when a notification goes to an SNS topic, etc.
You can do the same with a VPS, but I think VPS cost more because you pay for all the time you're not using them. With serverless functions you only pay for execution time. And with one-off, daily, hourly jobs you end up paying far less, unless your jobs are compute intensive (cron jobs tend to not be...?)
I have seen some VPS these days come in at like $2 a month though, so it's worth comparing the cost.
That sounds great but if I already have the ability to quickly prototype and stand up one off type services / endpoints in my code base what's the selling point here?
To me it's just full of risk and costs where as adding a new function, getting it reviewed and deploying takes all of a few minutes with none of the cost.
I totally agree with this. My only real complaint is managing these one off items. Team member 1 puts one on a VPC the other doesn’t, another team member ties one to RDS and doesn’t automate auth...etc.
It’s great but you have to put management of usage in place first.
I cannot tell you the number of times I have implemented "upload your photo and it'll get resized to (profile avatar size from design specs)". It's ridiculous, and it's one of those things that everyone burns time implementing their fun hook into Imagemagick. Now I have one lambda that gets pointed at a new record stream from an S3 bucket, and I'm done.
I cannot tell you the number of times I have implemented "when this user signs up, send them a welcome email." It's one of those things where you construct your email, point it at your MTA, do a ton of configuration, then it may work. Now I have one lambda that gets pointed at a new record stream from Dynamo, which calls SES, and I'm done.
I cannot tell you the number of times I have implemented "clear the Redis cache if a user changes their preferences". You write a clearUserCache hook into your DAO, or you paste it manually into your crud functions, and you always forget something, and six months down the line you start getting bug reports of people's zip code not updating, or something. Now I have one lambda that takes record streams from Dynamo, removes a key from Elasticache, and I'm done.
It's not that you couldn't do this before serverless, of course you could and you still can. It's that it makes that level of code reuse that much simpler. You have all of these helper infrastructure functions that you implement for every single project you work on, and reusing that glue code is so, so much easier in Lambda/GCF/AF/etc.
> It's that it makes that level of code reuse that much simpler. You have all of these helper infrastructure functions that you implement for every single project you work on, and reusing that glue code is so, so much easier in Lambda/GCF/AF/etc.
That sounds great until you need to add a feature or fix a bug in the reused code. Then you deploy a change to a Lambda function that impacts X other projects immediately, with no chance to test each of them individually.
Sure you can do that, and that works ok in some scenarios. But there are some problems.
In a scenario like:
> Now I have one lambda that gets pointed at a new record stream from an S3 bucket, and I'm done.
Ok, so you got AWS set up to fire your Lambda when an object is created in an S3 bucket. You decide you need another "stream", we'll call it, so you start dumping stuff into another prefix. How does one go about testing that the right function is invoked?
A smart person will probably say that they have a dev environment and they manage infrastructure with Terraform. Great! That's probably the best solution there is.
But that still leaves a massive, glaring problem: it's quite difficult to implement any sort of automating testing of this Lambda function setup. In all likelihood, you're probably just pushing a file up to an S3 bucket in dev and watching it run through.
Let's say you made a pass at automated testing, and let's continue with the example of creating resized avatar images. The end product of this Lambda resizing process is probably a different file somewhere in S3. So you fire off the automated test and it fails. How did it fail? Well, if you're lucky, the Lambda function actually had an execution that errored out. Then it's up to you, or your automation, to look up logs in CloudWatch to troubleshoot the failure. What if it didn't error out, and instead just put the file in the wrong place?
This kind of stuff is where Lambda falls over. Running Docker images on EC2 in some fashion puts way more sanity around testing as a whole. You have real Docker artifacts that you ran tests in, not just some zipfile abomination that does nothing to create a good local development environment.
I'm not gonna try and say that it should replace containerized applications (I would choose those 9 times out of 10 given the choice). Unfortunately, there are cases where there isn't a choice or Lambda is still a good choice (like maybe the grandparent's image resize thing, probably depends on a lot of things).
I tend to think of a Lambda as a custom piece of cloud infrastructure. So, in addition to unit tests, I just test them like I would any other Terraform module. I use Terratest to deploy a stack containing the resource under test and a surrounding harness. In this case, maybe the Lambda, a source bucket, a destination bucket, a DLQ, logs, etc. Then execute my test cases, poll for results, do assertions, etc. When it's done, Terratest destroys the stack.
I think a lot is hidden inside "which calls SES" versus "point it at your MTA, do a ton of configuration, then it may work."
Email is unfortunately a moving target. What worked before, gradually stops working as the big providers put up increasing obstacles to your own MTA doing a successful delivery.
I've heard Amazon SES also has delivery problems so take with a pinch of salt. But I would hope they generally try to maintain it.
I agree, but then I think then we're comparing using SES (or Mailgun/Sendgrid/Mandrill/whatever) versus maintaining your own MTA, which is orthogonal to the serverless vs non-serverless debate.
It would be nice if someone made a machine that would run all three of those discrete, generalized computing tasks on a single general purpose "thing". That way, you could infinitely scale your work up or down within available memory based on what needed to be done. It would even intelligently free something from memory if it wasn't being used.
I was hoping you'd end with "LAMP" for a second there. Many of the non critical path use cases that come up here sound like something you'd have deployed to a shared PHP hoster at the end of the 90s. Does a bunch of
php files on one of those count as a serverless API gateway yet? For those use cases it doesn't seem all that different, just with added buzzwords, nicer languages and better developer ergonomics. At the cost of replacing skills in maintenance and debugging of what's going on in the stack with maintaining and debugging whatever abstraction cloud provider X put on top of it.
I favorited your comment, but I'll be curious to see how rosy your outlook is in say 10 years time. My prediction: all the use cases you're using it for are not the ones that it's designed for (i.e. which pay AWS's bills). And historically, unintended/illegible customers have a way of being caught out as the vendor shifts between strategies.
Certainly, I hope you can be the remora to this shark for a long time to come. Just be aware of the benefits and drawbacks of the position you're taking.
I don’t think cloud functions in any of the providers are going anywhere. In fact seeing how google is investing in cloud run (docker images as cloud function) and other providers are catching on, this will be a growing trend.
The ability to take whatever crazy code with spaghetti dependencies and freeze it into an image and have a cloud provider auto-scale from 0 to ludicrous in seconds is phenomenal ability.
I love cloud functions. They make the perfect Webhooks.
You misunderstand; I'm not saying cloud functions will disappear. My claim is that the _details_ of how cloud functions work will gradually mutate in strange and subtle ways you can't anticipate today, in a manner akin to the way that Google Chrome's behavior has gradually mutated. Just because successful services will tend to follow their most lucrative customers' use cases. The long tail running tiny lambdas will lose influence over time.
This may take many forms. Pricing models may change. Use cases that gradually see diminishing use may get discontinued (Google Chrome). You might get on some sort of treadmill of having to update details every so often (Facebook API). I can't predict what exactly will happen, but I believe that if your use case doesn't fit "we run a bazillion Lambdas and send tens of thousands of dollars (at least) into Amazon's bank account every month," any service you receive is accidental and contingent.
Sorry but AWS Lambda (and the related event-based constructs on AWS) is exactly built for the use-cases GP is using them for.
I use it for similar use-cases GP pointed out: It is a breeze to setup but the best part is there's no devops, no sre (pretty much set-it and forget-it) which is pretty great for something that'd be highly-available yet not be expensive at all even for the smallest of businesses.
I'd point out two other aspect not mentioned here - first is that the complexity doesn't go away. Application architectures running on a 'serverless stack' have a lot more moving parts than a traditional server based equivalent. Try showing the you from 10 years ago a modern serverless architecture and they'll think you're mad. More moving parts is more maintenance/problems, but on the plus side, does allow you to throw more resources at a specific part of the whole.
Second - with each cloud provider's serverless service, I've noticed that the underlying complexity is shifted into your application and provisioning. They're a great example of leaky abstractions. The name 'serverless' may lure you into thinking that you don't need to think about servers any more, but actually you do have to adjust to the way they've implemented a service that you're using. Think of Lambda cold starts, or have a look at the DynamoDB pricing page. There's still a server running somewhere.
Personal experience: I've found that ECS (Fargate) is a decent step into the serverless realm, if you've been running normal containers on normal EC2s. It does take away the EC2 management aspects, and it's especially useful if you have some applications that are resource hungry, and some are not. It's a cheap way to run ETLs (we use the ECS Operator from Apache Airflow, which itself is on EC2). It doesn't take long to learn, and it's a good place to start. It does come with its own baggage... the 'stack' that you had on a single EC2 is now scattered about; you may need AWS Cloud Map for service discovery, ALBs for networking, SSM/KMS for secrets, Cloudwatch for logging.
That's when you realize, those sticker infested laptops at Devops conferences are actually modern architecture diagrams.
Seconded. Serverless has the benefits and drawbacks of Microservices: you build and deploy smaller units, at the consequence of having to debug the interconnects between everything, with poor tools.
I'm a fan of OpenFaas: you get the benefits (but not all the drawbacks) of containers and serverless. It's easy to mix and match running 100% locally, or mix-in baked 3rd party components, or running in Kuberenetes or other systems in production. Also, no vendor lock-in.
+1 for openfaas. I haven’t used it in production, but the option to run an entirely custom container underneath the function opens up the possibility of running a “proper” application underneath (the use case I’ve been meaning to test is a fully-featured rails app)
Infrastructure is always much harder to maintain (and test) than application code. Serverless architectures can risk moving complexity from (easy-to-test) application code to (hard-to-test) infra.
Hmmm maybe if you're talking about developing from scratch, but most developers have a lot of knowledge and assumptions about existing implementations of infrastructure; also, in many organizations developers don't have to think too much about implementing the infrastructure, someone else takes care of that. And at the very least, there's a separation of concerns: you don't have to think about both at the same time.
Actually, re-reading your comment, I'm not sure what you were trying to say. I may be agreeing with you.
---
I think it is possible for serverless to supplant PaaS et al, but I don't know if it's quite mature enough yet.
For stateful systems such as databases, that typically require complications like replication and backup, configuration, testing and maintenance is hard to get right.
But for stateless systems, IMO infrastructure is pretty straightforward.
I agree. I only generally use Fargate with containers, and control with cron or with always on APIs, with Cloudwatch for monitoring, and it’s seems like a good midpoint between ec2 and serverless.
The APIs cost a bit more a month (they’re on quite low cpu/memory) because they’re always on, but it minimises complexity and it’s nice to know that you can move those containers over to another platform if needed.
I never quite understood what problem serverless platforms were solving for companies that already deployed their own SAAS solution.
As the article says, serverless is one of many (many) ways to wrap a quantum of functionality inside an internet-accessible environment. You could have a chunk of python in a serverless setup, a small flask server in a container in k8s, as an endpoint in a monolith, etc. Each of these environments have difference performance, cost and maintenance characteristics, but I think of the star feature of serverless as the light deploy. For companies that don't have a deploy chain, that makes serverless very attractive, but if you've already invested in one it seems less compelling.
I think serverless products have a ton of potential for side projects and hobby projects. It's great that you can just type out python and access it on the internet immediately! That quality just never seemed to be what was holding back SAAS companies.
Another niche where it shines is the small, low-volume tool. You don't want to give it an entire VM, because that's a major waste. You don't want to have it share one "tools box," because it is awkward sharing like that, security becomes more of an issue, and it generally devolves into a mess. So if you've got other things running on AWS, then AWS Lambda is great for this. Do you need a random web-facing snippet to receive a webhook a few times a day and send a Slack message to your team? Perfect.
Not the core of your app, but a tool of great utility. On the other hand, this isn't exactly the Serverless Revolution either.
I'd add on top of that, that it provides even more value when volume is usually low and sparse, but unpredictable.
If you have a server that can handle up to 100 requests at a time, but you're only getting one or two a day, you could probably save money by switching to serverless. On the flip side though, you're also a bit screwed if 1000 requests all come in at once, since, even if you have some autoscaling solution, it probably won't be able to bring new servers up in time. Serverless provides a solution for that case as well, since you have almost unlimited resources.
But yeah, serverless is a tool for solving a certain set of problems. This idea of the "Serverless Revolution" was kind of silly from the start
That only works if you’re relatively latency tolerant. Serverless systems currently suffer badly from cold start issues, particularly when large spikes arise. This can be solved, but it involves enough silliness that paying for a VM is probably cheaper once you factor in engineering time.
We happily connection pool using sequelize by leveraging the fact that lambdas run in reusable containers.
If the next lambda invocation happens within minutes of the previous one ending, you can carry forward a db connection from the older lambda, no magic needed. Just place your db connection object in global scope (nodejs).
Yeah, they can use the same connection for a while. The problem is that 1000 lambdas cannot use 50 connections while waiting their turn. Each of them wants a connection to the DB server.
Now if you somehow get a spike of requests to some endpoint that goes over your DB connection limit, suddenly all your newly scaled functions fail because they cannot get a connection to the database.
AWS added a service for RDS to deal with this, but it all just feels like a big kludge to deal with a problem that shouldn’t have to exist in the first place.
If you are expecting one thousand simultaneous requests, serverless or not, you will need a db and an appserver which support one thousand simultaneous connections. There is no getting around that.
You will have to analyze your burst rate and the lifetime of each request to figure out the size of your connection pool.
I also am not clear how AWS introduced the problem of not being able to connect to the DB. We've known about the need for connection pools and connection reuse from before AWS was a thing, no?
If your function spends 80% of its time on logic and 20% on database time, with a server app, you can get by with 200 DB connections to handle that throughout, but will need 1000 connections in a serverless setup.
I don't think connection pooling works this way. Happy to be corrected.
A process is given a connection from a pool when it requests the connection and holds it until it relinquishes the connection regardless of whether the process was in the logic or data access portions of their lifecycle. It is up to you as the developer to ask for the connection and to return it on an as-needed basis...Which is exactly what you should be doing in a lambda as well.
Just because a lambda container is persistent does not mean the connection given to it is stuck to that lambda even after that lambda returns the connection.
Lambda does hold onto it in order improve response time. It connects to the database in a handler outside the context of a request, so you do hold onto N connections persistently (at least until the lambda scales down).
With a connection pool, you don’t need to hold onto a connection longer than necessary, but you are right that you could implement it in such a way that you hold it for life of the request.
I've never understood why people who would never dream of using a single-instance application server are ok with using a single-instance datastore. Of course your datastore needs to autoscale just like your application does. Duh.
All the platforms (GCP, Lambda, Azure Functions) have fairly long cold start times, but some platforms let you essentially pay for one or more cold start instances that are always on (I saw Azure had this, others are sure to follow). According to the article this "undermines" the model, but I disagree, as instance start time measured in the seconds is crazy difficult to achieve when doing VM automation. Typically it'll take 5m to get a fresh VM into the load balancer during scaling, could be as low as 1-2m if you do it just right. Buying one or two instances and then having the rest of the requests billed per invocation with a fast scale up time is very appealing.
Exactly our use case, esp. for things that are outside of k8s, for example, standard tasks that every cloud account of ours needs to perform (they are heavily segmented due to regulations) and developers or k8s ops do not get to touch. Almost all are on events or timers. Cronjobs without infrastructure to worry about are nice. Basically I wrote some of this stuff 5 years ago and didn't have to look at it since; it just runs.
> Do you need a random web-facing snippet to receive a webhook a few times a day and send a Slack message to your team? Perfect.
How many people have that use-case but not in multiples? How many of those people have such a use-case but would not be better served by platforms such as IFTTT or webhooks on other services?
To be honest, the serverless environments I dealt with had worse deploy experience than plain servers. There's a bit of a ritual around deploying a lambda for example and it requires a specific packaging process. I can't easily use poetry for python projects there.
I guess if we count fargate as serverless, it's much better for this use case.
While I am not that big fan of serverless projects (I am strongly convinced lambda should be used as a glue to automate AWS infrastructure and any other uses is just abusing it) you can use poetry with lambda.
I'm sure there are other ways of doing it, but the way I do it is using serverless framework (is a nodejs app) together with serverless-python-requirements plugin. The plugin understand poetry (make sure you don't leave requirements.txt because it will use that, you don't need that file if you use poetry).
Anyway it has warts, one big thing is that nodejs developers don't care about norms and standards and constantly reinvent the wheel so for starters you're pretty much forced to create project.json project-lock.json files, which then will create node_modules directory in root of your project. Then you need to start project through npx command (I believe you could still install serverless globally but looks like that's being depreciated).
The node_modules directory will contain over 200MB of javascript code after you install it.
But after all of those things, when you get it to work it is not terrible.
>(I am strongly convinced lambda should be used as a glue to automate AWS infrastructure and any other uses is just abusing it)
I've created a whole photo sharing website in AWS Lambda, including a complete user accounts system (register, login, forgot password, email verification, profile photos, etc), social aspects, as well as other things like youtube video search and transcoding in the same system.
I also had to create my own Lambda build system because what was out there wasn't what I wanted. I wanted to be able to hit save on a file and have it repackage my Lambda on AWS (using a Lambda to create the lambda) including shared code in 'Layers'. Kind of a "live-reload" for Lambda. It's all working very efficiently.
This was a lot of effort though, but now that I've got a basic system I can extend it to any kind of website.
My photo sharing site for friends costs me about $0.25 every few months, and that is mostly/all the cost of storing gigabytes of photos on S3. No, it doesn't see a lot of traffic, and that's exactly why I chose Lambda to build this site on, because I don't have to run an EC2 instance for my photo sharing site 24/7 if nobody is using it. It's worked out exactly how I wanted, costs me practically nothing to run every year.
And, if I did build a system with a ton of active users, Lambda handles the scaling of that. Another reason I spent the time to create this build system and write all the code to handle user accounts, etc.
> And, if I did build a system with a ton of active users, Lambda handles the scaling of that.
This is a bold claim to make without verifying it.
> Another reason I spent the time to create this build system and write all the code to handle user accounts, etc.
But did you also spend the additional time to scale test it and verify it won’t break in confusing and annoying ways when scaling / spiking up? (see the other discussions re: connection pooling above).
The reason a lot of folks are skeptical is that in practice there don’t seem to be a lot of great examples of stateful services using a “pure” serverless lambda pattern but with arbitrary dependencies that actually do scale smoothly and don’t break in annoying ways such that you end up building something more complicated than just using App Engine or Heroku...
I'm not really interested in having the 1,000th discussion about the definition of "serverless". The parent was wondering what the value of the paradigm is, so I provided a list of specific features that many companies consider valuable with serverless offerings.
I hope this "risk aversion" mindset starts to apply to code as well.
Can't wait for codeless, zero code, nocode, whatever name it gets called... something where normal people can build their own ideas without having meaningless discussion about tdd, frameworks and etc, nor the need to maintain codebases, ci/cd and all the jazz
"Low-code" is the buzzword to search for. It's been around a while now.
Of course like all the things the idea has existed a long time, but "low-code" development services seem to be increasingly popular and visible in the last couple of years, and they do seem to be getting more advanced and ergonomic.
Fargate is considered a serverless service. Serverless generally encompasses FaaS + BaaS. Think anything whose infrastructure you can use without touching the underlying VM(s): SQS, SNS, EventBridge, RDS, Lambda, Fargate, Athena, Kinesis and so on.
Serveless platforms like AWS Lambda are great for "glue" between services, or for workloads that are used infrequently. It can also be a great replacement for scripts that need to be run via a cron job (its surprisingly hard to create a highly available idempotent cron system using servers) provided all of the dependencies fit in the small space allotted.
I'm not sure I understand your point about not needing a deployment chain. I developed a framework (open sourcing it soon) using Jenkins Pipeline DSL for deploying Lambda functions and its very useful and helpful.
Regardless of the target environment the reasons for using a deployment pipeline are still there even if you use a server less platform.
One serverless scenario I find useful is for processing messages from low-volume queues. It avoids having to stand up "real" infrastructure that would be under-utilised.
Of course, if you already have servers with something like k8s or Swarm running, you might deploy a container there instead.
I’d be curious to see the data behind that claim that the hobbyist market is overwhelmingly PHP. I must not have any exposure to that ecosystem because PHP does not come to mind when I think of hobbyist development.
Not natively, but fairly easily. We’re using Laravel Vapor - it handles all the setup and deployment of Lambda bundles that include the app code and a PHP runtime. Smooth and fast so far.
Yeah; I'd imagine anyone technically savvy enough to use Lambda + API Gateway, in any form, is savvy enough to bundle PHP if they really want to use it. And if they're not savvy enough to do that, they're probably using hosted Wordpress somewhere and it's a moot point.
Kind of also surprised I think this author has not heard of Google Cloud Run (https://cloud.run/).
Cloud Run employs a vendor-neutral container runtime and API (from Knative open source project). It simply accepts any OCI container that can listen on $PORT number.
Similarly, the author talks about serverless not being able to run "entire applications", which again, doesn't apply to Cloud Run. Many people run fully-fledged .NET Core or Java apps on it with a single command to deploy.
Ditto for the author's "Limited Programming Languages" point, Cloud Run is container-based serverless runtime and it runs any language. Furthermore, you don't even have to write Dockerfiles anymore to build containers for plenty of languages thanks to Buildpacks https://github.com/GoogleCloudPlatform/buildpacks.
Furthermore, in my opinion, the serverless revolution is still going on with more services adding edge lambdas support or storage support for edge workers like Cloudflare did with Durable Objects Beta or Workers KV and there's definitely more to come.
Overall I'm not sure what the author was thinking while writing this. It seems they are aware of the intersection of containers & serverless and note that serverless is not just FaaS, but I note an intentional omission of many products that directly respond to his points in the article.
To me, Google Cloud Run is the sweet spot between Heroku and Serverless. I pay as much as serverless but there's no vendor lock in or specialized programming environments. Fantastic job.
By the way are websockets ever going to be supported on managed cloud run?
Yes! I have an application I can deploy to Heroku, Dokku, Google Cloud Run, AWS Fargate, old-fashioned VPS host, and I suppose a bunch of other hostings like DigitalOcean's newly announced PaaS which I haven't tested yet.
It's just a 12 factor app with a Dockerfile.
What makes Google Cloud Run special between all these is that it's fully managed with an easy auto-scale model and only charges you only for the time requests are being processed.
From an evolutionary point of view, I find serverless appealing and exciting - a romantic idea.
However, the frequent posts along the lines of "my Lambda function was buggy and now I owe Amazon three billion dollars" definitely kills the romanticism.
Isn't there some sort of security built-in, like "if suddenly the bill becomes 1500% the norm, kill the instance and start serving 404s"? (I know nothing about cloud, honestly curious)
No. Amazon very much prefers uptime over cost cutting; they rather have your website up than read a story about a bill limiter taking you offline. They also tend to be quite lenient on unintended bills.
Additionally, bill cutting is not that easy. Should they shut down current services? Stop sending in the middle of a newsletter? Delete your S3 Storage? Even then, a large company might accumulate hundreds or thousands of dollars within a second ; even if they decide to cut at all cost, minimal delays might break the limit.
I fully agree that it sucks for experimenting; it is very much the reason I don't have an AWS account. But I can see why they don't have it.
Also, Amazon bill’s monthly. They might be able to charge by the second, but they don’t actually have an accurate to the second running count of your total bill.
It's in cloud vendor's interests to charge more money regardless of mistakes. They do offer charge limits, but they don't want you to use them.
There are lots of posts online that document "talk to amazon and they'll take out the extra charges" but there are even more cases where the people just pay the bill and thus amazon makes money and profit.
Nope. As the GP mentioned, billing is delayed, but you could set alarms on your lambda usage. You could even use something like X-Ray to get more insight into what your usage looks like.
The problem is there's not a clear way to tell the difference between recursive calls and heavy traffic.
AWS components are fundamentally a network of nodes that are sending traffic to other nodes. But there's no unifying language that can model this, so you can't describe what this graph ought to look like. (e.g. you want to be able to say "lambda Foo should be called twice for each event to Bar".)
And that'd be hard to do since most nodes are general purpose computers, like Lambda and EC2, that can send traffic anywhere for arbitrarily complex reasons.
Since you can't describe what it ought to look like, trying to alarm on it becomes a problem of detecting anomalies. You can sort of do it, but AWS doesn't want to do it because they'd be making a promise they couldn't generally keep.
I think it's more a case of it coming full circle.
People realising that it's just another tool in the toolbox rather than 1 tool that can replace their entire toolbox.
Much of the cynicism seems to come from that.
I do agree with jason though - open ended serverless things charged per use are no fun. Now you can bankrupt yourself at scale with that bug & the 1000 instances.
People place such high expectations for new innovations solving all of their problems. No, it isn't a panacea: apply it only when it makes sense to do so.
I'm yet to meet such people in real life. At places I work people always treated serverless as a tool. Usually solutions would have a little lambda icing on a large monolith cake, so to speak.
I was baffled, bemused even, by the article's premise that anyone rational ever thought serverless was going to take over the software world. It's a tool for solving a certain niche problem, not a general replacement for the vast majority of software we run. It's quite a good example of both the potential benefits and the potential difficulties of working in the cloud.
Whenever I read these articles, I always wonder what proportion of tech buzzwords from the past decade are just obscure ways of saying "someone else's servers", "automation" and "unnecessarily complicated architecture".
>Pay per request needs to be within 2x magnitude of an always on server.
Are any of them? Stuff like cloud run is very competitively priced when assuming base case...but by its very nature it can scale up near infinitely. And with it bugs & bills.
The ability to cap things by 2 order of magnitudes would make me sleep much better. (1 would be better).
They don't, they are all crazy priced which means the cloud providers don't really know what they are doing, forcing people to deploy their own FaaS layers inside their cloud infra, which is a totally valid solution.
Currently, FaaS platforms should be viewed like a cron. The majority of the benefit is if the fn is called in the low thousands per day.
I've seen a few serverless projects that are really, really jacked up. 1000s of functions glued together with little intentional design... basically one big pile of spaghetti code.
Things like the Serverless framework[1] help, but ultimately today's serverless architectures feel like they forgot Dijkstra's warning: that the "Go To Statement Considered Harmful." [2]
I'm still big on the approach for the right use cases: variable work loads are a good example.
But ultimately, to be truly useful, we need to see either a) better coupling with non-serverless approaches, or b) a "serverless 2.0" approach that builds in the good parts and fixes the bad parts. (But please don't call it serverless 2.0... those of us who lived through "Web 2.0" will thank you)
AWS Lambda can run all programming languages via layers.
Cloudflare Workers can run all programming languages that compile to WebAssembly.
"Vendor Lock"
True, but being locked into Kubetnetes isn't a cakewalk either.
"Performance"
Cold-starts aren't a thing for Cloudflare Workers and can be mitigated for AWS Lambda. AppSync and API Gateway don't even have them if you directly integrate with AWS services.
"You Can't Run Entire Applications"
You can and it has been done multiple times. Sure, you shouldn't blindly build everything with Lambda and API-Gateway without measuring. But many services, especially Cloudflare Workers are quite cheap.
K8s can go almost anywhere any
time with fairly minimal ramp. With the frequency I am hearing about AWS billing disputes these days it's probably best to not seal the keys to your kingdom in their service.
But how many hours/days/years did it take you to learn how all the parts move before you became capable of setting something like that up within a few hours?
Anecdotally, stories of businesses abandoning entire cloud automation projects because they wasted weeks and never had anything to show from it don't seem unusual, so evidently a considerable amount of knowledge and skill is required to get value out of these tools.
My worry is less about getting it set up to run a container and more about securing it, monitoring it and being able to respond should things go wrong.
> AppSync and API Gateway don't even have them if you directly integrate with AWS services.
I was quite impressed when I exposed Kinesis via API Gateway, it just worked, although the Kinesis requirement that data be b64 encoded did require some fiddling about with mapping templates.
People always think serverless mean FaaS, while in fact you should use FaaS only to glue together other serverless services when it's not otherwise possible.
Sure, a container is better than a serverless function in many ways, but the function isn't what the container is competing with.
I am still on the serverless high, and I don't see it going away any time soon. For the large corporation that I work for, things like soc2, patching, server maintenance, etc. are a giant pain every single month. The quicker we switch as many things as possible to serverless the better. One of the services I converted over went a year and a half with absolutely no attention from our engineers. It was great seeing how old the latest deploy was. Coupling serverless with terraform and git makes everything even easier. It means doing things like adding another lambda to an API as simple as copy and paste terraform, replace phrases in terraform, write code. terraform apply and done.
How does moving to 'serverless' change the need for patching and other security maintenance? You've just moved it to a server owned by someone else. Sounds more like passing the buck rather than ensuring security.
So for example, if we have a service deployed in an EC2 instance, we have to update the instance(s) every month by a certain date. Sometimes, we have to update early if there is a severe security issue. We also have to manage things like security software that has to run on every instance, logging for soc2 compliance, access controls, manage ssh keys, etc. etc.
That's a lot for our engineers to handle every single month. A lot of context switching. It's not so bad if we only have one service to manage. But the actual issue is that we have many services to manage. So every month, our board had about 12 stories related to patching. Patch service X development. Patch service x staging. Patch service x production. Patch service Y development.... and so on.
For serverless, the team responsible for SOC2 compliance gets the reports from AWS without us ever having to do anything. No patching stories, no security software that breaks on occasion, no access management, etc.
All we have to worry about in terms of SOC2 compliance are the basics like principle of least privilege (i.e. lambdas do not have full access to DynamoDB), no public endpoints, we use VPCs blessed by the security team, and that's about it. Things we're used to doing everywhere else and things that we only worry about when writing the terraform.
Now when I add a Lambda, I only worry about writing the Python code for the lambda and copying the terraform code. That's it. There is no patching story for this service.
And yes, of course we're passing the buck. That's kinda what we pay for in the premiums that AWS gets from our massive bill every month. That's kind of the entire point for cloud-based systems, anyway. We're passing the buck on a ton of other things, like not having to manage hard disks, not having to manage network cables, not having to manage a building, not having to manage HVAC, not having to manage backup batteries, etc.
I guess in places where I've worked we've had Ops folks that handled things like access controls, managing secrets, servers, network, etc. Usually in space rented in a commercial datacenter, where the datacenter handles power, cooling, physical security, and where the building has a diversity of internet providers. Its a team effort. In my opinion, software developers shouldn't be responsible for OS level patches, OS access controls, etc. It makes sense in a startup situation but it doesn't scale and it only takes one misstep.
How do you use Terraform with Lambda? That's been one of the huge painpoints in my limited Lambda experience. All the CI tools (SAM, Serverless Framework, e.g.) seems to want to handle the IaaS part themselves and step on Terraform's toes. I just want to set everything up with Terraform and then have a nice simple tool to build and deploy the application.
We use terraform and terragrunt by themselves to deploy lambdas. No frameworks. It's slightly more work, I guess. But deploying and understanding how deployments happen is easier this way. Plus, getting any kind of framework approved in a corporation is annoying as heck, whereas we already use terraform and terragrunt.
The biggest annoyance is unit testing, although it forces certain good habits on us. So for example, since we don't use SAM, we have to do unit testing separately (our code is in Python). But because we can't exactly run a lambda via Python, we have to put most of our code in the layers that our lambdas import.
So we run unit tests on the layers, and the positive is that we can essentially copy and paste the code that our lambdas would run into the unit tests. And then we can keep it DRY because almost all the code is in layers, and that incentivizes us to create generic functions that can be used across all of our lambdas.
For things like unit testing DynamoDB, we use Moto which works exceptionally well.
We have build our apps and create a lambda zip file and deploy it out of band of our Terraform. We just upload the zip file to S3 after deploy and if Terraform runs and makes a change to the Lambda, it deploys the most recently deployed version automatically.
We did this so we could totally separate our applications and the infrastructure as code stuff since infrastructure changes at a drastically slower rate than applications.
Serverless computing refers to an architecture in which applications (or parts of applications) run on-demand within execution environments that are typically hosted remotely.
Like CGI or FCGI programs! But with YAML!
FCGI really is a "serverless" environment. Launches an application when called for. Starts more copies of the application if there are enough requests. Shuts down idle copies when not needed. Starts a fresh copy if one crashes. You can even scale with load balancers. Really, that's most of what you need to get work done.
That's still available from many hosting providers. It's very cheap. I run Go programs that way on Dreamhost. You're not stuck in the Perl era. I compile the executables on a Linux desktop and push them to the server. Costs under US$10/month. You can probably do 10-100 transactions per second.
I don't really understand serverless, but if it's running a process in response to a request, then of course there's still a server, because something received that request.
FCGI has a server running. And serverless has a server running, as long as its listening for requests.
Serverless architectures rely on some sort of gateway run by their cloud provider as their always on listener which spins up a process for your function when needed.
Essentially it’s a collective pooling which is cheap enough that it can be offered for “free” to the entity creating a serverless workflow.
The Serverless Revolution is still happening all around us, with products like Netlify and the new DO App platform, not to mention all the enterprises moving into the cloud and moving more of their systems into managed products, using serverless components to tie it all together.
Serverless is part of many different mini-revolutions that have been going on for around the same amount of time, with micro services, rebalancing to have more processing on the client side, managed cloud services within private clouds, and the no-code/low-code movement.
Some applications aren't meant for serverless; they need containers or even on-prem on occasion. But most applications are clients with some basic APIs, and many of those are being thrown up on Netlify and AWS and no one outside of the developers are really noticing.
Most of these seem like fairly weak reasons at this point.
Some degree of vendor lock-in is pretty much inevitable no matter what solution you go with.
Limited programming languages may have been an issue a few years ago, but with the support of Java, Go, Python, Javascript to name a few - most of the core user bases are covered.
Additionally, after the introduction of API gateway, most CRUD applications, including those that require async tasks, can be served by the lambda and serverless.
After having used serverless for a while, the biggest turn off for using it (still) seems to be the performance issues. Provisioned concurrency, i.e leaving function containers running, is really just a bandaid and runs counter-intuitive to the original motivation of serverless to begin with.
Serverless is really just a specialized niche of PaaS. Smaller lighter functionality akin more to scripts rather than full applications, with more billing and scaling flexibility, and typically leveraging other cloud services to get things done.
Another big reason it's stalled is because PaaS has rapidly improved and now deploying a Docker container running whatever you want is just as fast and easy. No need for all the vendor lock-in, frameworks, complex environments and everything else when you can just package up a typical webapp and run it anywhere, even with the same billing (like GCP Cloud Run). In return you get a much better dev environment with all the existing tooling and best practices.
Exactly, your software is still running on servers but you no longer worry about managing those.
The problem with PAAS and serverless until recently is that they still required a lot of devops activity. Most of that is pure drudgery: setting up networking, dealing with vendor specific weird shit (e.g. amazon's IAM makes everything harder than necessary), micromanaging instance types and trading off overpriced vcpus vs small chunks of memory, etc. Lots of things that you can do wrong, lots of subtlety, lots of poorly documented gotchas, lots of potential bugs, etc. And all of that is needed so you can say "go run this over there" where this is your packaged up docker application and there is months of work by some poor devops person to turn a large amount of poorly integrated tools into a vaguely coherent deployment experience. It's never simple. It's never cheap
I had a decent experience with Cloud Run recently. I was not in a mood or position to reserve 3 months out of my schedule to terraform myself a new server environment (which I actually know how to do). I already had the software and I wanted it running ASAP. So, I felt slightly dirty when I did this but got the job done in a few mouse clicks in their UI. This was shockingly easy after basically spending months piecing together arcane crap in AWS in my previous project.
This got me a cloud run deployment, a build pipeline against my github repository for continuous deployment, and a running service. I tuned the build file afterwards to do more than just docker build & deploy but that was relatively easy (similar to how most CI systems work these days).
They charge for cpu/memory used by requests. The whole thing has been running for a few months now. Less than 4 hours of work to figure it all out from me knowing absolutely nothing about Cloud Run to me having a service up and running. I reconfigured the deployment via the UI a couple of times to fiddle with memory and CPU settings.
I suppose I could sit down and terraform this at some point but I don't feel the need to do that right now. I have more interesting things to do. And technically me spending a day to do that would cost more than running the whole thing has cost so far. That's the point: the cost of devops is out of wack with the running cost for a lot of this stuff.
This is not helped by the fact that cloud providers are approaching (and even in some cases surpassing) the complexity and price of running standard compute instances again.
It can be a lot trickier to navigate the nuances of ECS+Lambda than it is to run standard EC2 instances.
Even GCP is not immune, though I find it better in these areas, a common problem I have with cloud run for instance; is container instances which don't have permissions to talk to their linked/associated database, instead you have to instantiate a credentials file programatically inside the container.. which seems superfluous when you're the platform provider and you control both resources.
Serverless is the new microservice. Fortunately, for the most part it’s fallen flat on its face and for good reason - most companies don’t need it: it adds unnecessary complexity for minimal gains.
It‘s not a new microservice, it’s a new name of grid computing with more than 20 years of history. I have been working with a grid in ~2001, running Java-based GUI apps on thin clients from a distributed cluster of several sparc machines. Was an interesting concept with unclear economics (likely useless with the internet connection speeds of that time). It’s not difficult to imagine it as an alternative to GSuite with much less resources required on the client.
Complexity always exists. Sometimes it just exists in the interactions between what you think of as your application pieces (most often the case with microservices and lambda architectures).
That being said, it's always possible to add a bit more complexity if you're not careful.
They are helpful in probably 10% of cases where they are indeed the right tool for the job and the benefits outweigh the downsides.
For the majority of companies out there it's just a tool to overcomplicate your stack and turn it into an engineering playground so you can justify 2-3x the headcount despite no significant productivity increase. But hey, at least your company can now be giving talks about how they solve their (self-inflicted) problems managing all the microservices and throw that buzzword on the careers page.
People on HN seems to believe that outside of Google it is impossible to have problems that are complex and high scale, therefor if you use microservices and/or serverless and/or nosql etc etc it's just that you're following the hype, not that you legitimately benefit from them.
I think people are jaded because a lot of these use cases stem from ignorance and/or incompetence of the underlying tech. The majority of people arguing the benefits of NoSQL aren’t professional DBA’s, they’re people that don’t know how to use EXPLAIN. The majority of people touting the “infinite scale” of serverless haven’t even benchmarked their server, never mind optimised it.
Prove to me that what you’re trying to do can’t be solved with a Python script, Postgres and a beefy machine. I don’t care about the theoretical benefits of whatever you’re pushing, I want hard numbers. Unfortunately the community has spent quite a lot of its time and effort these last few years making it easy to unnecessarily scale horizontally. I want no part of that.
I think people simultaneously underestimate how many jobs out there are extremely low volume, and also underestimate how many are high volume. I'm in a data analytics space, every product in my space is talking about 10's or 100's of terabytes a day per customer, and running many different forms of analytics on that data as quickly as possible.
It's always so strange, being in this space where everyone is dealing with tons of data, and hearing constantly on HN about how "you don't need this level of scale unless you're Google".
A lot of this just seems to be that Kubernetes works really well. Serverless seemed more compelling to me 5+ years ago.
But nowadays, I just setup my K8 templates with auto-scaling groups on GKE, and it pretty much works just as well as serverless was promised. Compute demand gets seamlessly transformed into compute supply.
I almost never have to think about managing individual hosts. And the cost is cheaper and performance is more predictable than serverless. Plus it's mostly all platform agnostic.
The issue is that they represent in some manner an 'alternative architecture' that's less mature and hard to piecemeal break out of.
You need to move a few pieces around.
I don't agree that 'once you've gone K8 why bother, just use that' - I don't think K8 is as elegant as the promise of serverless, and it has it's own 'lock in'. Just so happens you may need to have K8s anyhow ... so the pragmatic question then is 'We already have K8s because we have to have it ... so in that context, we can just use it'
If we had a 'severless' version of Docker, i.e. some de-facto standard for it, I think it would obliterate a lot of architectures, just because the promise is powerful.
Developers don't actually want K8s or even true DevOps complexity, we just want a giant computer we can run stuff on and not have to worry.
K8s lockin isn't the same. You can run it on any cloud, or run it locally. None of that is true for serverless functions (unless you go for knative, which is, yep, built on K8s).
Ah. I've been on the road all day and caught this thread too late.
The reason that the "Serverless Revolution" appears to have stalled is that the customers who have the most to gain from serverless technologies have the least ability to recognize that the technologies they're using are actually serverless. Nor do they care.
In its most reductive form, "serverless" just means "SaaS." And Shopify store owners, for example, don't care about how many servers they run -- they care about how many snowboards they can sell. They could maintain one, twenty, or twenty thousand, or zero -- if the cost to run these things and provide value on top is abstracted away by a few tools it doesn't really matter. So you can use i.e. AWS Lambda to solve their problems, or you could sell ShopifyStoreManagerPlus but they don't care what it's called. They just care that they sold more snowboards.
So "serverless" stalled because the target audience doesn't actually care about the implementation, so the smart companies selling "serverless" solutions all dropped the lingo. AWS Lambda is just another tool in the toolchain; an engine and crankshaft instead of a horseless carriage.
I recently ran into a scenario where using serverless caused a cascading failure. We have database searches that are executed directly via AWS lambdas. When we sent too many queries in too short of a time that took down the database and which then took down other things. (The fact multiple things are connected to this database is another thing to solve.) While the database was down the lambdas were still sending more and more requests.
If this was a standard service the standard service would have became overloaded very quickly, allowing the database to remain up but at a degraded performance and the only service that would have been down would have been the one that was getting too many queries. Instead, everything was effected.
I've came to the realisation that if your lambda requires something else to operate, lambda is probably not a good solution.
I find that this is such a bold claim, when the original study cited only states that 63% of respondents did not plan to deploy any microserve-based architectures in the upcoming years. However, 37% do. That is a LOT. And there is not a single data point relating to the years prior, so a claim that the "revolution has stalled" is actually impossible to make, because we do not know if there ever was one.
Micro services are one of the most thorny naming issues in CS. What exactly do you mean by “micro services”? The breadth of definitions for that specific term means that by some counts, there is basically no company in the world that doesn’t deploy some micro services, or there are basically no companies that do. Depends on who you ask.
Because it sucks. We’ve been working with serverless for 2 years now and we’re still dealing with issues for which the only solution appears to be to move back to ECS.
Serverless computing has been around for a while; it used to be called "shared hosting with CGI."
What's new is that it's become proprietary, instead of being based on open standards. This has made it hard to test serverless apps offline, and made it nearly impossible to move a serverless app from one cloud platform to another.
Interestingly what ultimately sold me on a serverless deployment for one of our apps was security. When I compare the attack surface area of us deploying our own OS in the cloud vs just a handful of lambdas, it seems like a great way to let Amazon do the security work for us. If we were doing more in the cloud it may make sense for us to own that, but otherwise it would require us to spin up a whole set of dedicated devops just to maintain it and monitor it that we can avoid with serverless.
I don’t believe that the FaaS revolution has stalled, though I fear it will follow the tech adoption curve because people are over hyping it.
I do wonder if AWS’s leadership in the area has stalled though.
GCP has innovated with their hybrid Cloud Run to allow running more traditional apps in a serverless way.
Azure and CloudFlare have paved the way to what I believe is the future with stateful functions with Durable Functions and Durable Objects respectively.
Meanwhile Lambda just seems to be gold plating it’s (admittedly great) but stale offering.
As a data guy, I've have an entirely different perspective on serverless. While it may not work well for running entire parts of an application, it works great for building out and managing internal data pipelines. You don't want your pipelines crashing because your jobs overlapped and you ran out of memory/cpu on the servers you set up. You want each stage of the pipeline to run independently and scale as needed.
In my eyes, the bigger problem with many FaaS products (which most equate with serverless) is the barrier to entry. The setup isn't very user friendly. The code doesn't "just work" like it does locally. Limitations on data size, runtime, etc. cause you to building workarounds in your scripts just to get them running. Not to mention that once you have it all set up, visibility into everything running is a nightmare and only available to the most technical users.
Based on my experience, I'm currently building a [platform](https://www.shipyardapp.com) to try and make serverless data pipelines easier for teams to setup and manage. Would love to hear someone else's perspectives on serverless setups for data management. I know I'm not alone on these existing frustrations.
Interesting, I have found AWS Lambda quite poor for data pipelines.
Batch jobs can't exceed 15 minutes. Memory limit is 3GB. Payload sizes are 256kb max.
If you are used to batch processing large amounts of data, Lambda seems to be the exact opposite of this. I think it is good for "small data, highly concurrent event processing" but this is a very different use case from batch processing data pipelines.
My workplace uses a lot of serverless features on AWS - Lambdas, SWF, SQS, API gateway, as well as auto scaling instances on ec2/eb.
Some of it’s nice. Most of it isn’t. Here are the pros: it’s great for quickly scaling up or down, and we don’t have to worry about physical infrastructure. Here are the cons:
Documentation is inadequate. The docs are often verbose, but outdated or incomplete in critical ways. There also seem to be very few resources/blog posts about how to use big aws features. I suspect there is a secret cabal of infrastructure engineers that hoard this information.
Replication of work is difficult. So much is done through the GUI. This seems really amateurish. How do you quickly capture the state of your entire aws configuration? How do you record and track changes? We have very little insights into our system.
Every microservice is a snowflake. Why do I have to use this apache templating language for api gateway? Weird. Who designed these libraries? Why can’t I rip as much as I want out of a queue? Weird.
If you're doing anything through the GUI, that is absolutely problematic - for all the reasons you mention. You should be using CloudFormation or Terraform exclusively and configure nothing through the GUI.
The Apache templating language is awful, no question.
- performance: first rule of architecture is that you don't build the entire thing on the needs of high performance...you only address performance as-needed
- vendor lock in: this is the worst reason. Most companies choose cloud vendors and stick with them over many years. I'd love any survey that showed some massive migration between vendors on a regular basis, but it does not exist.
- the monolith: granted, serverless is really best when building something from scratch. Trying to "migrate" to serverless is just a bad idea. Trying to "upgrade" portions is okay, but do it well using the autonomous bubble pattern, making sure you have a solid translation layer
I believe serverless is our best modern architecture, but I also understand that not every company is ready to do the work and/or pay the price of the change in development processes. I'd agree that it's stalled, but moreso out of fear and ignorance than anything else.
>"I'd agree that it's stalled, but moreso out of fear and ignorance than anything else."
How about simple common sense instead of "fear and ignorance". I have a C++ monolith that serves me just fine already for years. The performance is great and I am able to rent dedicated servers that can serve at least 10 times more requests that I have at a price that is a tiny fraction of AWS for the same work.
Why would I switch and incure more expenses? What's the ROI?
What change? I am not going to worry about some abstraction. You have to be more particular. In my case I change features on on need basis with no problems. Hence I have 0 issues with the monolith. Micro- services actually do not really make life any easier. All that immense cloudy push is driven by handful of very big corps who'd build architecture for scale that most of the businesses would never even dream to approach. So I'll let them worry about their problems and I'll worry about mine. I am not in business of transferring my money to cloud paying 10x and more comparatively to my current solutions.
> vendor lock in: this is the worst reason. Most companies choose cloud vendors and stick with them over many years. I'd love any survey that showed some massive migration between vendors on a regular basis, but it does not exist.
Why do you think companies stick with vendors for so long instead of switching?
We’re not buying into IBM mainframes anymore and although there are still customers tied to Oracle databases, vendor lock-in just isn’t that big of a deal anymore.
I guess serveless isn't any faster to debug than restarting a Tomcat locally once you factor in the cloud deployment dance and network latency. In fact, it's Websphere all over again...
I've read through a bunch of comments but don't see anything to answer this question.
Is there a way to use one "serverless" framework in one place and then use another in another place?
For example, I love dokku. It is so easy to have the heroku like experience on the cheap.
But I wonder: what happens if I need to scale my application bigger than the dokku server it's on right now? I really like the immediate prototyping possibilities of dokku but the long term plan seems less exact when I read their documentation.
So, is anyone else using dokku locally and Google Cloud Run globally? Or another combination?
This is why I dislike firebase (maybe this has changed). It's incredibly powerful, magic. But, you have to go all in both locally and globally, and there were gaps that caused me a lot of pain.
The tendency would be to use a framework that gives you the best of both worlds, local/dev plus production.
But, why couldn't we use the best solution in one place and the other best solution in another.
That seems like an opportunity for a smart PM to build on an awesome development environment. I wish, for example, that the Google Cloud Run people would talk about that. They probably need to plan for world domination and have a roadmap that includes their own way of doing it, sadly.
Caching is another big one. Beyond the simplest of services you typically need to cache some info in memory, create connection pools, etc at startup to improve latencies. With serverless that is not possible without killing your cold start time. So it's hard to go serverless except for the simplest of applications.
My company tried really hard to make serverless work on a new part of the platform we rolled out. Our cloud team and management drank the AWS koolaide on serverless and went 100% all in for all new use cases implemented in this particular platform. It was probably one of the biggest mistakes in the history of the company because doing serverless right is at least as much work and running workloads on well-managed servers (which we have). Management won't resource it appropriately because the whole point of higher costs in serverless is to eliminate Ops budget in the first place.
As for running actual code, many types of workloads currently do not map to serverless at all and you won't find out until you hit the limits. Anything that needs to read in a lot of data from a file or do a lot of in memory computations are a lot less efficient or impossible in Lambda given the memory constraints. If any single invocation of your lambda needs more than the alloted memory, it'll just fail and you'll either need to implement some heuristics based routing to the different lambdas (same code with the appropriate amount of memory) or you'll just eat cost on every lambda invocation. You'll find you end up doing things like re-writing lambdas from Python into Go to make it easier to fit more in memory. How is that not insane?
Although it's not 100% true anymore, at the time serverless development was often easiest using SAM which ties you directly to AWS CloudFormation which is the biggest mistake any company can make. CloudFormation is easily the worst thing that AWS has ever created. It's fine for infrastructure with lower rates of change but for applications it's an absolute nightmare to the point where it comes up in every sprint retro on some of our teams.
I /won't even go into how crappy testing is. It's a complete afterthought and it shows.
That team has started moving towards ECS/Kubernetes and even running Docker containers directly on EC2 instances in some cases because it's actually saving them time over futzing with Lambda and it's enabling them to do more faster.
I see serverless working really well for a few niche tasks. Finite, stateless interactions that don't happen often. A good example will be a site say example.org that has a donate page. On form submit, a lambda can take the payload and create a subscription at a payment service like Stripe or Razorpay. The donate page at https://internetfreedom.in works the exact same way.
The broader idea is a service (or static site even) sending messages to a different service. Given how popular no-code platforms are getting, tiny serverless pieces can help automate operations between these platforms.
The things that put me off were hard resource limitations (e.g. runtime/memory), idiosyncratic vendor set up and vendor lock in. The former ends up making the latter so much worse.
The wins also didn't seem great enough to overcome these twin beasts.
An interesting rant to be sure, but what the author misses (in my opinion of course) that serverless isn't new, it just isn't recognized by a lot of programmers for what it is.
Serverless computing is exactly mainframe computing. And by exactly I mean exactly. You see when I was taking CS classes for my degree students bought something called "kilocoreseconds" (kCs) and they are exactly what they sound like, they are a unit measured in the combination of amount of RAM + the amount of compute per unit time your project consumed of the shared computing resource, the mainframe.
You also got a quota of storage with which to hold your projects before you "launched" them on the mainframe with the all powerful "run" command :-). For those of us who liked to explore, it was a very real possibility that you would have to buy more kCs to get through the semester. There was vendor lock in because Doh! you weren't writing operating system code, you were writing application code.
And once you make this connection, you can see all the problems that "serverless" has are all the exact same problems mainframes have for all of the same reasons.
Basically mainframes are "good" when you want to have a small team do all the compute resource management and you have many teams that will want fractional usage of that resource over time. You can developed a closed form solution to the question "what does this compute resource cost to operate per unit time" and you minimize all of the things that affect that cost. Everyone maintaining it are all in one building, all of the equipment is in one room, all of the maintenance is logged and verifiable, etc.
When do mainframes kind of suck? Well when you don't know whether or not you want to use them. Or when you have a different set of OS services you, and only you, need but burden every user of the mainframe.
It is hilarious when you think about it, imagine this giant homogeneous blob of compute, the mainframe, that people disliked because it was "far away" and they didn't have any control, so desktop computers became a thing so that everyone could have their "own" computer, but then networks got bigger so people started virtualizing their desktop compters so IT departments started building bigger and bigger computers to hold many VMs and then because people figured out they could scale by using someone else servers, they started putting their VMs in data centers with a bunch of other company VMs and the data center people optimized by making bigger and bigger data centers, until even the lower cost of a VM was really more than someone wanted to pay for just running a single program so the Data Center types built a layer over all of their machines and services to make it seem like one giant machine. Ending up, nearly exactly back where they started except that now one company owns a mainframe and dozens of companies use it to run their bits of code.
I literally laughed out loud when a low employee number senior engineer at Google proudly announced to the folks that Google was building this new concept of "Data center sized computers." This was like 12 years ago so I'm sure their vision has evolved but it never occurred to them (I checked) that maybe everything old was new again. But I still remember touring computer rooms that took up an entire floor of the building and were basically one computer.
When you get this, you can then go look at where mainframe computers are today. Look at how the IBM Z series can slice and dice itself into logical partitions or LPARs and dynamically become a compute nexus of just the right size for your application, in either bare metal or OS based versions. That is where "serverless" should be looking for inspiration. Modern "web 2" architectures still partition various services into clusters. So you have the EC2 cluster, the S3 cluster, the Elastic Search cluster, Etc. Make the atoms in your world the pivot point and dynamically allocate resource based on your exact needs. Then bill it out in the gigacorepackethours or some other unit of measurement that captures what fraction of the overall datacenter your application consumed and for how long.
Just don't bring back punched cards. They didn't really add value as far as I could tell.
I generally agree that there is nothing new under the sun. But I also think history doesn't repeat, it rhymes.
> When to mainframes kind of suck? Well when you don't know whether or not you want to use them. ... Ending up, nearly exactly back where they started except that now one company owns a mainframe and dozens of companies use it to run their bits of code.
I think this reveals that there is an economic distinction between mainframes and serverless, along two fronts: size of commitment and economies of scale. Mainframe time could be extremely precisely subdivided when billing users, but the capital cost of getting to the first kilosecond was extremely high. In serverless settings the capital cost for the first unit of consumption is experienced as zero by everyone except the vendor.
(Yes, IBM runs mainframes-as-a-service, but they don't do it on the cheap).
The second difference is that the hyperscalers enjoy economies of scale and scope that no other organizations can match, including mainframe operations. If IBM is choosing between two different parts for a mainframe, they can pick the more expensive part and foist most of the cost onto their buyers, for whom there is little alternative. Meanwhile, for the hyperscalers, customers are much more price sensitive. More expense falls to their bottom line. But because they operate at such high scale, even tiny improvements are rational to pursue.
Similarly, economies of scope come from the fact that they can pool much larger amounts of variance, which makes their workloads more predictable overall than any smaller organisation can achieve. This is just the central limit theorem mercilessly steamrolling noise into smooth, predictable shapes.
I really like this observation as well, especially the "cost to first kCs" which, for me, is the one differentiator.
One of the things that amazed people was the Blekko, the search engine company, operated its own clusters in a data center rather than using AWS. We got a lot of side-eye for that but the numbers were pretty stunning. For our app, the cost of running that data center was about $120K/month, on Amazon (or IBM's cloud) that cost was at least $1.2M/month so 10x higher. What was even more of a pain that getting all the machines in the same racks so that they didn't share "east west" bandwidth with other racks in the data center was a huge ask for these folks, the proposed solution of over provisioning, made it just that much more expensive.
From an outside view it seems like AWS has definitely arbitraged costs over time to collect a lot of money. Raw capacity that costs them $1 today costs them $0.80 next year, but they still sell it at the price they chose to cover the $1.
Having said that, there will be someone sprinting breathlessly into the thread to tell me that AWS has lowered prices on X a dozen times and on Y fourteen times and so on. But these decisions are made intermittently, by humans with bonus schemes, and AWS is harvesting the surplus in the meantime.
It feels from the outside as if retail Amazon pays serious, ongoing attention to selling at the minimum possible price, but AWS just doesn't feel the same pressure to do so.
What serverless does is to force the developer to have a sharp separation of code and persistence. For all cases. Even saving a small inter-request variable that lives for 5 seconds or less.
What serverless does is to force scalable architectures.
The price is slim to no control over build and execution environment as a consequence of not taking the underlying architecture into account. For many people this is not an issue. These people will be very content with serverless and should definitely use these types of solutions.
Personally I love serverless, but this article definitely mirrors my own experience.
In two jobs now I've been told to do things "serverless first" and both times I've ended up with an ECS solution. My reason always boils down to the other tooling, particularly in regards to HTTP endpoints. AWS API gateway is an absolutely infuriating product.
My current thoughts are if you're working in the AWS ecosystem, with AWS tools, Lambda works fantastically. Using Lambdas to set off alarms and run playbooks has never let me down, and I've done a lot of data processing hanging lambdas off streams and it just works. Really well in fact.
Where it falls over is interacting with the outside world in a sane fashion. HTTP endpoints are a nightmare to set up and deploy in a semi-automated fashion, I've had things that matched what was in the docs that just don't work because of some random gotcha, the undocumented silliness around a "HTTP API Gateway" vs a "REST API Gateway" is another mind bogglingly frustrating thing. Not to mention all the latency issues I've had, which are far too many to blame on a cold boot of the lambda.
Serverless has its niche and it works really well, and real cheaply to boot. But it's not a generalised tool like the loudest evangelists like to pretend it is.
Serverless has been a clear case of a solution looking for a problem, IMO. The vast majority of the software I’ve deployed in my career gets called a lot; deploying them onto a serverless platform would’ve increased our cost and maintenance burden significantly above simply dockerizing some web app.
There are certainly use cases for the occasional script run, but that feels like more of job for some cron-like software, and also feels like a far cry from what serverless promised.
I've lived through the genesis of the modern internet, the move from command-line to GUI, dot-matrix, Java's birth, smartphones, .Net's birth, AMPS and GSM, the iPod, NTSC, all the web tech from JavaScript, CSS, MongoDb, Bitcoin, JQuery, Node, to agile, modern JS libraries and of course serverless, and countless others.
The result of it all is cynicism. Serverless takes a lot of work to learn and get good at. The benefit is negligible - no more than deploying a "serverless" thing to an existing web server we use for cross-cuting concerns like exception handling, logging and communication. 10 seconds of automated configuration management, another 10 to deploy. The Change Approval Board meeting to rubber stamp the deployment takes longer.
The breathless hype machine has and always will be just that - hype. Someone's "new, improved, OMG I found this you must use it now" perspective is a pattern that's older than Gartner's hype cycle (which in itself has no basis in fact). It works well in the startup world, but has virtually no place in a business-critical environment, let alone healthcare, defence, telcos or similar.
Serverless conceptually was too tightly bound to the function as a service model. As we peel that back what we're finding is serverless really means "not my server". Someone else manages that piece of infrastructure, it can be anything, a VM, a database, etc. But in this case it feels like serverless is falling squarely into the PaaS category, building a better Heroku. Google Cloud Run is one prime example. DigitalOcean App Platform is another. I think this is likely the primary way software gets run in the cloud in the future BUT with one huge caveat, its not clear its going to be the way forward for large scale development itself.
Cloud lacks what mobile lacked before the iPhone, a development model. When we finally figure that out, it'll become obvious that table stakes here is an actual full end to end experience with one language and a set of primitives to help scale development for the cloud. That's not FaaS. Its something entirely new. The language will be popular, maybe Go, the focus will be on cloud APIs. Everything else will be become a client. The web era is over, thats not what the cloud is about. Cloud is all about APIs.
I'd agree with the part of maintaining your own server. like personal one. I promised myself like 5 years ago I'll never be my own (and family and friends..) sysadmin ever again. ... but serverless needless mean tiny programs in nodejs. serverless is basically a super category for what we used to call PaaS simply saying the granularity of what you deploy can be smaller. it doesn't mean it has to be smaller. as far as I am concerned it can also be way bigger than what heroku represents. This is what composability on infrastructure means. It means you have the same interface to deploy a single line of JS... and a service that is itself composed of a couple of monoliths each with its own data backend. The "form of the program" should be dominated by the problem it solves not its deployment target.
That is the thing we are trying to achieve at platform.sh where I run product. A uniform interface to a composable infrastructure. We are not quite yet there... this is not a trivial problem. And I don't think that "just ship your machine" Docker style solves this. To be really composable you need both finer grain and higher abstractions. What I mean is that at every level you compose you need to be able to define dependencies (my function requires this version of nodejs and this system lib) my already composed monolith that includes a dependency on MySql and Redis) is a dependency of this static site builder.. at each layer the semantics are very close. But not the same. Just shipping a container eludes these issues and basically reproduces the model of "good old servers" in a way this is contrary to what true serverless has to offer. Which is absolutely not caring at all about the runtime. And caring only about the design of your system such as that it espouses best the form of your problem.
There is a great misunderstanding as to what serverless actually is. So many people, including this article, equate serverless to cloud functions. There are plenty of services like Google Cloud Run or FaunaDB that are also serverless.
IMO what makes something serverless is that you don't need to provision or manage infrastructure, and also no need to worry about scaling as it will go up and down as needed.
> IMO what makes something serverless is that you don't need to provision or manage infrastructure, and also no need to worry about scaling as it will go up and down as needed.
This is why serverless is such a stupid term to begin with, IMO. It's not even an accurate descriptor (the stuff is running on a server _somewhere_), so it really could mean anything.
Something like "cloud functions" to describe Lambda and "managed Kubernetes" for Fargate. Of course, these terms are less sexy, so they'd never catch on.
With that definition, wouldn’t something like beanstalk qualify? I think that definition is overly broad. However, you’re right in that it does make me wonder where the definition should start or end.
You're responsible for managing the server in beanstalk, but it's not far off. I suppose it's just an earlier generation of the same idea. WHat I mean is, beanstalk runs on EC2 instances that you are responsible to make sure are patched, configured correctly and have everything you need for your application to run.
Fargate/Cloud Run are probably better serverless examples - you bring a container and config and the cloud provider will handle the rest.
It provisions EC2 for you, and scaled up or down. To really simply, it’s the basic equivalent to having an infrastructure admin turning EC2 instances on and off as load goes up or down.
I like the term "nano services" is it probably the wrong term.
But I like it.
We went monolith -> micro services -> micro services + nano services
Everything that adds complexity in a micro service system, adds far more complexity with nano services.
Debugging and tracing is a nightmare.
And yes, logging, a lot of logging and instrumentation helps.
But if you add too much of it then the whole point of the nano services is lost.
Versioning can get complex, depending on your deploy model.
Presumably if you push 100% of your nano services with every deploy you can get around it.
We have about 600 nanoservices running on AWS now
across 4 different systems.
My advice, that nobody would ever ask for, keep AWS Lambda for certain special tasks where they are a very good fit.
As always run the numbers, performance, cost to run, cost to maintain. Will the immense ability to scale really help your
use case?
In most of our cases, no, they will not.
For some companies it will be a great solution and solve
pain-points, save money
So many comments saying "Serverless just added complexity", then you read what they did and it's, "so I tried to move our entire infrastructure to Serverless and..." Umm, yeah?
And did we mention there's still a server?
Serverless is really not that complicated. But trying to plug a toaster into a super collider just might be.
Serverless has an edge usage case and its not in the middle of your web requests. Due to the load on AWS regional lambda compute pool, now your have a 500-2000ms cold start to your lambdas or you can't use it for tools and services with long-lived TCP connections (hi redis). Good luck. Not for me
I love how we're going back to PHP's original model (effectively stateless "functions") and giving it a hot new name. People have tried to convince me (unsuccessfully) that this is entirely different. Somehow.
If you really want serverless, install apache with mod_php and call it a day.
No, no you don't, but you do have to study and learn many competing standards and syntax. These are liable to
completely rewritten in 3 years from now, or else defunct and forgotten.
If you already have a server somewhere you could consider spinning up a container instance or kubernetes ( assuming you have at least linux 1 server with excess ram available )
If you deploy your service on kubernetes than you should be able to scale effectively.
Overall isn't serverless just a cost optimization for intermittent compute workloads? That's handy, sure, but I also figure that costly workloads like that are rare enough that it's obvious why the "revolution" would stop well before conquering the whole cloud market.
> I'm not against serverless solutions per se. I promise. It's just that developers should realize – especially if they are exploring serverless models for the first time – that this technology is not a straight replacement for servers.
Ha. So the name didn't give that away then?
In all seriousness though, I think another avenue for serverless to still explore seriously is declarative application definitions rather than just functions/runtime as a service. Hasura[0] are doing this with GraphQL but I think we'll soon seen (are already seeing?) a surge in these kinds of cloud abstractions that do much more for you than a simple on-demand runtime ever could.
I'm not sure about the hater-ade in this article (old, dead languages?). I've built infrastructure for multiple companies on top of AWS Lambda. Usually bog-standard REST endpoints in Node or Python. There are a few weird bits and a few "huhs" (DB connection timeouts after a Lambda wakes up from sleep) but that's true of most systems.
True, we were only doing a few req/s but Lambda was easy to setup/use and was mostly maintenance free. As is said so often on this site: it's a tool; use it for its intended purpose.
Our development environment was very reasonable: Express or Django locally; aws-lambda->Express/Django on the lambda.
Many apps don't need scaling. For B2B, getting first 100 customers is a big deed per say. But app owners buy the cool aid of cloud vendors and make things complicated. Don't trade simplicity for scaling unless you REALLY need it.
Because serverless is not serverless. Fundamentally this doesn't work because if you've gotta have a server in the end, why not either admin yourself or use one of the cloud vendor's other products for more control?
Serverless just means that your infrastructure is programmatically defined through configuration files rather than manually stiched together, so (in theory) the underlying silicon is abstracted away in a similar fashion to the x86 ASM that your code gets compiled to. In practice that isn't always the case (horror stories of rogue $xx,xxx/day lambda functions) but from my understanding that's the serverless end game.
Serverless is an economic description, not a technical one. A "serverless" service puts ownership and the immediate costs of ownership on the other side of an economic boundary.
It's a silly term, but successful. If anything, arguing against it made it more successful through mere repetition of the word.
Serverless is great or it sucks are extreme viewpoints. Reality is somewhere in the middle. For some specific use cases, (image manipulation, Alexa skills, etc.) serverless is fantastic. Dealing with persistent storage and DBs is challenging. But continues to evolve.
Managing server fleets and keeping them patched is no small feat. How much of the productivity can we gain if we didn't have to do all that with serverless? It's not possible for all use cases today. But can we move from say less than 5% of the use cases today to say, 40-50% of the use cases in another 5 years?
In my view, the serverless revolution is stalled because of lack of interoperability with other more complex application architectures. Serverless is IMO best suited as something available in-addition-to other more complex application architectures for a limited but growing set of use-cases. The current architectures I've seen make the "we <3 serverless" folks go all-in on it and then regret their decision and walk back to using services / docker / Kubernetes, or even a Heroku style PaaS for more power and flexibility.
One of the fundamental problem of serverless is intimately tied to its very selling point: that your code is run on interchangeable servers.
What that leads to is that everyone's code runs on any server. The requires strict inter-server security, authorization and roles.
On a normal server, you can run multiple processes (DB, front-end, back-end, etc) and rely on the fact they're running on the same server to not having to design roles and auth.
In serverless, you need to manage roles and auths for everything.
You see the same issues for data pipes, triggers and queues.
I'm working with a different type of serverless approach.
For a few hackathons now my team tries to push a client-side frontend + smart-contracts on a blockchain as a backend solution. This has a UX hit (some downsides of it could be mitigated though) but it made our business logic easier to implement. Depending on the project it has additional benefits to go with this stack. This cannot be applied to every project though and usually the UX hit or the lack of private transactions is enough to favor more traditional stacks.
I don't agree to the authors points. Serverless is another tool in your belt, do you still need to make decisions with tradeoffs? Yes. Do the big 3 support most languages you need? Yes. Is vendor lock in real? Maybe but how many companies actually move cloud providers often, or ever? Most companies I know are still trying to get into the cloud. You can't run full applications in serverless? That's not the use case, again with decisions you need to make.
While I believe that the serverless hype is mostly, well, hype, this article makes some odd points against it.
> One is to optimize your functions for whichever cloud-native language your serverless platform runs on, but this somewhat undermines the claim that these platforms are "agile."
Like this one, don't see how optimizing for a specific language makes something less agile. Then again, "agile" means so many things in the field that I might just be misinterpreting.
To implement serverless, you'll need to re-architecture your entire infra. For that itself, I don't think it's worth it; because serverless is pretty much all event driven (which many things might be, but might not be able to fit on serverless; even it does, it might just be easier to plop it on a simple instance).
I do see a big potential for small and simple projects though. No need to setup instances and worry about networking: just deploy it and go.
Serverless is not so shiny when compared to a cloud-managed Kubernetes cluster.
I get to deploy stuff "the regular way", that is with a constantly running application, I don't have to re-train my developers, I have extra power in the cluster should I need it, I don't depend on a cloud provider in particular, and my costs are bounded.
Cherry on top: I don't maintain the infrastructure either, there isn't even a SSH daemon running.
It feels like serverless done right. You just deploy your regular Laravel app and don't need any special tooling while developing. Has anyone tried it?
I haven't seen it noted in the discussion below but there's an earlier incarnation of the so-called serverless architecture. Early mainframes could be leased outright or you could buy time on them, in some cases on an ad-hoc basis for one-time batch workloads.
Can someone show me an application (other than a real time one) that can't be solved with simple javac my_script.java, tomcat, and a database of ones choosing?
Dare I say that it might even be possible to run twitter at scale with just a simple NoSQL database and some scripts?
"For those new to the term, a quick definition. Serverless computing refers to an architecture in which applications (or parts of applications) run on-demand within execution environments that are typically hosted remotely."
Serverless just moves complexity around and makes some things a bit more awkward, like debugging and logging. You have to be all in on the platform. I’ve just taken over a serverless app after the lead left and there are so many moving parts to make it all work.
Serverless is the new mainframe. In 10 years everyone will be complaining about their AWS being a mess or their lambdas costing too much. There will be consultants selling modernization to get onto whatever the newest thing is. What is old is new again.
For constantly running applications it can be cheaper just to run a server. This fact has been used to avoid migrating to serverless by a lot of orgs i’ve worked with, even when it doesn’t apply to their use case.
I went to a webdev convention, and it ended up being a serverless hype train. Industry experts with a financial incentive to promote serverless went on stage and told me they can't debug their code, or run it on their machine. They showed me comically large system diagrams for very simple use cases, then spent an hour explaining how to do not-quite-ACID transactions. Oh yeah and you can only use 3-5 languages, and each import statement has a dollar amount tied to it.
More importantly, all those skills you develop are tied to Amazon, or some other giant. All the code you write is at their mercy. Any problem you have depends on their support.
Am I supposed to bet hundreds or thousands of man hours on that?