PSA: porting an existing application one-to-one to serverless almost never goes as expected. Couple of points that stand out from the article:
1. Don’t use .NET, it has terrible startup time. Lambda is all about zero-cost horizontal scaling, but that doesn’t work if your runtime takes 100 ms+ to initialize. The only valid options for performance sensitive functions are JS, Python and Go.
2. Use managed services whenever possible. You should never handle a login event in Lambda, there is Cognito for that.
3. Think in events instead of REST actions. Think about which events have to hit your API, what can be directly processed by managed services or handled by you at the edge. Eg. never upload an image through a Lamdba function, instead upload it directly to S3 via a signed URL and then have S3 emit a change event to trigger downstream processing.
4. Use GraphQL to pool API requests from the front end.
5. Websockets are cheaper for high throughput APIs.
6. Make extensive use of caching. A request that can be served from cache should never hit Lambda.
7. Always factor in labor savings, especially devops.
The way read above comment is - If you can live with following limitations, then use lambda/serverless, works great. I have got to a point where for any internal systems used by internal users, lambda is my defacto standard. Very low cost of operation and speed to market. For anything that is external facing I prefer not to use lambda, specially if growth of usage is unpredictable.
You are not wrong. But it is all about saving money on labor. The rest are just the constraints of the system you use. (Aka requirements) its like complaining about the need to use posix for Linux.
20 years ago. We had enterprise java, it’s still “there”, but running spring is very different from what it used to be.
You’d simply upload an ear or ear and the server and deployed would handle configuration like db etc.
It worked perfectly (ear/war, the persistence framework was too verbose and high level imo, but that was replaced by hibernate/jpa). There was too much configuration in xml, but that could easily be replaced my convention, annotations and some config
Again.. we are running in circles, and this industry will never learn, because most “senior” people haven’t been around long enough.
> Again.. we are running in circles, and this industry will never learn, because most “senior” people haven’t been around long enough.
And that likely won't change in our lifetime, given the rate of growth in demand for software: we literally can't create senior engineers fast enough for there to be enough to go around.
As an aside, I have the privilege of working with a couple of senior folks right now in my current gig, and it's pretty fucking fantastic.
The percentage of seasoned engineers is so low that 'senior' as a title often seems to stretch to "whoever is most experienced around here". That's probably fine, since people understand that experience is not reducible to that title. But this does bring to mind a metric for finding "objectively" senior engineers:
What's the biggest idea you've seen abandoned and then reinvented with a new name?
I feel like we're just transferring the labour from ops to dev though. Where I work we still haven't got as good a development workflow with lambdas as we did with our monolith (Django).
Optimistically, it could represent a positive trade-off that replaces perpetual upkeep with upfront effort, and all-hours patching and on-call with 9-5 coding.
In practice, I think a lot of those fixed costs get paid too often to ever come out ahead, especially since ops effort is often per-server or per-cluster. The added dev effort is probably a fixed or scaling cost per feature, and if code changes fast enough then a slower development workflow is a far bigger cost than trickier upkeep.
Moving off-hours work into predictable, on-hours work is an improvement even at equal times, but I'm not sure how much it actually happens. Outages still happen, and I'm not sure serverless saves much less out-of-hours ops time compared to something like Kubernetes.
I see your point though POSIX imposes very few (if any) architecture decisions on application developers. The kind of design choices we’re talking about are very different from those of POSIX-like utilities so I’m not sure if that analogy is a good one.
There are some HUGE benefits to this type of architecture (services + lambda where required) for large corporations, the main one being an insane reduction in a bunch of worthless crap that you no longer have to do:
- OS version patching, patch windows & outages, change mgmt relating to patching and reporting relating to OS patching
- antivirus, the same patching management and reporting as above
- intrusion detection / protection, host based firewalls, the same patching and management as above
- Other agents (instance health monitoring, CMDB, ...)
- Putting all this junk on a clean OS image any time something changes, re-baking and regression testing everything
This all adds up, and can be a significant cost to an organisation - team(s), licenses, management, etc.
That's basically why we're doing it, and we're seeing some really good costing implications from implementing various things using Azure Functions. Not perfect, but an extremely capable system, and there are things some of our teams are doing with Durable Functions that I was completely stunned by when they explained how it all works. Microsoft have some very good technology there.
The only thing I'm sad about is that you can only host it on Azure. I'd love an open standard for function-like programming and hosting.
Of course, not everything is suitable for this model, and it certainly won't be the most cost-effective or performant way to do it for everything either.
I think the comment is exactly opposite of what you are suggesting.
The comment is saying that Lambda has limitations and works best when considering those limitations. If those limitations don't fit your use case, you shouldn't be using Lambdas - or, at least, don't expect it to be an optimal solution.
90% of Reddit videos are unwatchable for me : they start OK then the quality is downgraded to something unwatchable and there's nothing I can do about it.
I even tried to download them with youtube-dl but it doesn't work.
100% agreed. The initial buffering time on them is ridiculous. I've started uploading to streamable and just posting that link rather than upload video straight to reddit.
In 2010 I was 24 yr old and built a myspace clone SaaS with music and video hosting with everything it implies: large uploads, background jobs, compiling an nginx patch to support range requests, ajax to have videos and music playing while browser, with Django on a 20 bucks/month server.
If you're not convinced I don't know what will.
I think the point is, with your Django approach, you'll be stuck doing ops work 9-5 once you start getting customers, whereas with serverless you can spend the time to do more feature dev work.
Not quite sure about that, nowadays I require a couple of 20 bucks/month servers to have one CI/staging and one training/production deployments. I'm at the point where I practice CD as in "deploy each git-push to branchname.ci.example.com and run cypress on it" and still am able to deliver in half a day what the customer would expect to happen in two weeks.
And of course, baremetal provides a much better ROI than VMs/VPS/<insert glorified chroot/jail here>.
You seem to have gotten your deployments down and I really think that's good. In my own experience, though, managing your own infra always works well until it doesn't anymore. And when it stops working well, it crashes and burns and sucks up all the time. Going with managed services like serverless helps to get around that.
I just had no marketing, no partner for that, but I could rebuild that a similar time frame if only I had a plan to make profit out of it. Made it into a local newspapers but that's was the beginning and the end.
It's from the times where I was finding musicians in the streets to pay me 50 bucks per website and hosted them on grsecurity hardened gentoo ... which I don't practice at that cost anymore of course.
https://www.charentelibre.fr/2010/02/09/article-5-internet-a...
> I've hacked a few lambdas together but never dug deep
Then why comment? You clearly don't understand the use-case that AWS fits.
I've had jobs that took 18 hours to run on single machine finish in 12 minutes on Lambda. I could run that process 4 times a month and still stay within AWS's free tier limits.
For the right workloads it is 100% worth realigning your code to fit the stack.
>I've had jobs that took 18 hours to run on single machine finish in 12 minutes on Lambda. I could run that process 4 times a month and still stay within AWS's free tier limits.
Ok I'll bite. What takes 18 hours to run on a single machine but finishes in 12 minutes on Lambda.
I worked on a service a year ago that would stream a video from a source and upload it to a video hosting service. A few concurrent transfers would saturate the NIC. Putting each transfer job in a separate lambda allowed running any number of them in parallel, much faster than queuing up jobs on standalone instances
That’s true, the cost needs to be factored into the model. But the near infinite bandwidth scalability allows the service to exist to begin with. If every job saturates your up and down bandwidth and takes 10 minutes, and you have 100 coming in a minute, you would need to design a ridiculous architecture that could spin up and down instances and handle queuing on the scale of thousands based on demand. Or you can write a simple lambda function that can be triggered from the AWS sdk and let their infrastructure handle the headache. I’m sure a home grown solution will become more cost effective at a massive scale but lambda fits the bill for a small/medium project
Right, but without the lambda infrastructure it would be infeasible from infrastructure and cost perspective to spin up, let’s say 10,000 instances, complete a 10 minute job on each of them, and then turn them off to save money, on a regular basis
Isn't that also possible with EC2? Just set the startup script to something that installs your software (or build an AMI with it). Dump videos to be processed into SQS, have your software pull videos from that.
You'd need some logic to shut down the instances once it's done, but the simplest logic would be to have the software do a self-destruct on the EC2 VM if it's unable to pull a video to process for X time, where X is something sensible like 5 minutes.
We developed a web-based tool that described water quality based on your location. We generated screenshots of every outcome so the results could be shared to FB/titter. It was something on the order of 40k screenshots. Our process used headless chrome to generate a screenshot then it was uploaded to S3 for hosting.
Doing that in a series took forever. It took something like 14 hours to generate the screenshots, then 4 hours to upload them all. Spreading that load across lambda functions allowed us to basically run the job in parallel. Each individual lambda process took longer to generate a screenshot than on our initial desktop process, but the overall process was dramatically faster.
The parallelism argument doesn’t pass muster because you can do the same thing with a cluster of free tier t2.micro machines with any good orchestration platform, not just lambda.
This argument is basically: no counterpoint to the original post, but you can do things that are also easy on any other comparable platform.
This still doesn’t make sense. There are portable systems that do the same, and have fully managed options, such as kubernetes.
In my mind the thing that makes lambda “easier” is they make a bunch of decisions for you, for better or worse. For real applications probably for the worse. If you have the knowledge to make those decisions for yourself you’re probably better off doing that.
> This still doesn’t make sense. There are portable systems that do the same, and have fully managed options, such as kubernetes.
The whole value proposition behind AWS is that they can do it better than your business due (directly or indirectly) to economies of scale. I think Kubernetes is super cool, but rebuilding AWS on top of Kubernetes is not cost effective for most companies--they're better off using AWS-managed offerings. Of course, you can mix and match via EKS or similar, but there are lots of gotchas there as well (how do I integrate Kubernetes' permissions model with IAM? how do I get Kubernetes logs into CloudWatch? how do I use CloudWatch to monitor Kubernetes events? etc).
I can't support point 7. enough. People often forget about the cost of labor.
We migrated our company webapp to Heroku last year. We pay about 8 times what a dedicated server would cost, even though a dedicated server would do the job just fine. And often times, people tell me "Heroku is so expensive, why don't you do it yourself? Why pay twice the price of AWS for a VM?"
But the Heroku servers are auto-patched, I get HA without any extra work, the firewall is setup automatically, I can scale up or down as needed for load testing, I get some metrics out of the box, easy access to addons with a single bill, I can upgrade my app language version as needed, I can combine multiple buildpacks to take care of all the components in our stack, build artifacts are cached the right way, it integrates with our CI tool, etc, etc.
If I had to do all of this by hand, I would spend hours, which would cost my company way more. In fact, I'd probably need to setup a Kubernetes cluster if I wanted similar flexibility. By that point, I'd probably be working full-time on devops.
Once you factor in the learning time for AWS per a developer the cost is even higher.
At my previous company we had project with an AWS deploy process that only two developers could confidently use. Teaching a new developer & keeping them up to date was a big time sink.
For comparison we had a Rails app setup on heroku that on day one junior devs were happily deploying to (plus we had Review apps for each PR!)
This is a good point. Expecting developers to understand how to configure service to service IAM permissions and all the other nuances of AWS infrastructure is a fool's errand. Also one of the reasons we started Stackery.
I'm curious. Did you look into Googles AppEngine? It seems to have a lot of the benefits that Heroku offers, but is much cheaper.
Granted that it does impose some limitations, and therefore isn't right for all apps. But it does seem like it would work for a large percentage of web apps and REST api's.
The cost you're talking about is really hard to measure. Were they able to reduce the team sizes and get rid of positions after the change? Did the payroll reduce at all?
> PSA: porting an existing application one-to-one to serverless almost never goes as expected.
Came here to post this and agree 100%. Moving to Serverless requires evaluating the entire stack, including the server side language choice, and how the client handles API calls.
Often a move to serverless is better accomplished gradually in stages than the quick "lift and shift" that AWS like to talk about so much. Sometimes you can simply plop your existing app down in Lambdas and it runs just fine, but this is the exception not the rule.
> Don’t use .NET, it has terrible startup time. Lambda is all about zero-cost horizontal scaling, but that doesn’t work if your runtime takes 100 ms+ to initialize. The only valid options for performance sensitive functions are JS, Python and Go.
Shouldn't this not be a problem if you're doing 10 million requests a day? If you have enough requests, your lambdas should stay hot most if not all the time.
If the lambdas are always hot, what is the advantage over having a server? I thought the big selling point of serverless was not having to pay for long stretches of time where you don't have any requests.
If you have 10m requests uniformly distributed, then yes it’s less of a problem, but that’s unlikely. (Even then lambda containers will be recycled multiple times throughout the day, so there is still a small penalty.)
Oh interesting, maybe I should look into this more, I feel like it would be useful, but one day it just showed up in my projects and spammed out and drowned all my debugs in the log so now I rip it out as soon as I can because it drives me nuts and I couldn't find an easy way to turn down the messaging.
If you're still reading this, you can disable Application Insights during debugging (why would you need it anyway during debugging). To do this you make an application variable like 'EnableApplicationInsights' in your web.config so you can set per environment whether or not it should be on.
Then if this is false, in your Application_Start() you can set this: TelemetryConfiguration.Active.DisableTelemetry = true;
Having used serverless a bit, I’ve run into many of the same issues and generally agree with the advice but depending on your use case it may not be worth contorting your architecture to fit serverless. Instead, I’d look at it as a set of criteria for if serverless would fit your use case.
At this point in time the only places I’d use lambda are for low-volume services with few dependencies where I don’t particularly care about p99 latency, or DevOps-related internal triggered jobs.
I don’t think anybody advocates for rewriting all existing projects as serverless. But if you’re starting a startup, going all in on serverless will let you deliver better products faster. If Paul Graham’s Beating the Averages would be written today, the secret weapon would be serverless, not Lisp.
> going all in on serverless will let you deliver better products faster
Can you show some empirical evidence that supports this? In my experience this is another nebulous serverless hype claim that doesn't withstand scrutiny.
I don’t think it’s possible to produce empirical evidence to prove or disprove this claim, but it’s just common sense:
Using managed services leads to writing less code and minimizing devops work, both resulting in more time for feature development that takes less time overall and produces higher quality service (latency, availability, etc). Then there is the added benefit of clear financial insight into the inner workings of the application (one can trace capital flows through functions) which will result in better resource allocation decisions.
No. There's nothing common sense about it. It only seems plausible if you read the sales brochure from a cloud vendor and have no experience with all the weird and whacky failure modes of these systems and the fact that none of the major selling points of serverless actually work as advertised unless you dedicate significant engineering time to make them work - as the GP comment has demonstrated. The amount of engineering time required to make serverless work quickly catches up to or even exceeds just doing the damn thing the normal way.
And that engineering time is not transferable to any other cloud vendor, and neither is your solution now. So congratulations you just locked your business in.
Serverless only makes sense if you have a fairly trivial problem and operate on really narrow margins where you need your infra and associated costs to scale up/down infinitely.
> Serverless only makes sense if you have a fairly trivial problem
That’s exactly the point. The web application needs of most startups are fairly trivial and best supported by a serverless stack. Put it another way: If your best choice was Rails or Django 10 years ago, then it’s serverless today.
If your best choice was Rails or Django 10 years ago you probably don't have a viable startup today. Why? Because it's 10 years later. Technology moves on and market niches get filled. There are orders of magnitudes more people with the skill to setup a basic CRUD webapp, and about 15 years for the markets that these can serve to have been filled.
As a side note, I've learned that the existence of a large number of viable ways to accomplish a task is a pretty big anti-signal for the desirability of accomplishing that task in the first place. When I started my career in 2000, there was a huge debate over whether the "right" way to develop a desktop application was MFC or .NET or Java or Visual Basic or Qt or WxWindows. The real answer was "don't develop desktop apps, because the web's about to take over". When the big web 2.0 businesses were founded from 2005-2011, there were basically two viable options for building a webapp: Rails or Django. Now that everyone's arguing about microservices vs. Docker vs. Kubernetes vs. serverless vs. Beanstalk vs. Heroku, it's likely that the real answer is "develop a blockchain app instead".
> If your best choice was Rails or Django 10 years ago you probably don't have a viable startup today. Why? Because it's 10 years later. Technology moves on and market niches get filled. There are orders of magnitudes more people with the skill to setup a basic CRUD webapp, and about 15 years for the markets that these can serve to have been filled.
That's... not true. The choice of web stack – and, in fact, the whole software – is just a piece of what a startup may need.
Seriously, look at the list of YC startups on 2018 and tell me if most couldn't use either something like Rails, or a Single Page App In React With A Serverless Backend. And it wouldn't matter one bit.
> Using managed services leads to writing less code and minimizing devops work, both resulting in more time for feature development that takes less time overall and produces higher quality service (latency, availability, etc).
Well, not necessarily? This assumes that the implementation is sound but it is not at all uncommon for abstractions to leak which end up causing more pain than they solve
Is there really that much hype? I feel like I haven't heard that much. Serverless isn't even really much of a new thing, there have always been providers that hid the underlying aspects of running a web site on the internet. I think for most people they just don't want to have to worry about patching a machine, rolling logs, watching disk space, etc, if they don't need to.
I will argue the opposite. Startups take on enough risks as it is. Unless your startup requires or is about a novel architecture. Why add more risks with non battle hardened technology.
Software professionals often sees benefits without understanding the tradeoffs.
Large scale reliable systems are antinomic with “launching a startup”. You’re going to go through 2 or 3 pivots and as many refactors, large scale is the last thing you want to optimize for.
You forget C++. It’s a great choice for Lambda due to startup times. Python startup time is actually terrible and should be avoided if the call rate is really high.
Actually, Lambda instance is reusable and after spinning up it will be used to handle multiple requests (if ones are coming often enough).
I measured startup of the runtimes a long time ago, and back in the days of node.js 010.x at least, Python 2's startup time was twice as fast as Node.js's, and Java's wasn't much worse than Node.js. I don't know how .NET fares compared to Java, but I imagine it's about the same.
Furthermore, eople like to compare runtime startup times, but this tells a very small portion of the story. For most applications, the dominant startup cost isn't the startup of the runtime itself, but the cost of loading code the app code into the runtime. Your node.js runtime has to load, parse, compile, and execute every single line of code used in your app, for instance, including all third-party dependencies.
Compare, for instance, the startup cost of a "hello world" node.js function with one that includes the AWS SDK. At least, six years ago, the Node.js AWS SDK wasn't optimized at all for startup and it caused a huge (10x?) spike in startup time because it loaded the entire library.
I would argue that the only languages that are a really good fit for Lambda are ones that compile to native code, like GoLang, Rust, and C/C++. The cost to load code for these applications is a single mmap() call by the OS per binary and shared library, followed by the time to actually load the bytes from disk. It doesn't get much faster than that.
Once you've switched to native code, your next problem is that Lambda has to download your code zip file as part of startup. I don't know how good Lambda has gotten at speeding that part up.
On "7) Always factor in labor savings, especially devops":
DevOps is not a synonym for "ops" or "sysadmin". It's not a position. DevOps is like Agile or Lean: it's a general method with lots of different pieces you use to improve the process of developing and supporting products among the different members of multiple teams. DevOps helps you save money, not the reverse.
Yep! Thank you so much! I really need to update the documentation, but I think `cross` (https://github.com/rust-embedded/cross), a drop-in cargo replacement, is the probably the best available solution for building musl-based binaries.
I haven't thought about step 3 before, but makes sense. Maybe I should show this to the guy who used Google Cloud Functions to upload images in our previous project :)
I guess the reasoning would be that this way the actual time spent in serverless code is shorter and by proxy the service becomes cheaper?
Saves time and money by writing and executing less code + S3 is optimized for this task, so it will always perform better than an ad hoc serverless function.
Number 3 - thinking in events instead of REST actions, can't be stressed enough. Of course, some things must be actions (or another for that is commands), and in those situations, you need something that will turn a command into an event, this is one of the features of CloudState (https://cloudstate.io), which offers serverless event sourcing - you handle commands, and output events that can be further processed downstream by other functions.
As general rules these sound great at first sight, but don’t really address the main culprit from TFA - like for like API Gateway costs a lot more to process n number of requests.
Well, given the feature set of API Gateway compared to a Load Balancer I think it should be expected that it costs more. But that’s also beside the point which is to use managed services to do the heavy lifting. Eg. if you need a PubSub service for IoT, that shouldn’t go through API Gateway and Lambda, there is a specific AWS service for that.
If you use Cognito for identity management, then there isn’t even need for that. You can just assign users the appropriate IAM role and you can upload directly from the front end.
These are usually the read N items from a database type of queries that GraphQL makes trivial to batch together. Will barely increase response time, but will provide a better experience for users on bad connections.
> 1. Don’t use .NET, it has terrible startup time. Lambda is all about zero-cost horizontal scaling, but that doesn’t work if your runtime takes 100 ms+ to initialize. The only valid options for performance sensitive functions are JS, Python and Go.
I always sorta assumed that Amazon pre-initialized runtimes and processes and started the lambdas from essentially a core dump. Is there some reason they don't do this, aside from laziness and a desire to bill you for the time spent starting a JVM or CLR? Does anyone else do this?
I did the same experiment as OP and ran into the same issues, but eventually realized that I was "doing serverless" wrong.
"Serverless" is not a replacement for cloud VMs/containers. Migrating your Rails/Express/Flask/.Net/whatever stack over to Lambda/API Gateway is not going to improve performance or costs.
You really have to architect your app from the ground-up for serverless by designing single-responsibility microservices that run in separate lambdas, building a heavy javascript front-end in your favorite framework (React/Ember/Amber/etc), and taking advantage of every service you can (Cognito, AppSync, S3, Cloudfront, API Gateway, etc) to eliminate the need for a web framework.
I have been experimenting with this approach lately and have been having some success with it, deploying relatively complex, reliable, scalable web services that I can support as a one-man show.
> You really have to architect your app from the ground-up for serverless by designing single-responsibility microservices that run in separate lambdas, building a heavy javascript front-end in your favorite framework (React/Ember/Amber/etc), and taking advantage of every service you can (Cognito, AppSync, S3, Cloudfront, API Gateway, etc) to eliminate the need for a web framework.
At least I don't have to learn that complex "systems admin" stuff.
Exactly, it's like saying to someone running a restaurant that buying their bottled water from a convenient store is more expensive than buying it in bulk from Costco.
It's entirely missing the point. At the end of the day, you have to look at your specific usage pattern and pick the best option for you. Obviously, as with any other technology, anyone who forces a specific option in every possible situation is most likely wrong.
To eliminate the need of a web framework? I don’t understand the rationale, if I can get all what’s mentioned done with a good web framework, I will be more than happy to do that.
With your own server and Web framework, you do all the work in provisioning the machine, configuring services, installing dependencies, building deployment and integration pipelines and, worst of all, maintaining all that when updates are released / something breaks. It is also harder to scale.
A serverless solution that eliminates the Web framework (and thus the stack in which is being run) does most of that for you, at the expense of extra cost or infrastructure deployment complexity, but once it is done, scaling and maintenance are easier.
Firebase now makes most of these painless. They've done a really good job. If your starting from the grounds up and can stomach using a google product Firebase is the easiest to work with by far.
First and foremost it is client side SDK's (web, mobile) for their database products, their newest being Firestore that provides better query capabilities compared to their original Firebase Realtime database (while still offering real-time capabilities).
Along with that is Firebase Authentication, which manages user accounts and authentication.
The real magic comes in with Cloud Functions (their version of Lambda) which allows for hooks in to all sorts of events occurring from usage of these database and authentication products (and other cloud services).
Hook into database writes, updates, deletes, user creation, Google Cloud's pub-sub events and many more. They also offer static website hosting as well as hooking website serving into cloud functions (for server side code execution).
In the context of a website, all of these work together to allow for usage of the JAMstack[0] architecture which decreases your infrastructure resources you need to manage and cost.
Firebase does a lot of stuff. Originally it was a small company that focused on providing a real-time JSON-like backend store for web apps. But then they got bought by Google and seem to have evolved into Google's answer to a lot of AWS services, ie hosting, real-time DB, serverless, and probably more I'm not aware of.
I've often looked at and played with Firebase since it does so much of what I need to back a React Native-based app for simple mobile games and utilities. I always end up talking myself out of it due to Google's history of pulling the plug on (what seem to an outsider to be) perfectly good, stable products that wouldn't do any harm to keep around indefinitely.
As a hobbyist, I wouldn't have the time or motivation to completely rewrite a project if that happened, which would be necessary since a Firebase app (like a heavily AWS-integrated serverless app) is not just technologically but architecturally tied to that environment.
It seems like a pretty reasonable concern to me. We can presume the odds are pretty low, yeah. But the consequences are very high - you would need to redesign and possibly rewrite in another language your entire application. People complained up a storm about Reader dying, but it was a 5min process to export your subscriptions and import into another web reader that had basically the same feature set. Conventional Linux hosting, or even Docker, could be pretty easily re-hosted as-is on any of hundreds of other places.
I usually go with a Django or RoR monolith, something that I know I can run on a cheap DO droplet or similar, and while there's a fixed monthly cost it's reasonable and I can scale up with ease (albeit manually). If I were explicitly needing the realtime DB aspect I'd probably look to Phoenix with the same hardware approach.
It would be nice if there was a dead-simple Firebase-like tool you could self-host. ie just a single instance that you could point all your toy apps at to give them a little real-time persistence.
This is how a conversation with a colleague who were enthusiastic about Serverless, and who's company was mostly on Java/JVM stack went:
Colleague: Lambda is awesome, we can scale down to zero and lower costs! We love it! We use cool tech!!
Me: What did you do about JVM warm up?
Colleague: We solved it by having a keepalive daemon which pings the service to keep it always warmed up.
... Me thinking: Uhh, but what about scale down to zero?
Me: What do you do about pricing when your service grows?
Colleague: We use it only for small services.
... Me thinking: Uhh, start small and STAY SMALL?
Me: How was performance?
Colleague: It was on average 100ms slower than a regular service, but it was OK since it was a small service anyway.
... Me thinking: Uhh, but what about services who _depend_ on this small service, who now have additional 100ms times to comprehend with?
Overall, I think his answers were self explanatory. Lambda seems to be a fast prototyping tool. When your service grows, it's time to think how to get out.
My thoughts EXACTLY. The great power in "serverless" architecture (i.e. AWS Lambda + AWS RDS + AWS Gateway) is how it empowers prototyping a new product.
Counterintuitively, it's future-proofing. You should know in advance that it's too slow & expensive. But you get to spin up a prototype backend very rapidly, pay only for what you're using while prototyping, and Lambda's inherent limitations force devs to build modularly, start simple, & stay focused on the product's main goals.
When the time comes to need scale, either at launch or even later when user count goes up, your "serverless" backend can be relatively easily replaced with servers. Then, just like that, in response to scale need your product's costs and response time go down instead of up.
It's a nice way to build a software product: rapid prototyping plus easy future cost decreases built-in.
Can't you just do something on your local machine?
There's stuff like dotnet new for .NET where I can just run that and have a skeleton project for a backend and I can start writing code immediately. I assume there's template creators for other languages as well.
My use case was a prototype for an iOS app I had in beta testing. It had a tiny but globally distributed user base, and serverless was a fun thing to learn on top of being relatively quick to set up. I'm sure if I had wanted to, some dynamic DNS and a small machine in my house would've sufficed. But hey--that's future decreases in cost. :)
If you're doing a prototype, wouldn't firebase or AWS AppSync be better options? You're going to lose a lot of time dealing with devops tasks (setting up IAM accounts, configuring storage services, etc.)
For me, the absolute best use cases for serverless is for really infrequent, small tasks.
For example, I have a few data scrapers written in JavaScript but my regular stack is lamp.
So I don't have any need to run a node server 24x7 just for those once a day tasks.
But I have even found myself not needing serverless for that because everything is running in a kubernetes cluster. So I can just setup a cron to run them which launches the needed node containers.
So I guess in effect, I am just using a sort of self-managed "serverless".
It's the same argument for Python over C development. Prototype in python and migrate portions to C as performance is needed. You'll often find that large portions of your codebase will never need to migrate out of the "prototype" stage.
> ... Me thinking: Uhh, start small and STAY SMALL?
This does happen. We have a serverless API forwarding service on Azure that was designed to simply format and forward calls from a vendor. We know the volume, there will not be any surprises, and it is immensely profitable over the old solution to the tune of thousands of dollars per day. Our use case is probably pretty uncommon, however.
It's a good sign that people who only talk about FaaS when they say "serverless" didn't understand serverless at all. And I see this as a failure on the serverless proponents side.
The serverless proponents are selling their paradigm as simple solution, which leads many people to believe simple means FaaS.
Throwing Lambda on all backend problems is a setup for failure. Often transfer and simple transform of data can be done serverless without a Lambda, which cuts costs AND leads to better performance.
Keep-Alive daemon doesn’t work during scale up. If you go from 1 simultaneous request to 3, it will have to slowly spin up those 2 lambda’s in response to a user a request.
It’s useful for services which fit its design: using an extremely heavy environment like Java will rarely be a good fit but for even Python/Node it works much better, without even considering things like Go/Rust.
Your “thoughts” are applying his solution to the wrong problem. “Start small and stay small” I’m not sure what that even means. Are you saying every service has to grow to some size or required amount of compute? LOL
The 100ms extra time is nothing. I mean - are you trying to solve at Google or Amazon scale?
I run simple Lambdas that read from some SNS topics, apply some transforms and add metadata to the message, and route it somewhere else. I get bursts of traffic at specific peak times. That’s the use case and it works well. The annoying part is Cloud Formation templates but that’s another topic.
You’re making some bizarre assumptions. Not everything is front end user facing.
Let’s say I’m processing messages off a queue. P90 @ 50ms vs p90 at 100ms doesn’t necessarily make a difference. What are my downstream dependencies? What difference does it make to them?
At the end of the day, value is what you care about - not necessarily chasing a metric because lower is absolutely better. What’s the cost of another x milliseconds of latency considering other trade offs (on going operational burden, extensibility, simplicity, scalability, ease of support etc etc).
If 50 ms latency means I can have a solution that can auto scale to massive spikes in traffic due to seasonality or time of day vs a reduction of that latency but I have to spend time capacity planning hardware and potentially holding extra capacity “just in case”, then again, optimizing for a single metric is pointless.
Something about the Lambda/FaaS/serverless hype reminds me of early 2000s enterprise Java, when everyone was trying to replace code with XML configuration.
It's obviously at a different point in the stack, but the promise is similar: "Just say what you want and it magically happens" — and indeed that's the case for FaaS when your problem is small enough. But XML-configured frameworks also offered that convenience for their "happy problem space". You could get things done quickly as long as you were within the guard rails, but as soon as you stepped outside, the difficulty exploded.
I'm not convinced AWS Lambda is all that different from a web framework, it's just on a higher level. Instead of threads responding to requests, you have these opaque execution instances that hopefully will be spun up in time. Instead of XML files, you have a dense forest of AWS-specific APIs that hold your configuration. That's how it looks from the outside anyway.
The promise of serverless is pretty simple, and pretty useful for the right use case - be it unpredictable load, or just very low load, or very frequent deployments, or pricing segmentation, or you don't have anyone as DevOps, and so on and so forth.
I don't recall anyone saying there's any magic involved. The premise is exactly same as cloud compute - you (possibly, depends on ABC) don't need to provision and babysit a server to perform some action in response to http request (or in case of aws lambda, other triggers as well).
Disclaimer: I work for Salesforce, Heroku’s parent organisation.
I have had so many conversations with devops managers and developers who are individual contributors and the Lambda hype reached frothing levels at one point.
Contradictory requirements of scale down to zero, scale up infinitely with no cold starts, be cheap and no vendor lock in seemed to all be solved at the same time by Lambda.
Testability? Framework adoption? Stability? Industry Skills? Proven Architectures...? Are some of the other question marks I never heard a good answer for.
You’re always locked into your infrastructure. People don’t willy nilly change their infrastructure once they reach a certain size any more than companies get rid of their six figure Oracle infrastructure just because a bushy tailed developer used the “repository pattern” and avoided using Oracle specific syntax.
And the “lock-in” in lambda is over exaggerated. If you’re using lambda to respond to AWS events, you’re already locked in. If you are using it for APIs, just use one of the officially supported packages that let you add a few lines of code and deploy your standard C#/Web API, Javascript/Node Express, Python/Flask/Django... app as a lambda.
Testability? Framework adoption? Stability? Industry Skills? Proven Architectures...? Are some of the other question marks I never heard a good answer for.
If you haven’t heard the “right answers” for those questions you haven’t been listening to the right people.
Lambdas are just as easy to test as your standard Controller action in your framework of choice.
Do you have any resources on testing a Lambda? When I was fooling around with it, the only thing I ran into was that AWS Sam-client or whatever. Thing looked like an absolute nightmare to get up and running.
You’re doing it wrong (tm). You setup and test lambda just like you test your controller action in an API.
Your handler should just accept the JSON value and the lambda context and then convert the JSON to whatever plain old object your code needs to do to process it and call your domain logic.
AWS has samples of what the JSON looks like for the different types of events. You can see the samples by just creating a new lambda in the console, click on test and then see the different types of events.
You can also log the JSON you receive and use that to setup your test harness. I don’t mean an official AAA type of unit test, it can be as simple as having a console app that calls your lambda function and passes in the JSON.
For instance in Python, you can wrapped your test harness in an
if __name__ == "__main__":
block in the same file as your lambda.
This is the same method that a lot of people use to test API controllers without using something like Postman.
> Thing looked like an absolute nightmare to get up and running.
So you tried it? I don't remember it being hard to set up, at least compared to a DB. Or, you can use the underlying docker images (open source, https://github.com/lambci/lambci) to run your Lambdas in. SAM provides some nice abstractions, e.g. API gateway "emulation", posting JSON to the function(s), or providing an AWS SDK-compatible interface for invoking the functions via e.g. boto3. This way you can run the same integration tests you would run in prod against a local testing version.
This! To me the only upside to the whole architecture (from a devs perspective) is that you can deploy these things to production independently of other parts of the system. If you can't do that with confidence because you're attempting to test it's functionality in some larger deployed context you've turned your monolith into a distributed monolith and now you have the worst of both worlds.
The good news; you should be able to accomplish most testing locally, in memory. The bad news; your test code is probably going to be much larger and your going to have to be very aware of the data model representing the interface to the lambda and you're going to have to test the different possible states of that data.
I've found unit testing to work fine for Lambdas. The biggest difference between running as a Lambda and running locally is the entry point. With a Lambda you have an event payload that (usually) needs to be parsed and evaluated.
I'll typically write all the core functionality first, test it, then write the lambda_handler function.
You call it just like you call any other function. Your handler takes in either a JSON payload and a context object or a deserialized object depending on the language. You call it from a separate console app, your test harness etc.
We deploy to a test stack and run a full integration test of the code running in AWS. I believe it's also possible to self-host locally, but we never really looked into it.
Why do HN folks find this so difficult? It's like putting your shoes on the wrong way around. After you've done it, it's clearly uncomfortable. So don't do it.
Serverless is specifically a stateless paradigm, making testing easier than persistent paradigms.
> Framework adoption?
Generally we use our own frameworks - I do wish people knew there was more than serverless.com. AWS throw up https://arc.codes at re:Invent, which is what I'm using and I generally like it.
> Stability? Industry Skills? Proven Architectures...?
These are all excellent questions. GAE, the original serverless platform, was around 2010 (edit: looks like 2008 https://en.wikipedia.org/wiki/Serverless_computing). Serverless isn't much younger than say, node.js and Rust are. There are patterns (like sharding longer jobs, backgrounding and assuming async operations, keeping lambdas warm without being charged etc) that need more attention. Come ask me to speak at your conference!
> Serverless is specifically a stateless paradigm, making testing easier than persistent paradigms.
No because Lambdas are proprietary which means you can't run it in a CI or locally.
Also, it becomes stateful if it pulls data from a database, S3 or anywhere else on AWS which it almost always does.
> Serverless isn't much younger than say, node.js and Rust are.
AWS Lambdas which I consider to be the first widely used Lambda service was was released in April 2015 which is 6 years after the release of Node.js.
Also, Node.js is way more popular and mature than Lambda solutions.
Overall Lambdas are only useful for small, infrequent tasks like calling a remote procedure every day.
Otherwise, things like scheduling, logging, resource usage, volume and cost make Lambdas a bad choice compared to traditional VPSs / EC2.
> No because Lambdas are proprietary which means you can't run it in a CI or locally. Also, it becomes stateful if it pulls data from a database, S3 or anywhere else on AWS which it almost always does.
Lambda is a function call. So it makes no difference if it’s proprietary or not.
Are you saying that it’s difficult to test passing an object to a function and asserting that it’s functioning as intended?
Lambas are not simple functions because your environment is different in local compared to production.
If I run a Node.js function in AWS Lambda, my Node.js version might be different, my dependencies might be different, the OS is different, the filesystem is different, so I or one of my node_modules might be able to write to /tmp but not elsewhere, etc.
It's the reason people started using Docker really.
If you don't have the same environment, you can't call it reproducible or testable for that matter.
Nothing you mentioned has anything to do with the ability to test a Lambda. You’re trying to use limitations and restrictions as friction to backup your inability to test.
There’s a lot of annoying things about lambda. And a lot of stuff I wish was easier to find in documentation. But that doesn’t change the fact that Lambda is more or less passing an event object to your function and executing it.
Writing a function in node 12 and then running it on node 4 and throwing your hands in the air cos it didn’t work isn’t the fault of Lambda.
It's great to see that factual evidence is answered with ad-hominem by the Lambda hype crowd.
In any case, if you have a Node.js module or code with a native C/C++ build, that runs shell commands, that writes to disk (not allowed besides /tmp in Lambda) or makes assumptions about the OS, your "simple" function will absolutely return different results.
e.g: My lambda is called when somebody uploads an image and returns a resized and compressed version of it.
This is done using Node.js and the mozjpeg module which is dependent on cjpeg which is built natively on install.
If I test my function on my machine and in Lambda it's very possible that I get different results.
Also, certain OSs like Alpine which are heavily used for Docker don't event use glibc as compiler, so again, another difference.
"In any case, if you have a Node.js module or code with a native C/C++ build, that runs shell commands, that writes to disk (not allowed besides /tmp in Lambda) or makes assumptions about the OS, your "simple" function will absolutely return different results."
This is true, but it's not Lambda qua Lambda. That's just normal production vs. testing environment issues, with the same basic solutions.
Lambda may offer some minor additional hindrances vs. something like Docker, but I wouldn't consider that catastrophic.
You are absolutely right that you could recreate a similar environment to Lambda in Docker. But you would first need to reverse engineer Lambda's environment to discover how it is actually configured and the limits that are set.
Even if you did find a way, you would still need to keep it up to date in case AWS decides to update that environment.
Logged in to say that this has actually been done (not by me) and my team has been finding it very helpful for local “serverless” SDLC: https://github.com/lambci/docker-lambda . It‘s billed as “ A sandboxed local environment that replicates the live AWS Lambda environment almost identically – including installed software and libraries, file structure and permissions, environment variables, context objects and behaviors – even the user and running process are the same.” We test our functions mocked and against local deployments of that lambci container . There also lambda “layers” (container images for building custom runtimes for AWS Lambda) but we have not used that feature at this point. Interesting space with lots of room for improvement in this tool chain though for sure
I’m not 100% sure as I didn’t create the image (though I’m evangelizing as someone who has found it truly helpful for daily dev.) . I believe the creators tarball’d the entire distro/execution environment from a running lambda so the file system layout and libs likely match Amazon Linux if that’s the default lambda execution distro image. If not I assume it matches the default
At least the Docker image used by AWS SAM CLI is created by AWS.
Also, you compile before packaging, so you dev/CI system already has to be able to compile for Lambda, independenly from testing/debugging with Docker.
> > Writing a function in node 12 and then running it on node 4 and throwing your hands in the air cos it didn’t work isn’t the fault of Lambda.
> It's great to see that factual evidence is answered with ad-hominem by the Lambda hype crowd.
I don't think that was a personal attack.
We've answered technical questions with technical answers.
- You have a definition of stateless which includes having no persistence layer, which is at best at odds with the industry.
- You think serverless was created with AWS Lambda which we've been kind about, but most people would say you're simply wrong.
- You're advocating for containers, which are well known for having their own hype as people write their own cloud providers on top of the cloud provider their employer pays for with dubious benefit.
Saying that local dev and Lambda are different is a strawman. How is that harder than developing on a Mac or Windows (or even Linux) and then testing on a different OS and config via CI/CD?
You shouldn't be testing "on your machine" - that's the oldest excuse in the book!
You should build your function in a container based on AWS Linux, just the same as you should for a Lambda deploy. That guarantees you the same versions of software, packages, libraries, etc. It makes it possible for me to develop Lambda functions on a Mac and test binary-for-binary to the deployed version.
"Nothing you mentioned has anything to do with the ability to test a Lambda" is not ad-hominem, it's a statement of fact.
Why not then have lambda run the same container you can run and test locally?
I don't use lambda but we have our jenkins spin up the same ec2 to run tests that we would spin up to do development so that we never run into this problem.
I'm not sure I understood your question correctly.
If you mean running a Docker container in Lambda, that is to may knowledge to possible.
You could schedule Docker tasks in AWS ECS (their managed container service) but it's not meant for anything realtime and more for cron job type tasks.
If you mean emulating the Lambda environment in Docker, then I wrote an answer with the difficulties of doing that below to another user.
No, your lockfile doesn't care about build steps so any post-install script might run differently for the many other reasons listed.
> Agree, but how much does one Linux+systemd different from other Linux+systemd? How much does the FS?
Plenty. For example filesystem change events are known to have filesystem and OS dependent behaviours and quirks / bugs.
When a Node module runs a shell command, it's possible that you have a BSD vs a GNU flavour of a tool, or maybe a different version altogether.
The Linux user with which you are running the function might also have different rights which could become an issue when accessing the filesystem in any way.
> VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless.
Maybe, but serverless doesn't answer those questions at all. It just hand waves testing and vendor independent infrastructure.
Then you're not talking about dependency versioning are you? you're talking about install order. In practice it hasn't been an issue, I should find out how deterministic install order is but I'd only be doing this to win a silly argument rather than anything that has come up in nearly a decade of making serverless apps.
> For example filesystem change events are known to have filesystem and OS dependent behaviours
> When a Node module runs a shell command, it's possible that you have a BSD vs a GNU flavour of a tool
Are you generally proposing it would be common to use an entirely different OS? Or a non-boring extX filesystem?
All your issues seem to come from edge cases. Like if you decide to run FreeB or ReiserFS locally and run a sandbox it it, fine, but know that's going to differ from a Linux / systemd / GNU / extX environment.
> > VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless.
> Maybe, but serverless doesn't answer those questions at all.
Serverless exists precisely to answer the question. I can throw all my MicroVMs in the ocean with no knowledge of dockerfiles, no VM snapshots, no knowledge of cloudinit, no environment knowledge other than 'node 10 on Linux' and get my entire environment back immediately.
> Then you're not talking about versioning are you? you're talking about install order.
I didn't mean build order but install scripts and native module builds.
The first type can create issues when external resources are downloaded (Puppeteer, Ngrok, etc.), which themselves have different versions or that fail to download and where the Node.js module falls back to another solution that behaves slightly differently.
The second type can occur when you have say Alpine Linux that uses MuslC and Amazon Linux uses GCC or when the native module tries to link with a shared library that is supposed to exists but doesn't.
> Are you generally proposing it would be common to use an entirely different OS? Or a non-boring extX filesystem?
I haven't checked but Amazon Linux by default uses XFS on EBS disks so I wouldn't be surprised if Lambda's used the same. So not a boring extX filesystem. ZFS is also relatively common.
> Serverless exists precisely to answer the question.
No, it clearly doesn't because your function will fail in local and succeed in Lambda, or the reverse, exactly due to the issues I mentioned in my various comments here and you will be left debugging.
Debugging which starts by finding exactly the differences between the two environments which would have been solved by a VM or Docker.
> I didn't mean build order but install scripts and native module builds.
OK. Then you're still not talking about your dependencies being different. The dependencies are the same, they're just particular modules with very specific behaviour...
> external resources are downloaded (Puppeteer, Ngrok, etc.), which themselves have different versions or that fail to download
That's more a 'heads up when using puppeteer' than a indictment of serverless and a call to add an environment management layer like we did in 2005-2015.
> Linux by default uses XFS on EBS disks so I wouldn't be surprised if Lambda's used the same.
That's worth checking out.
> Debugging which starts by finding exactly the differences between the two environments which would have been solved by a VM or Docker.
I see what you're saying, but planning your whole env around something like a given puppeteer module dynamically downloading Chrome (which is very uncommon behaviour) isn't worth the added complexity.
> No, your lockfile doesn't care about build steps so any post-install script might run differently for the many other reasons listed.
You shouldn’t be uploading node_modules folder to your deployed lambda so this is an issue of your development environment, not lambda.
> Maybe, but serverless doesn't answer those questions at all.
“Serverless” or lambda/Azure functions, etc, are not a silver bullet that solve every single scenario. Just like docker doesn’t solve every single scenario, not does cloud servers or bare metal. It’s just another tool for us to do our job.
I have the latest Windows insiders build on a spare machine. I've found all the aws tooling including SAM works pretty well under WSL 2 now that WSL uses a true Linux kernel.
Docker largely works fine on macOS? At least for testing, haven't run into any issues other than bad filesystem performance if you're writing a lot of data to a mounted FS. Our team uses it daily - a far cry from "unusable".
> No because Lambdas are proprietary which means you can't run it in a CI or locally.
Arc has a full sandbox, so yes you can run a lambda in a CI or locally. You should know this - if you're developing your apps by deploying every lambda as you edit it you're doing it wrong. Most lambdas don't really need you to simulate AWS.
> Also, it becomes stateful if it pulls data from a database, S3 or anywhere else on AWS which it almost always does.
Sure, persistence exists, but when people say 'stateless' they mean 'has no transient state'. They don't mean 'has no data'.
> AWS Lambdas which I consider to be the first widely used Lambda service
OK. You don't consider GAE (2008) widely used. I disagree - mainly because I was using GAE back then and thinking about this new world of no-memory, timeouts, etc - but lambda is definitely more popular.
That's not what I've understood "stateless" to mean. Sure, anything more complicated than a calculator app is going to rely on data stored elsewhere, and in Lambda world that means you're reading from DynamoDB, S3, RDS or whatever. Those are definitely dependencies and that's where the state lies. But the pattern encouraged by Lambda is that your instance contains no state of its own. The disk is used as scratch for each invocation, objects are created for each request and not shared between them, etc. That's what people mean by stateless.
I see your point, but this definition is so lax that it applies almost perfectly to any application I've ever deployed to the cloud. Using external services to save state isn't unique to lambda, any app hosted on a normal ec2 instance needs to follow the same pattern because the instance and its filesystem can go away at any time (unless you attach a volume, but I've always considered that to be bad practice).
Sure, it's a common pattern and the right one in many scenarios. But especially for non-cloud deployments it's very common to store session data in process memory or temp data to local disk, etc.
On-disk state isn't what people mean when they say stateless. * They mean there is no memory state, which is true and which absolutely means serverless functions are easy to test.
* you could argue that 'stateless' should include no long term persistence, but you'd be fighting an uphill battle. Like saying 'serverless' isn't correct because there are servers.
State is saved somewhere, sure. How many useful applications are truly stateless when considered in their entirety?
This isn't the sense in which servers are usually said to be stateless, however. A Lambda that reads and writes state from a database when invoked is not maintaining state outside of the request-response cycle itself, so it can be said to be stateless.
It's funny because the only use case I was considering lambda for was a pure / stateless function. As soon as you add state / side-effects I assume you've greatly increased your complexity and testing (you'd want to have separate tests for all of the inputs / outputs that resemble production... which is when things can get complicated).
I'm probably looking at it wrong I guess. I considered using lambda to unload some CPU intensive calculations / logic and that's it. I figured it would be good for that as long as the latency didn't outweigh any benefits.
Yes, but the parent said "statelessness that makes it easier to TEST". It is not stateless in that sense: purity makes it easier to test. Here you need to mock all interactions with external services, just like you'd do with non-serverless applications.
It is stateless in that sense. To run your lambda, there needs to be zero things in memory, unlike a traditional express/django/rails or other non-serverless apps.
If your lambda involves persistent storage, your testing might want to wipe or prepopulate the DB with some objects first, but that's not hard and doesn't add complexity, and as mentioned you don't need anything in memory.
It's concerning how typical the hype machine is in IT. I believe Serverless has its place and value. So does Kubernetes or many other products that are often discussed on HN.
But let's be clear, we are talking about commercial products and there is a capital interest in selling these services to all of us devs and engineers.
So while use cases exists and benefits wait to be reaped, as a consultant I strongly feel that we should be MUCH more insistent in pointing out when a product does not make sense instead of jumping onto the hype train.
I am basically surrounded by "LETS TRANSFORM EVERYTHING TO KUBERNETES THIS WEEK!" exclamations, conferences are basically "DID YOU ALREADY HEAR ABOUT KUBERNETES?" and so on ...
It reminds me of Ruby on Rails, a mature and well-developed framework used by global tech firms (correct me if wrong: Airbnb, ~Stack Overflow~, Github) to handle parts of their backend in 2019. But for half a decade now even tiny companies have been screaming around about FancyHTTPTechThisYear (tm) because scale while reporting 1/500th of the traffic of some famous Rails users.
This is not engineering with objectives in mind, it's more akin to the gaming community yelling for a new console.
> It's concerning how typical the hype machine is in IT.
Software engineering is still a young discipline. Probably half of the people have less than 4 years of experience. They learned from other employees that also had less than 4 years of experience. And that can be repeated for 10 generations of developers.
We are learning, and re-learning the same lessons again and again. Everything seems new and better and then we are surprised when it's not a silver bullet.
Software and cloud vendors do not help. They are the first ones to hype their new language, framework or platform. Technology lock-in is a goal for tech companies offering hardware or services.
> This is not engineering with objectives in mind
Curriculum driven development is a thing. And I cannot blame developers for it when the industry hires you and sets your salary based on it.
We need to be more mature and, as you suggest, think about our technical and business goals when choosing a technology instead of letting providers lock us in their latest tech.
> Everything seems new and better and then we are surprised when it's not a silver bullet.
We need to live and breathe a culture that makes even young developers aware that this mistake has been done over and over until a growing collective of developers has recognized the pattern behind this.
After all, in the analogue trades even apprentices are taught many "don'ts" right from day one. Software engineering should not be any different.
It's a mindless horde, an entire industry of half experienced idiots and half burned out savant engineers conning and rescuing one another in a cluster fuck of confusion. For the old school greys, the cloud itself is a con, as running a server oneself is actually quite easy, and exponentially better in every respect than slices of someone else's oversold hardware.
They're doing over 100rps if they're doing 10M requests a day. That's not a good use case for Lambda. If you're going to be that heavily utilized it makes more sense to run your API on EC2 or ECS/Fargate/etc.
Lambda is a good use case for when you have lots of not-often-used APIs. Lambda is a great solution for an API that's called a few times a day. It's also great for when you're first starting out and don't know when or where you'll need to scale.
But once you get to 100rps for an API, it's time to move to a more static setup with autoscaling.
I've always found Troy Hunt's tech stack of haveibeenpwned.com interesting. The API does 5M requests a day with Azure Functions and Cloudflare caching. Ultimately only costing him 2.6c per day.
I think the problem is that moving from Lambda/FaaS to a container-centric alternative (ECS and friends) requires a complete re-architect of your stack. Whereas starting with a simple, single container solution and organically evolving that into container-orchestration is much simpler - because the fundamental building block hasn't changed. It's all just containers.
Personally I'd like to see the industry coalesce on "serverless" containers rather than FaaS which are organised around functions being the fundamental blocks. Please just run my Docker container(s) like a magic block box that is always available, scales as necessary and dies when no longer needed.
Aren't there abstractions to reduce the impedance mismatch between serverless offerings, e.g. Serverless (javascript framework)[1], which should allow easier portability to self-hosted solutions - openfaas or openwhisk etc - including running in containers on more traditional infrastructure, which is cheaper for this particular scale and use-case?
Sure, they're still FaaS which seems to be the unit of deployment for the serverless movement. For (hidden) managed server container deployment, Fargate is the offered solution I believe.
Not necessarily. The reason I decided to try this was exactly because I found a tutorial showing you could easily host a bog-standard ASP.NET web app on Lambda with the serverless framework. I had to add a small startup class, and one config file to our existing app and I was up and running.
Recently we had to test a posibility of our core application being hosted in contenerized environment. It was really hard, because some tweaks we used are not really documented well. Kernel Tweaks had to be tested separately because we were not sure whether they work or not, recource management had to be done from zero level. JVM optimalizarion from zero level. Service Discovery configuration from zero level.
---
Sure it works from the go for punny small apps. But when it comes to huge corporations running extrordinary workloads - no its not that easy.
I think the problem is that moving from Lambda/FaaS to a container-centric alternative (ECS and friends) requires a complete re-architect of your stack.
Not really. I converted a Node/Express API running in lambda using the proxy integration to a Docker/Fargate implementation in less than a couple of hours by following a Fargate tutorial. Most of that time was spent learning enough Docker to do it.
The only difference between the Docker implementation of the API and the lambda implementation was calling a different startup module.
There is nothing magical about any lambda, from the programming standpoint you just add one function that accepts a JSON request and a lambda context.
Converting it to a standalone service (outside of APIs) is usual a matter of wrapping your lambda in something that runs all of the time and routing whatever event you’re using to trigger to a queue.
Agree with you, Lambda would make a lot more sense if it was kind of a scaffold for your application.
Say you only have 1k users and don't want to spend too much time on infrastructure, lambda is a perfect fit for it. Your application is taking off and now you have 100k: just click a button and migrate it to a Fargate/ECS. That would be the perfect world.
AFAIK the only framework that supports this kind of mindset is Zappa (https://www.zappa.io). I use it in production but never had to migrate out of AWS Lambda so I'm not sure about the pain points related to it.
Is there any fundamental reason for this, apart from AWS' pricing model? It seems to me that ideally, serverless should scale from extremely small to very big without too many problems.
You're basically hiring an AWS devops position since you don't need to manage anything yourself. Great for the small startup but not so great for the already established enterprise that has some devops guys anyway.
An AWS DevOps should be able to manage a lot more than one company's lambdas. That's one of the big reasons the cloud can save money. I don't see why it would be different for serverless setups.
"It's also great for when you're first starting out and don't know when or where you'll need to scale."
To me this is probably the most significant benefit, and one that many folks in this discussion strangely seem to be ignoring.
If you launch a startup and it has some success, it's likely you'll run into scaling problems. This is a big, stressful distraction and a serious threat to your customers' confidence when reliability and uptime suffer. Avoiding all that so you can focus on your product and your business is worth paying a premium for.
Infrastructure costs aren't going to bankrupt you as a startup, but infrastructure that keeps falling over, requires constant fiddling, slows you down, and stresses you out just when you're starting to claw your way to early traction very well might.
I thought the point of Lambda wasn't for not so often used APIs but for APIs where you need instant autoscaling where you may need 100 servers in 2 minutes and only need them for 20 minutes.
1. Don’t use .NET, it has terrible startup time. Lambda is all about zero-cost horizontal scaling, but that doesn’t work if your runtime takes 100 ms+ to initialize. The only valid options for performance sensitive functions are JS, Python and Go.
2. Use managed services whenever possible. You should never handle a login event in Lambda, there is Cognito for that.
3. Think in events instead of REST actions. Think about which events have to hit your API, what can be directly processed by managed services or handled by you at the edge. Eg. never upload an image through a Lamdba function, instead upload it directly to S3 via a signed URL and then have S3 emit a change event to trigger downstream processing.
4. Use GraphQL to pool API requests from the front end.
5. Websockets are cheaper for high throughput APIs.
6. Make extensive use of caching. A request that can be served from cache should never hit Lambda.
7. Always factor in labor savings, especially devops.