>I'm interested to know what someone who doesn't identify as an early adopter is using for lambda.
I was actually an early adopter of Lambda. At the time I didn't realize it was new, I was just looking for a way to host a new nodejs project in a scalable way. I did not want to spin up an EC2 instance for an Express based project and then have to manage load balancing and everything else that comes with it when the project needed to scale. I thought, why not make it scalable from the start? So I searched and found Lambda, which had launched only a few months earlier. It seemed great and exactly what I was looking for. I didn't realize it at the time that it was so new. I found it during Christmas vacation 2014 where I had a couple of weeks of "downtime" away from work to focus on my personal project and Lambda quickly won me over. Lambda was released in November 2014, so it was really new and I was unknowingly a very early adopter of it.
There was no "Serverless" project at the time. So I created my own toolchain that I still use to this day. It's very simple. The dev program I created simply watches a folder for file changes, and then .ZIPs up the files, sends them to S3, then a companion Lambda function does "npm install" within a Lambda instance, then .ZIPs up the entire package and then updates the Lambda function I'm working on - this is done so any dependencies that require compilation will compile for the Lambda environment and not my local dev machine environment. It just works. I've added support for Layers when those became a thing, but other than that it's very simple and has withstood the test of time. I did try "Serverless" last year for another project, and it was not to my liking at all. I don't like giving "Serverless" access to my AWS account, and lots of other things made it not a good fit for me. YMMV.
Yeah, there are some frustrating times with Lambda, but those are usually related to getting stuff running like a headless browser, or ffmpeg, or other heavier lifts to run inside a Lambda. But, I've got all that stuff working, and once it works, it just keeps working. I haven't touched some of the older Lambdas in years, some are still running on node v12.
The only thing I kind of dislike is decommissioning of older nodejs runtimes so they aren't available at all for new Lambdas anymore. It's not that big a deal until I run into a situation where updating an older Lambda requires me to update the runtime, and there would be breaking changes due to different runtimes. But, it really is best if my code is updated to run on the latest nodejs version, although when I sometimes fall into this situation I start to question my choice of using Lambda. My regret is always short lived though, because Lambda is great in so many other ways.
That's very interesting, thank you for typing it up!
> Express based project
Ahh, so you use a monolith in a lambda. I should have gone this route. I was concerned about package size (due to lambda limits) and so went the "one function per endpoint" route instead. That seemed to be what AWS was saying you should do but it really screws you in a number of ways, size is the only place where it really helps. Provisioned lambdas, for example, are so much better for a monolith.
Did you not have size issues putting all the code in 1 package? That was a real limit I hit when I was (naively) packaging all my code for every endpoint instead of just the handler and the code it needed. I'd be nervous of pushing up against the limit. I guess the escape hatch is moving to a docker container on lambda since that gives you 10GB.
> The only thing I kind of dislike is decommissioning of older nodejs runtimes so they aren't available at all for new Lambdas anymore.
Yep, I've been burned by this and forced to update.
One more question, how do you handle local development? Do you just run Express directly? That's one reason I've leaned into the serverless frameworks out there. My function-per-endpoint setup doesn't have any default scaffolding, running via lambda is the only way to run the code (or one of the project that tries to replicate lambda locally). SST has been the best on this front since you deploy your dev environment like normal (spin up real queues, etc) but the lambdas are just proxying the request to the code running locally on your computer. This has resulted in the least "gotchas" when it comes time to deploy (though arch differences are always a blast).
I abandoned Express when I moved to Lambda. Express was just handing the routing, which I didn't need anymore with Lambda, API Gateway handles that now. My front-end also calls most Lambdas directly using Cognito for auth.
The only time I have problems with Lambdas growing too large is when I try to include a really big payload that includes something like Chrome for headless screenshots, or FFMPEG, or other large binary programs. But for normal Lambda functions I don't have any size issues. Recently when trying to get headless Chrome working in Lambda, I had to boot up EC2 running AWS Linux, and create a .zip file there with library files that were missing in Lambda, then .zip up everything and upload to S3 and then import the .zip into Lambda. After this, I can modify my .js files locally and have my toolchain upload the changes to Lambda to update the Lambda. But this time the payload was too large for importing into a Lambda, and too big for a Lambda layer. The solution was saving the .zip file with Chrome in it to an S3 bucket, and when the Lambda machine starts up, it first downloads the .zip file to /tmp, unzips it, and now the Lambda has the Chrome binaries in the /tmp folder. This bypasses the size limits associated with Lambda. It's rare that I need to do this, and it's not fun, but there are always going to be some edge cases like this with Lambda and getting large architecture specific binaries running in Lambda.
I actually put many different functions into a single Lambda, and I call the Lambda with a specific action parameter to perform a specific function.
Local development... when I create a Lambda function I may add some functionality so I can also run it on the command line if I'm still early in developing a complex function. But most of the time I just use my self-built toolchain which watches my local code for changes, then creates/updates the Lambda in AWS, which takes about 2 to 5 seconds typically. Then I test the code as it runs inside AWS. If I didn't have internet access, then it's impossible to develop in this way, but that's almost never the case.
I was actually an early adopter of Lambda. At the time I didn't realize it was new, I was just looking for a way to host a new nodejs project in a scalable way. I did not want to spin up an EC2 instance for an Express based project and then have to manage load balancing and everything else that comes with it when the project needed to scale. I thought, why not make it scalable from the start? So I searched and found Lambda, which had launched only a few months earlier. It seemed great and exactly what I was looking for. I didn't realize it at the time that it was so new. I found it during Christmas vacation 2014 where I had a couple of weeks of "downtime" away from work to focus on my personal project and Lambda quickly won me over. Lambda was released in November 2014, so it was really new and I was unknowingly a very early adopter of it.
There was no "Serverless" project at the time. So I created my own toolchain that I still use to this day. It's very simple. The dev program I created simply watches a folder for file changes, and then .ZIPs up the files, sends them to S3, then a companion Lambda function does "npm install" within a Lambda instance, then .ZIPs up the entire package and then updates the Lambda function I'm working on - this is done so any dependencies that require compilation will compile for the Lambda environment and not my local dev machine environment. It just works. I've added support for Layers when those became a thing, but other than that it's very simple and has withstood the test of time. I did try "Serverless" last year for another project, and it was not to my liking at all. I don't like giving "Serverless" access to my AWS account, and lots of other things made it not a good fit for me. YMMV.
Yeah, there are some frustrating times with Lambda, but those are usually related to getting stuff running like a headless browser, or ffmpeg, or other heavier lifts to run inside a Lambda. But, I've got all that stuff working, and once it works, it just keeps working. I haven't touched some of the older Lambdas in years, some are still running on node v12.
The only thing I kind of dislike is decommissioning of older nodejs runtimes so they aren't available at all for new Lambdas anymore. It's not that big a deal until I run into a situation where updating an older Lambda requires me to update the runtime, and there would be breaking changes due to different runtimes. But, it really is best if my code is updated to run on the latest nodejs version, although when I sometimes fall into this situation I start to question my choice of using Lambda. My regret is always short lived though, because Lambda is great in so many other ways.