Serverless isn't saving me any of that pain. I still have to configure ACLs for everything, web folders in S3, ensure that my backend isn't hitting any concurrency or timeout limits, ensure that all of my routes are set up properly in the API gateway, that my DB queries are properly tuned and that someone is watching that DB...
All it's really buying me is not having to write Ansible scripts, but instead I have to write CloudFormation templates. Sure, I have to think about maintenance and troubleshooting less, but when I do have to do troubleshooting, I'm for a long, frustrating day.
As easy as it is to create VMs anymore, the serverless story is nowhere near as compelling as it should be. It makes the easy things easier, and the hard things really f'ing hard.
Concrete example: I wanted to just store the contents of a github webhook post in S3, after verifying the hashed secret. Should be a simple case of wiring together the API gateway to S3, right?
First bug: You can't test an API Gateway -> S3 connection if they are both in the same region. Known issue from back in 2014.
First hurdle: You can't pass the contents of a post to an auth hook in API gateway, just a single header value. That means I can't use the API Gateway authentication hook for this purpose; github creates a HMAC hash of the post contents.
By this point, I was wishing I had just set up a t2.tiny server running flask.
I'm currently working on a side project (which if successful can easily be a product in itself) to create a service that would be the Digital Ocean of FaaS (lots of talk - I understand).
What would be the core features that devs/users/managers are looking for? (something you downright expect for v1 that would compel you to put the service to use for development/production)
Currently some of the items top of my list (in no particular order) are:
* API development (like AWS's APIG)
* Support for more languages by default
* Ability for users to a custom defined language easily
* Creation of an opinionated framework / workflow for FaaS development (you don't need to do this if you don't want to...but FaaS first apps would be done better/easier this way)
* Rewriting common products (Wordpress, Ghost, Jekyll etc.) for to be served via FaaS.
Things I don't want to work on (or would prefer to not jump into):
* Storage service (S3 alternatives)
* Databases (there are so many out there...building a cloud is hard, building a database is even more so)
Who do I feel would pay for this?
* Front end & Mobile devs who don't want to complicate a backend
* Mobile devs who don't want to complicate a backend
* Enterprises maybe?
2) Multi-datacenter -- I want to deploy to two places at least so my risk goes down
The lack of database and/or storage would be a big showstopper. It would mean that if I want build anything with state, I need to store it in another service, which costs money to move data out of. Even a very basic key store would better than nothing. You could actually build one pretty easily on top of Postgres or even SQLLite (although you'd need one per customer w/ SQLLite).
Edit: To the downvoters: self promotion is allowed on this site, and in fact encouraged!
1. Loop detection does seem harder to implement but I'll definitely look into it (if the api service is used, it would be trivial to implement)
2. but multi-datacenter has almost been implemented (you can select from different service providers, regions or a combinations of each - and the functions would scale accordingly).
As for a database/key store...would integrating with database solutions from other cloud providers be of some consideration? (from GC / AWS / Azure?).
Keep in mind that all users would have access to environment variables (across projects/functions etc.) by default.
I would also like to have something I could point at a docker api which would manage the end to end lifecycle of a function. Building it into a container, exposing itself over an API for management, telling it when it should run and of course running it on my docker machine(s). This idea I don't imagine is appealing to your audience however.
> I would also like to have something I could point at a docker api which would manage the end to end lifecycle of a function. Building it into a container, exposing itself over an API for management, telling it when it should run and of course running it on my docker machine(s).
Could you please go into a bit more detail? I don't understand what you are trying to do. (or it's use case)
However...it does have some current caveats (can only run on a single machine, and definitely not something you'd want to run in production).
But it's (with more features obviously) something I've considered for v1 as an enterprise/maybe OSS feature (as most enterprises would love to host it themselves/behind their firewall/on their private cloud)
But, to answer your question, AWS does constitute lock-in and migrating away from AWS is a major pain. However, companies from big to small see this as a price worth paying for the flexibility of AWS.
That said, I don't think it's the only viable kid on the block any more. The competition, e.g. Google, is getting to the point where choosing AWS is no longer an automatic and should be weighed carefully. This is where the downside of lock-in starts to factor in.
Are you worried about vendor lock in with your power company?
P.S. As we're both in the same region of California, we're both with PG&E and can't do anything about it. It's kind of okay because they're so regulated, but how do you know we're getting a good price per kWh?
Disclosure: I work on Google Cloud (and met jedberg in person once).
As for comparing power utilities to public cloud, I don't think the two are remotely comparable within our timeframes -- utilities are highly regulated and provide an almost entirely undifferentiated service. Public clouds are the opposite, and that's why you want to be mindful of developing your codebase to an API you pay rent for.
And comparing Amazon to heavily regulated power company does't make sense.
Vendor lock-in is a well understood anti-pattern. I don't think I'm required to defend it.
> Are you worried about vendor lock in with your power company?
... yes? You aren't?
When Google App engine came out it was a big concern but now it's hip.
We use SWF to trigger over 50 separate lambda functions in processing. We've got some very nice internally developed tools to identify which functions are out of date and help with deployment. I'm just curious what else is available to handle DevOps tasks in a Serverless environment (i.e. deploying library updates, etc).
This will flush hot lambdas, so if you have a lot of traffic it can be a bit disruptive. I can imagine comparing deployment and commit timetamps make-style, or calling update_function_configuration(Description=...) to save and later compare version/commit metadata.
AWS recently came out with https://github.com/awslabs/chalice, which is pretty sweet if you're using Python (although it doesn't seem to directly address the concern of deployment orchestration).
API-Gateway => Lambda isn't fun to trouble shoot.
That said, once you get it working, it works really well, and it's ridiculously cheap.
The biggest pains are:
* Mapping parameters in API Gateway
* Setting the IAM Roles for API Gateway & Lambda (the latter is required to provide execution for API Gateway)
* Setting up custom domains for your API (it uses cloudfront, but must be done via the commandline, and the process kind of sucks)
I've done this work now 3 times. Feel free to email me any questions you have: firstname.lastname@example.org and I'll be happy to answer, and provide guidance.
pip install zappa
It supports all of the Languages for Lambda, not just Python.
FYI it is also possible to go directly from the API Gateway to S3 without going thru Lambda
1. I need to allow other people to simultaneously evaluate stories, and a database works well to coordinate multiple people accessing the story queue.
2. Having metadata and stories in dynamo/S3 allows for much more flexible querying of ALL stories, to look for patterns, model the data, etc.
Content is stored in our public github repo, and we use TravisCI to build Jekyll pages on commit/merge, which are uploaded and served from S3.
Interactive functionality is provided via React in the front end, and API-Gateway/ AWS Lambda in the backend.
The API Gateway/Lambda provide various proxies to our various sandbox API environments, so Developers can quickly try our APIs directly on the site.