There exists a cohort of internal customers at medium-to-large sized software companies that need to ship new applications without being bogged down by the decisions of how to get that thing running and playing nicely with the rest of the ecosystem. For the same reason they don't give their end users CRUD APIs to use their products, it doesn't always make sense to give our developers the entire suite of AWS and say "figure it out".
There's need here but I'm not sure we've figured out what the right answer is.
> a cohort of internal customers at medium-to-large sized software companies that need to ship new applications without being bogged down by the decisions
Well, if it's a medium-to-large sized company they will be bogged down by governance processes anyway. Who approves the budget for this? Who signs off the cybersecurity and data protection compliance policies?
I consider that as something that would need to be abstracted away from individual contributors who aren't part of the infrastructure team(s) and baked into any self-service solution. These are the things that slow teams down and we want sorted before they even think about spinning up a new app.
Right, and I think that strikes at the core of the issue; AWS allows you to configure things in insecure, poorly architected ways. Then they'll tell you "oh this is bad, you should fix that", as if they weren't the ones who built it to do that!
The classic example is public-read S3 buckets. My favorite-er example is actually a security guidance which reads: nothing should use the default VPC (usually phrased from the perspective of, all the SGs on the default VPC should restrict all traffic). Follow that line of logic: why did you give me a default VPC then? Well, they'd argue, it makes onboarding easier, you can just deploy all these cool resources without worrying about the complexities of the networking. Wait hold on. All of this is within their power to control, they made the networking insanely complex, they made the networking layer integral to the Cool Product, they created the idea of a default VPC, they created the security rule that says its bad, and it really paints this picture of a house of cards where no-one is in control, no one has vision, its just a bunch of heads arguing with each other about their worldview on how things should be.
Of course, one head yells "backwards compatibility, legacy, more knobs more knobs" and because its Amazon (and Microsoft) that head gets a megaphone.
Because of this, as the OP wishes, AWS accounts are an extraordinarily useless thing to hand over to a development team. They're nothing like a Heroku or DigitalOcean account; at best, they're so complex that you need a domain expert to understand what to do, so all that ends up living in the CloudOps Org. At worst, the team will shoot themselves in the foot, and maybe take the company's bank account (or worse, customer data) with them.
Thus, the New Hotness in BigBoyTech is to build a PaaS on top of AWS. Because Amazon isn't capable of doing it! Lightsail is a joke. Lambda is probably the closest they've gotten. This PaaS asserts the security, compliance, performance, architecture, etc opinions the company wants, and dev teams can build on it.
Fine, whatever, but at some point you have to question why we're so obsessed with inefficiency that we tolerate knobs we won't use, or we'll miss turning to Secure, when alternatives do exist. Do you know what a compliance official's wet dream is? "Hey man I need somewhere to store this file in the cloud" "Use Google Drive" DONE! They bought Google Drive. Maybe tweaked a couple settings. Secure by default. AWS is nothing like that, and it can take millions in engineering to get an active AWS account to that state (let alone delegated management of multiple accounts across an org).
It's actually collective insanity that we put up with it, when the only half-reason why it is so is because JP Morgan has a COBOL service built in 1987 that can only run if its hardwired with a telephone jack directly to an ISP interchange in Ashburn, and they're too cheap to modernize it so just tax the rest of the world by abusing Amazon's "never say No to a paying customer" policy. I'm being dramatic, but that describes 98% of the knobs on AWS.
as an indie dev who is NOT a DevOps person i'd love more tooling that enabled me to configure AWS and have good safe defaults preset for me. Right now you really need to be a pro or spend a lot of time learning the nitty gritty configuration stuff of services you need but don't actually care about.
Any decent DevOps / SRE / cloud ops type person has been striving to build something like this for a long time and has been frustrated at the kind of experience developers get out of the investment they put into all these crazy tools.
The problems I’ve seen is that like most software in general there is no one size fits all approach to scaffolding and management infrastructure in AWS or any other cloud, especially when we add in the realities of business pivoting being likely.
On the other hand, there are offerings such as AWS Landing Zones that sorta help if you’re running a big ol’ sprawling company but it’s totally overkill and costs more than anyone really wants to pay for if they’re just getting started.
IMO, the AWS journey is probably better off for most folks using Lightsail and scaling with several of those instances making sure they can be containerized and have a brain dead easy deployment workflow that can withstand instance failures before getting more complicated. A lot of people are pushing serverless at first but it’s cumbersome enough for application developers that it’s distracting from the most important thing early on - building your dang product, silly.
Otherwise I’d suggest going hard into containers and using a PaaS until it really, really hurts to where you can afford to pay the eye watering compensation of most decent SREs (expect $15k+ USD / month total spend if you have few advantages in terms of access to the market).
I am not a fan of over-planning infrastructure at all but have also seen what happens when developers take too long to hire someone specialized in infrastructure and has seen the common mistakes that can gut company growth hard as they’re trying to scale up and out.
That was sort of the idea with the various PaaS implementations out there. But invariably they either go complicated (basically transitioning to a Kubernetes-based container platform). Or they were missing some capabilities, making them less interesting to enterprises in particular.
Is there a PaaS out there that enables some guardrails? An example would be "we only want to use Fargate, don't manually provision EC2" and that option just doesn't show up as something you can self-serve.
At my old job, SecDevOps built a metric fuckton of monitoring of all AWS accounts in the Organizations, as well as some pretty complex pipelines that checked the typs of infrastructure being built.
Everything built in AWS was done with Terraform (minus some bootstrapping of accounts and orgs). Custom terraform modules were built which were guardrails of underlying raw Terraform providers.
I'm not saying it's ideal, I'm just throwing it out there.
There exists a cohort of internal customers at medium-to-large sized software companies that need to ship new applications without being bogged down by the decisions of how to get that thing running and playing nicely with the rest of the ecosystem. For the same reason they don't give their end users CRUD APIs to use their products, it doesn't always make sense to give our developers the entire suite of AWS and say "figure it out".
There's need here but I'm not sure we've figured out what the right answer is.