As we move to more abstract systems I wonder how well we (as in companies) are keeping basic systems management capabilities in place at a personnel level.
At $DAY_JOB we recently scuttled most development efforts for a week for our teams. Our nightly backup job that sanitizes PHI ballooned overtime to, say, 20GB+1Byte and ran out of disk space. Because we are running Kubernetes on Fargate we don’t need a full time operations guy, right?
Commence the company (me) scrambling to learn how to use a Peristent Volume and Peristent Volume Claim because a career programmer should be able to perform systems administration takes because DevOps has the word “Dev” in it, right?
So we lost a week of productivity to disk space, but in reality we lost a week to poor personnel and capabilities.
I worry about the same thing - cloud provider and especially specialty hosted solutions (Heroku) are getting more expensive as dependency on them grows and smaller shops basically have to use them to get off the ground.
In the end what you have is a stack of technology that the company cannot actually maintain on their own. Now I know that 'shoulders of giants' is a thing and that we shouldn't expect everyone to be able to do every part of work that came before them, but as your example shows it is a real business risk when things go south. Part of the original promise from these providers was that there would be a 90%+ reduction in supporting work, when we all know the reality is that you become specialized in solving issues with Heroku/AWS/providerX
> Part of the original promise from these providers was that there would be a 90%+ reduction in supporting work, when we all know the reality is that you become specialized in solving issues with Heroku/AWS/providerX
For all but the most simple setups, providerX solutions are still better than maintaining your own bare metal solutions with their associated hardware / software / datacenter vendor relationships.
Also, it’s not hard figuring out how to stand up a datacenter if it comes down to it and you want to pivot at some point. What’s actually hard is _growing_ and _distributing_ your datacenter to meet demand / legal requirements.
That’s been my preferred model since the 2000s, too. I’ve seen a lot of projects where someone wanted to do something which they couldn’t do with RDS, etc. and every time the results didn’t match the early optimism.
At $DAY_JOB we recently scuttled most development efforts for a week for our teams. Our nightly backup job that sanitizes PHI ballooned overtime to, say, 20GB+1Byte and ran out of disk space. Because we are running Kubernetes on Fargate we don’t need a full time operations guy, right?
Commence the company (me) scrambling to learn how to use a Peristent Volume and Peristent Volume Claim because a career programmer should be able to perform systems administration takes because DevOps has the word “Dev” in it, right?
So we lost a week of productivity to disk space, but in reality we lost a week to poor personnel and capabilities.