The way I think about it this: not using the cloud is like building your own code editor or IDE after assembling your own laptops and desktops. It may make you happier and it’s a great hobby, but if you’re trying to run a business you need to do a cost benefit analysis.
We currently have double digit petabytes of data stored in our own data centres, but we’re moving it to S3 because we have far better things to do with our engineers that replace drives all day, and engineering plus hardware is more expensive than S3 Deep Archive - but it wasn’t until Deep Archive came out.
We put out hundreds of petabytes of bandwidth, and AWS is horribly expensive at first glance, but if you’re at that scale you negotiate private pricing that brings it to spitting distance of using a COLO or Linode/Hetzner/OVH - the distance is small enough that the advantages of AWS outweigh it, and it allows us to run our business at known and predictable margins.
Besides variability (most of our servers are shut nights, weekends and when not required), op ex vs cap ex, spikes in load (100X to 1000X baseline when tickets open), there’s also the advantage of not needing ops engineers and being to handle infrastructure with code. If you have a lot of ops people and don’t need any of the advantages, and you have lots of money lying around that you can use on cap ex, and you have a predictable load pattern, and you’ve done a clear cost benefit analysis to determine that building your own is cheaper, you should totally do that. Doesn’t matter what others are doing.
You're not building everything from scratch on-prem.
There are excellent tools for deploying and managing infrastructure like Terraform, Ansible, Puppet that funny enough are used to deploy to cloud as well.
Add a self-hosted Kubernetes cluster to that and your on-prem is not that different than a cloud.
As for ops people - you might not need an engineer to replace failed hard-drives, but you'll need a DevOps person to manage Cloud Formation templates and such, and they cost more.
maybe it doesn't need a SV-salary engineer to swap harddrives. Just pick (hehe) three good ones of the AMZ-warehouse people, train him/her and you'll have a very happy employee which is around fulltime (or three of them) and paid well (1/3 of your engineers salary). They'll enjoy working predictable shifts in a nice office and in time you can even train them to do 2nd level support.
We currently have double digit petabytes of data stored in our own data centres, but we’re moving it to S3 because we have far better things to do with our engineers that replace drives all day, and engineering plus hardware is more expensive than S3 Deep Archive - but it wasn’t until Deep Archive came out.
We put out hundreds of petabytes of bandwidth, and AWS is horribly expensive at first glance, but if you’re at that scale you negotiate private pricing that brings it to spitting distance of using a COLO or Linode/Hetzner/OVH - the distance is small enough that the advantages of AWS outweigh it, and it allows us to run our business at known and predictable margins.
Besides variability (most of our servers are shut nights, weekends and when not required), op ex vs cap ex, spikes in load (100X to 1000X baseline when tickets open), there’s also the advantage of not needing ops engineers and being to handle infrastructure with code. If you have a lot of ops people and don’t need any of the advantages, and you have lots of money lying around that you can use on cap ex, and you have a predictable load pattern, and you’ve done a clear cost benefit analysis to determine that building your own is cheaper, you should totally do that. Doesn’t matter what others are doing.