Basically, AWS (or any cloud provider) is great for young startup with 0 money, medium sized companies who can optimize to some degree and big sized companies too big to organize efficiently the logistic of buying servers and the instrumentation of infrastructure.
It leaves the mid sized startups which have an interest of buying their own infra, but migrating away from a cloud provider also have a cost of itself.
It's not as clear cut as I am stating, you also have cheaper alternatives than AWS, Azure and GCE. You can go for OVH or DigitalOcean, less possibilities and technical offering, but cheaper prices which can compute with hosting things yourself mid-term.
Cloud *aaS is best suited to several major use-cases:
0. Experimental projects of limited duration
1. "Peaking" overflow capacity for burst of transactions or daily sinus maximum load (/.-resistance)
2. Batch jobs (ephemeral computing)
3. Disaster Recovery/Business Continuity (DR/BCP)
4. Informal IT to bypass bureaucracy
Source: Hi, I'm a former client-facing Fortune 200 AWS consultant from back in the day. I don't own Amazon stock or have any current conflicts-of-interest.
1. AWS resource list, tags and spend.
2. Datadog utilisation.
From this sheet a derivative sheet was created that had functionality on it, so that the data sheet could be regularly updated. The sheet was sorted in order of cost, and a cumulative sum totalled up all the spend. The column next to that gave a cumulative percentage of total spend so you could quickly see how spend was distributed.
There was a set of indicator columns at the end of table calculated by formula which show 0 or 1 dependent on whether the indicator applied. The indicators where things like:
1. Can down-grade instance.
2. Can kill instance.
3. Consider for contract.
4. Cheaper on Azure.
5. Can be on-demand.
6. Is unreliable.
As we thought of new things I'd add them to the spreadsheet. This was of working was very effective.
For predictive autoscaling, boring old-fashioned forecasting techniques appear to work fine and are very fast and very cheap.
For reactive autoscaling, boring old-fashioned control loops appear to work fine and are very fast and very cheap.
There's still no substitute for understanding your application's design.
If you tried to design and build a factory the way folks propose to rely on autoscalers and ML ("we'll ignore what's in the building and rely on a thin, unintelligible gas of floats!!"), you would be fired.
Examples of hard-coded autoscaling strategies:
Shopping cart or ad delivery network -> anticipate load with extra capacity by scaling up instances before they're needed and kept until they aren't beneficial.
Neighborhood social network:
- scale +1 instances after avg latency > X0 ms for Y0 minutes
- scale -1 after avg latency < X1 ms for Y1 minutes
Adding on this, my current thinking is that it would be possible to better surface this tradeoff by modeling autoscaling as an inventory problem. There's a stockout cost and a holding cost; for a given probability of hitting a cold start and for a given cold start lag, it should be possible to compute the optimal instances on-hand.
Then there are, as you point out, many special business rules. Where I work we have a pool system for testing environments. It works reasonably well until a particular team, sitting upstream of approximately half the company, makes a release. Then suddenly everyone's pipelines kick off simultaneously. So now they have a system which watches for signs of impending release and pre-emptively scales up.
Autoscaling should not be your first line or final line of defence. It is a powerful tool, not a magic wand.
It has a few users and I haven't figured out yet if it could be monetized. The basic thing that I wanted was a daily email of my cost changes from AWS.
How have you found trying to integrate Google Cloud and AWS billing? That's where the real value in this type of product is to me.
True, it requires a plan for 1 to 3 years, but it can reduce your bill by a significant amount.
Also, wasn't AWS deprecating links like this?