Some of these "cloud" places, though, don't have that kind of oversight. They build their systems to just auto-scale up, and up, and up. Soon, there are dozens, then hundreds, then even thousands of these crazy things. If they write bad code, and their footprint goes up every day, they don't even realize it. Odds are, the teams themselves have no idea how many "machines" (VMs in reality) they are on, or how quickly it's changing.
There's no equivalent of my friend to stand there and tell them to first optimize their terrible code before they can get more hardware. The provisioning and deploy systems will give it away, and so they do.
What usually happens is that someone will periodically run a report and find out something mind-blowing, like "oh wow, service X now runs on 1200 machines". How did this happen? That's easy. Nobody stopped them from needing 1200 machines in their current setup. People wring their hands and promise to do better, but still those stupid instances sit there, burning cash all day and all night.
Not only "cloud", but 3rd party APIs/managed services as well. Slab based pricing, pricing on various rates, pricing on data. what not!
I have come to believe that most of the cloud/API vendors make bill tracking so convoluted and hard on a definitive purpose, just so that people can't get easy visibility and early-warning of impending disasters.
Any firm/outfit that says "we want to use cloud for its elasticity" must be prepared for that "elasticity in the billing" too. If they are not, they will find themselves in deep troubles.
How do backend developers determine what is and is not reasonable performance for a given service?
I see so many stories from companies of "we switched to the right tool for the job and saved so many CPU cycles!". From the outside it sounds like half of the problem is determining you have a problem in the first place.
Well it depends on what the backend serves. It's either "the frontend is not perceived to be sluggish" or "batch jobs don't overrun".
I roll my eyes as I look at my desk with a multimeter, oscilloscope and pile of hardware with multiple (more or less buggy) board revisions full of jumper wires and cables going to serial pin headers..
I shudder to think about all the open tickets about hardware related issues. Of course, probing boards is only partial help since the other half of hardware lives in drivers that may or may not be based on reverse engineering, incomplete or missing specifications, etc.
The network is being odd, sometimes, and I still don't know why, and I can't reproduce it reliably.
Sure, virtualization adds more complexity.. but sometimes it's nice to know that hardware is someone else's problem.
A fun comparison would be the same program implemented as a monolith on metal vs micro services on the cloud. Why monolith on metal will run in cache and not do rpc network calls. That must be like a hundred times faster.
We have global warming programs should be maximum efficient.
Performance numbers everyone should know
In the 80s, I wrote in assembly. In the 90s, I saw Microsoft release operating systems which were incredibly bloated and memory hungry. At the time, I thought it was insanely inefficient.
In hindsight, they were right and I was wrong. It's true that Microsoft's software was bloated, but they were prophetic enough to see that RAM and storage was going to get cheap. The hardware became powerful enough to make the bloat of the operating system unnoticeable.
No it didn't. Lots of stuff that was instant on the computer from the past, is slow on modern computers. Basic things like displaying content of a directory etc.
What changed is public perception - people accept bloated and terrible software as something normal.