- Power failure where all of the above occur at once?
- Power surge big enough to take out all the breakers, UPS, servers, desktops, and laptops at once?
- Ransomware that had time to take control of the whole network including backups?
- Theft of a single laptop?
- People, yea I agree with people can be an issue.
And all of the above across multiple offices with WFH staff and on call staff?
Of all these catastrophic failures it's just as likely to be a fire, or earthquake, etc, where you should have multiple layers of redundancy in place anyway.
> Power failure where all of the above occur at once?
Sure, it's easy enough. I've had UPSes that only noticibly failed once the power failed, and those outages outlasted my fully charged laptop battery.
> Power surge big enough to take out all the breakers, UPS, servers, desktops, and laptops at once?
Sure! One lightning strike to an unprotected network link, for example, can threaten to do just that. https://www.youtube.com/watch?v=Ev0PL892zSE&t=354s (note: he even added some protection and another lightning storm still caused damage... so he resorted to fiber!)
Even if it doesn't destroy all of your gear, destroying all the main gear your IT crew has login passwords for can slow things way down as they figure out alternatives.
> Ransomware that had time to take control of the whole network including backups?
Even if it hasn't taken over the whole network, you might not be sure which nodes have been taken over (ransom messages might not appear until a lot of data has already been encrypted), and may wish to leave more sensitive nodes offline for forensics, or to preserve data that hasn't been backed up yet, or to avoid needing to resort to slower restoration from offline backups.
Hopefully you have offsite and cold nodes... good reason to have some bootable USBs with recovery images on 'em ready to go as well.
> Theft of a single laptop?
"but that's not a guarantee" was meant to point out that larger scale theft beyond the typical "just a single laptop" isn't unheard of.
> And all of the above across multiple offices with WFH staff and on call staff?
Runbooks can potentially be useful for single-office businesses, and means fewer games of telephone relay even for multi-office businesses.
> Of all these catastrophic failures it's just as likely to be a fire, or earthquake, etc, where you should have multiple layers of redundancy in place anyway.
And runbooks can point you towards those redundancies, many of which may require manual intervention by design (e.g. restoring from cold offsite backups.) You'll note such things as evacuation plans are often printed and displayed prominently near emergency exits, not left in a git repository ;)
And, of course, just because you should have multiple layers of redundancy, doesn't mean you actually have multiple layers of redundancy, even if you think you do.
- Power surge big enough to take out all the breakers, UPS, servers, desktops, and laptops at once?
- Ransomware that had time to take control of the whole network including backups?
- Theft of a single laptop?
- People, yea I agree with people can be an issue.
And all of the above across multiple offices with WFH staff and on call staff?
Of all these catastrophic failures it's just as likely to be a fire, or earthquake, etc, where you should have multiple layers of redundancy in place anyway.