> a library worker doesn't need their computer at 9am so badly that they need an on-call IT worker (and presumably some kind of middle-of-the-night monitoring) to ensure it before they get in.
A single worker probably doesn't, yes. In contrast, hundreds of people being unable to do anything when they come to work in the morning is not a terrible reason to call someone and have them login for a few minutes.
Back to the IA, it seems to me our hundreds of thousands of users, employees and volunteers in time zones across the globe will be unable to access the service until we fix this is a decent reason for an institution that takes its own mission seriously to have an on-call. But you do you.
> Hell, most banks in the US are "closed" for maintenance online for several hours every weekend (or at least they were as of a couple years ago) and no one cares. It's fine.
(this is my industry) Some of their online services are closed, yes. The hours they'll be closed are typically posted ahead of time to minimize the inconvenience to everyone concerned and to display an understanding on the part of the institution that downtime is taken seriously and to signal it is not the result of carelessness. In contrast to online services, most institutions consider it a major failure if something like the ATM system is down for more than a few minutes.
> This attitude is how you waste a ton of money overcomplicating everything while making working conditions terrible for literally no reason.
That's remarkably backwards. The need for humans to be on-call is a result of systems being overcomplicated, not a cause.
If it's only a few minutes, you could just as well fix it at the start of the day. And if it's not, oh well? People can still read books. The librarian can still help them to find a book. If they want to manually track until the computer is back up, they could even still do checkouts.
If those users just say "oh, okay, I'll check back in a couple hours", then okay. Again, this is not hundreds of thousands of people without power or water or something. It's viewing or uploading to an archive. Like, if you're a user, do you really want someone woken up at 3am because you can't use the wayback machine right now? Because "they should take it seriously"? There's a person actually having to deal with that because... you really want to get that one Grateful Dead concert video from their free archive right now? I'm happy to say that yeah I hope their engineers aren't on-call after hours. I would much prefer that they get a good night's rest rather than fix it for me. I hope they're a cool place to work without delusional, psychopathic management.
Perhaps put another way, is there any computer service now that doesn't need a guarantee? Or is everything just critical now?
My point isn't that on-call is overcomplicating, it's that inventing and chasing an SLA often leads to expensive complicated systems (which, ironically, can often directly lead to lower uptime), and on-call has a human cost.
At this point I think you’re just committed to the bit but you cannot possibly believe what you’re writing, as it’s a blinkered analysis and truly misanthropic in its effect. A metropolitan library system employs hundreds of people in a variety of roles and if we can prevent them from having a frustrating and wasteful start to their day, it’s worth an on-call person’s time.
> and on-call has a human cost.
In the context of this thread, we’re talking about this job description:
Senior Datacenter Network Infrastructure Engineer Full Time San Francisco, CA, US 30+ days ago Requisition ID: XXXX Salary Range: $140,000.00 To $200,000.00 Annually
It’s not like we’re talking about indentured servitude or something.
> Perhaps put another way, is there any computer service now that doesn't need a guarantee? Or is everything just critical now?
That’s an interesting strawman you have there. I thought I killed it upthread when I wrote “Depends on the service.”
No, I quite strongly believe it actually. I likewise find it hard to believe you're serious, so I'll raise my question again: if not even a municipal library (which, I'll remind you, aren't even open the majority of the time and we all get by just fine), what is an example of a non-critical service in your mind? Maybe the online checkin for a specific Great Clips? It depends on what? Pointing out that "it depends on the service", and that a library clearly isn't critical was my whole original point.
Misanthropic to me is making every IT job require getting up in the middle of the night to save a 100*epsilon of theoretical frustration lest we be perceived as lacking the requisite seriousness. And when someone points out that actually a given case is really not a big deal, we're met with "but hundreds of thousands of people might think 'oh, darn' for a moment!" And for a library, IME librarians tend to be really chill, so I suspect they'd, like me, attest that actually they'd prefer you not work at midnight on their account. Like I said, as a citizen funding and using my library, I sure hope they don't treat their employees that way.
> a municipal library (which, I'll remind you, aren't even open the majority of the time and we all get by just fine),
At least half of a modern library's lending happens digitally. But let's put aside the example of the library.
> what is an example of a non-critical service in your mind?
I thought the example above of hundreds of people being unable to start their work day until someone solves an especially serious IT problem was universal, not unique to a library system's network. It's relevant to the Internet Archive, and even relevant to...
> Maybe the online checkin for a specific Great Clips?
Maybe not? Their entire WAN being down, affecting dozens of stores, possibly yes?
I'm not sure what's difficult here. Paying a person to be available to deal with a problem serious enough to justify the call makes sense. Sending trivial, less serious problems that can easily wait until business hours their way at night makes less sense.
Taking those calls isn't necessarily a bad job, but somehow that assumption is baked into what you're saying.
> Misanthropic to me is making every IT job require getting up in the middle of the night
This is really interesting to me. I wonder how many more hours of downtime The Archive would have experienced in 2025 if no one was ever woken up out of bed to fix it. (If anyone was)
I also wonder the cost differential.
Got me thinking about these incredible beautiful museums that are closed all night long… you’d still want someone there early enough to catch if the roof leaked so nobody missed their opening-hour tours. Back to libraries, bummer if I can’t renew my book online one night but would set reminder to try again later. Could affect my perception of my library’s tech prowess! That’s a risk for TIA! (Though I experience many a delay and downtime… perhaps my fault for using iCloud Private Relay like I bet scrapers sometimes abuse… but a few more hours overnight here and there wouldn’t be catastrophic for my current perception.)
Mainly would want overnight support to be cheap since stuff’s donation run. - Love TIA! An internet-billionaire should make history becoming a billionaire donor today (hope one’s reading this :) ).
World class art museums have some staff on site 24/7, FWIW.
> I wonder how many more hours of downtime The Archive would have experienced in 2025 if no one was ever woken up out of bed to fix it. (If anyone was)
Wasn't 2025 the year a kooky pro-Palestinian hacker collective decided IA was a good target for their idiocy? There was a great deal of overtime involved there.
A single worker probably doesn't, yes. In contrast, hundreds of people being unable to do anything when they come to work in the morning is not a terrible reason to call someone and have them login for a few minutes.
Back to the IA, it seems to me our hundreds of thousands of users, employees and volunteers in time zones across the globe will be unable to access the service until we fix this is a decent reason for an institution that takes its own mission seriously to have an on-call. But you do you.
> Hell, most banks in the US are "closed" for maintenance online for several hours every weekend (or at least they were as of a couple years ago) and no one cares. It's fine.
(this is my industry) Some of their online services are closed, yes. The hours they'll be closed are typically posted ahead of time to minimize the inconvenience to everyone concerned and to display an understanding on the part of the institution that downtime is taken seriously and to signal it is not the result of carelessness. In contrast to online services, most institutions consider it a major failure if something like the ATM system is down for more than a few minutes.
> This attitude is how you waste a ton of money overcomplicating everything while making working conditions terrible for literally no reason.
That's remarkably backwards. The need for humans to be on-call is a result of systems being overcomplicated, not a cause.