Hacker News new | past | comments | ask | show | jobs | submit login
Due to the fires and power outages in California, oreilly.com is unavailable (oreilly.com)
55 points by accidentaldev 22 days ago | hide | past | web | favorite | 16 comments

No geographic redundancy?

"Well, this is awkward…"

Doesn't quite cover it I think, especially given that these problems are not a bolt out of the blue.

Yeah this is really weird. I remember we had fires in California and one of our Data Centers was at risk. We migrated the apps to another primary DC.

You would think they would move it ASAP if their DC was at risk.

OTOH that's what makes it really awkward.

Surprising that they don't have a geo-failover location.

When I worked in the DR field many years ago all but the smallest companies had some sort of business continuity plan if disaster strikes, but perhaps this is an example of positive bias as only companies that already cared a lot about DR would get a DR consultancy.

Those few companies I saw that had no DR and BCP whatsoever were usually small companies that grew recently and their insurance provider told them to make such plans so they have lower insurance premiums.

Having been through a variety of audits where this is invariably a topic that comes up at couple of different companies you also have to keep in mind that it’s one of those situations where one size doesn’t fit all.

A company with a call center had to have a plan in place for replacing office space (and, yikes, call center personnel) because having people available to answer calls and meet their SLA requirements was the biggest risk. The servers... meh. It wouldn’t be pretty, but grabbing a machine at the nearest Best Buy and picking up one of the semi-recent offsite backups from an external hard drive at the CEO’s house would get everybody through.

At a dev shop with a variety of state and federal government clients the office couldn’t matter less, what was important was geo redundancy, so we had racks in VA and in somewhere out west, I forget where. Another instance where everything probably could have been run (for a few days at least) off a couple off the shelf boxes stuck under someone’s desk, but it would have to be immediately, and no one lived anywhere to do that practically, so the second DC was invested in.

It’s all about identifying your specific risk and figuring out what to do to mitigate it to a reasonable degree without breaking the bank. Apparently Oreilly didn’t think geo redundancy for their main site was worth that trade off.

As a matter of fact, we're on GKE for most of our stuff, having moved out about 6 months ago. There are some remnants left, however, and that's what's causing the issue today.

The site is online, via learning.oreilly.com. The anonymous pages and login were having issues, because the login was tied to something (not familar with the exact details) in the DC offline.

So if you were already logged in, you won't notice much of anything (hopefully!). But if your session expired and you have to re-log in or were trying to see an preview of a book, that's where the problem existed.

Really surprising. Even if they self-host everything, you can still design a cloud based emergency spin up configuration.

Thank god I store all my learning material (among other things) offline, so that my education doesn't have to depend on firefighters from the other side of the planet.

A wise choice, but the economics are a bit different from cloud software, like Adobe: the costs for books and training videos can add up to several thousands of € per year and most of that knowledge will become obsolete in a couple of years.

Many tech books should probably be rented, while the classics and very good ones should indeed be bought.

That's an argument for _allowing books to be rented_, but not an argument for _disallowing books from being bought or downloaded_.

Besides, your argument is already served by libraries. The problem in this case is the publisher wants to be _the library_.

It's all fun and games until you get a GDP drop. For the areas affected, 3 day = 1% of yearly GDP drop. But don't forget the knock-on effects.

I just got an email today from the O'Reilly news letter.

Subject: 'How’d you weather the dot-com bust?'

Maybe they should focus on how to weather the climate bust.

They include a separate link if you’re looking for online learning, which gives you a login page, so clearly the part you’d care about as a customer is still available.

Since that would also seem to be the most intensive part to keep operational I’m confused...

Last night when I tried it, links to the online learning site (both to the login page and to specific chapters of books I'm in the middle of reading) ended up at the page about the fire. The link to the online learning site from that page just came back to that page, making me wonder what the point was.

It doesn't seem unavailable. It's online, innit?

It seems to be a single static page served directly from a machine at their Sebastopol HQ, not from their datacenter in Petaluma

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact