
DE-Cix – Power failure leads to Downtime in germany - usernam33
http://www.heimatzeitung.de/nachrichten/bayern/2904220_Internet-Nutzer-melden-DSL-Stoerungen-Probleme-bei-DE-Cix.html
======
virtuallynathan
Kinda fun to see the increase in AMS-IX traffic: [https://ams-
ix.net/technical/statistics](https://ams-ix.net/technical/statistics)

Keep in mind this is only one of the many DE-CIX datacenters, they exist in at
least 8 datacenters in Frankfurt:
[https://twitter.com/goebelmeier/status/983453047543214081](https://twitter.com/goebelmeier/status/983453047543214081)

Only those peers who are single-homed to Interxion FRA5 / DE-CIX6 will have
experienced an outage.

~~~
AdamJacobMuller
In fairness, we're in FRA8 but DE-CIX is not directly, so we're connected to
their switches in FRA5.

------
usernam33
Imagine it being 8pm, you sit in a operating room of germanys largest internet
knot with up to 6,027 Tbit/s throughput and then the power goes down.

I am concerned that this even happened in a place important like this.

pic related:
[http://img.pr0gramm.com/2018/04/09/9286baa4b2261184.jpg](http://img.pr0gramm.com/2018/04/09/9286baa4b2261184.jpg)

The article states internet was down for users at around 10pm (I expirenced it
live). So they might have been on generators for two hours (speculation).

Shouldn't we decentralize more?

~~~
virtuallynathan
Only those peers who are single-homed to Interxion FRA5 / DE-CIX6 will have
experienced an outage. The whole of DE-CIX isn't down.

~~~
AdamJacobMuller
> Shouldn't we decentralize more?

> The whole of DE-CIX isn't down.

Largely agreed with you, though as general advice I would say that redundancy
by adding more exchanges to your network serves you better than adding diverse
connections to a single exchange.

~~~
virtuallynathan
Yep, that makes a lot of sense, especially since 10G/100G waves and such on
the FLAP route are so inexpensive. Getting a remote peering port on AMS-IX or
something is probably a good move.

------
AdamJacobMuller
We were hit by this.

In fact, we're hit by this again, down again at 22:09:19 EST for me and still
down. Their website is broken too ([https://www.de-cix.net/](https://www.de-
cix.net/)) and their email servers are offline which makes contacting them
amusing.

Not much to be done, really, fortunately the impact for us is very trivial.

------
avh02
General home connectivity also suffered, at least a few people (and myself)
had connectivity issues, if you search for @vodafone_de on twitter you'll see
a lot of people complaining (though I assume it was not just vodafone with
issues)

------
mntnman
Just received an email update from INTERXION acknowledging they did indeed
lose power again at FRA5. This leaves us in a bit of a mess.

