
All Circuits Are Busy Now: The 1990 AT&T Long Distance Network Collapse (1995) - beagle3
https://users.csc.calpoly.edu/~jdalbey/SWE/Papers/att_collapse
======
wolfgang42
Longer-form article from 1992, which talks about the history of the phone
system and covers this event:
[https://stuff.mit.edu/hacker/part1.html](https://stuff.mit.edu/hacker/part1.html)

> On January 15, 1990, AT&T's long-distance telephone switching system
> crashed.

> This was a strange, dire, huge event. Sixty thousand people lost their
> telephone service completely. During the nine long hours of frantic effort
> that it took to restore service, some seventy million telephone calls went
> uncompleted.

> Losses of service, known as "outages" in the telco trade, are a known and
> accepted hazard of the telephone business. Hurricanes hit, and phone cables
> get snapped by the thousands. Earthquakes wrench through buried fiber-optic
> lines. Switching stations catch fire and burn to the ground. These things do
> happen. There are contingency plans for them, and decades of experience in
> dealing with them. But the Crash of January 15 was unprecedented. It was
> unbelievably huge, and it occurred for no apparent physical reason.

~~~
vimy
‘Outage’ wasn’t a commonly known word?

~~~
adrianmonk
I don't think it was uncommon, at least not for power outages.

Here's a Washington Post article from 1977 that uses it:

[https://www.washingtonpost.com/archive/politics/1977/07/15/b...](https://www.washingtonpost.com/archive/politics/1977/07/15/blackout-
paralyzes-new-york-city-for-day/eb6f9906-db2c-4561-9ca9-f3d054f01c08/)

And a NY Times article from later in 1977:

[https://www.nytimes.com/1977/10/17/archives/around-the-
natio...](https://www.nytimes.com/1977/10/17/archives/around-the-nation-fire-
at-electricity-plant-blacks-out-san-antonio.html)

------
macintux
This presumably was pulled from the Level 3 outage discussion. Relevant
comment:

[https://news.ycombinator.com/item?id=24323402](https://news.ycombinator.com/item?id=24323402)

phkahler's addendum should be painfully familiar to many:

> Contrary to what that link says, the software was not thoroughly tested.
> Normal testing was bypassed - per management request after a small code
> change.

[https://news.ycombinator.com/item?id=24323904](https://news.ycombinator.com/item?id=24323904)

------
pgrote
It was front page of the NY Times.

[https://www.nytimes.com/1990/01/17/us/at-t-pinpoints-
cause-o...](https://www.nytimes.com/1990/01/17/us/at-t-pinpoints-cause-of-
phone-disruption.html?searchResultPosition=1)

"Company executives said the malfuction occurred in a program at a giant
computer switching station in the New York metropolitan area, causing the
computer to send out alarm messages to other switching stations."

Interesting to see MCI taking the opportunity to talk about backup providers.

------
at-fates-hands
Interestingly enough, this was initially blamed on the Legion of Doom hacker
group:

 _It was a frightening outage that rang alarm bells in more places than just
AT &T headquarters. Months earlier, the Secret Service had arrested a 16-year-
old hacker nicknamed Fry Guy who had told them, under interrogation, that a
hacker gang he was associated with had been planning to crash the system on a
national holiday._

 _That gang, the Legion of Doom, was suddenly on the law enforcement radar
screen in a big way. But they weren’t alone. The Texas-centric group was in
the midst of an online feud with a New York-based rival, the Masters of
Deception. LOD and MOD, as they were known, were at each others throats and
the battleground happened to run through the phone switches and computer
systems of America._

From the article:

[https://www.cybersecuritymastersdegree.org/legion-of-doom-
vs...](https://www.cybersecuritymastersdegree.org/legion-of-doom-vs-the-
masters-of-deception/)

------
alrs
It felt that monumental at the time. I recorded the evening news to VHS that
night.

~~~
rglover
Do you still have the tape?

~~~
alrs
Unlikely.

------
jimmaswell
I'm not sure I really get the bug with the break statement. What should the
code have looked like instead?

------
monkpit
Why are there so many typos...?

~~~
empthought
Optical character recognition.

~~~
monkpit
Makes sense! Thanks!

