

The Classic Unix Horror Story - theunamedguy
http://www.lug.wsu.edu/node/414

======
PeterWhittaker
I did something similar once, but realized pretty quickly I had been in the
wrong directory when I hit return after "rm -rf *" \- I cannot remember why I
realized so quickly, it's been over 20 years, but it seems to me I had
multiple terminal sessions going (probably via screen) and freaky stuff was
happening in one.

If I remember correctly, I was in /dev when I meant to be in a sub-directory
I'd created for testing purposes. Some of the freaky stuff included losing
command echo, e.g., since my TTY no longer existed.

Fortunately, since my work involved backups, modems, and a few other things, I
had a pretty good grasp of major and minor numbers and was able to recreate my
TTY using mknod. After that, panic dropped considerably and I could at least
poke around and figure out exactly how borked the system was.

Another time, a colleague and I spent hours trying to restore from tape to a
filesystem we were pretty sure was corrupted. We tried every trick we knew -
and learned a few more along the way - but eventually, at least in part
because we were stressed out and sleep deprived, we had something along the
lines of the following conversation:

"The FS is borked, right?"

"Right."

"We're confident the backups are good, right?"

"Right." (We'd built the backup system, it was in use on hundreds of servers
across the company, we'd never had a failed recovery....)

"So let's reboot into single user, newfs the disk, restore from tape, and boot
back into multiuser."

"Uh..."

"I mean, we cannot possibly make things worse, right? This machine is
thrashing."

"Yeah...."

"So?"

"OK."

It worked. When we returned to our senses, we realized we were lucky it had,
realized we were deluding ourselves (we could have made things worse), and
realized that if it hadn't of worked, we would have been fired in all
likelihood: We'd made a decision way above our pay grade, as it were, with no
management knowledge at all.

The next day, as far as they were concerned, we'd spent hours slaving after
midnight to save a build machine and we kept the design group working. We just
smiled, awkwardly.

Oi.

------
gumby
Nowadays you could sync and then force a shutdown (hard), then mount your
drive on another machine. A hard thing to do in the days of vaxes.

~~~
greenyoda
On the first machine I used Unix on, a PDP 11/45, we had the root file system
on an RK05[1], which was a removable media disk - all of 2.5 MB! So if you
fried the root file system, you could just pop it out and replace it with a
backup disk pack.

[1] [https://en.wikipedia.org/wiki/RK05](https://en.wikipedia.org/wiki/RK05)

