Stuff like this happens... but the real question is why FB had to send employees to physically go to datacenters to fix this. Sounds like their OOB management had a dependency on FB being online which seems like a bad design. If you're that paranoid about security issue revokable one time pads to a select set of employees.
