
IT blunder permanently erases 145,000 users' personal chats in KPMG's Teams - salesynerd
https://www.theregister.com/2020/08/24/kpmg_microsoft_teams/
======
chmod775
Wow. The third such story in about a week[1][2].

This is the flipside of automation.

People said "store your data in the magically overpriced cloud, it's much
safer than running your own servers! And also automate everything!" However by
now I am convinced that people are more likely to lose data clicking the wrong
thing in the horrifyingly complex mess that is AWS and the like than a single
dedicated server becoming unrecoverable.

Maybe instead of this "automate everything" we should pick and choose which
things should be automated, and what should be performed manually. Probably
shouldn't have automated _data loss_.

Other professions have known the risks of a dangerous thing being _too easy_
or _too fast_ for ages.

And I don't mean "slap another password prompt and a second confirmation
dialog on it". I mean: want to perform some destructive/dangerous action?
You'll have to perform it for each server/customer individually. Yes it's
going to take days, but at least you'll only fuck up for some of them before
you realize your mistake.

"Really? But can we _please_ automate that?!" Sigh. Just make sure it stays
slow and still takes days, so you can stop it before everything is gone.
Incremental rollouts are the second best thing.

[1]:
[https://news.ycombinator.com/item?id=24196131](https://news.ycombinator.com/item?id=24196131)
Canon

[2]:
[https://news.ycombinator.com/item?id=24229864](https://news.ycombinator.com/item?id=24229864)
Adobe

~~~
lowdose
But it is pretty ironic accountant firms sell technology consulting to their
customers while they haven't automated their own core business. Audits start
at 100k for SME's because it is still billed by the hour. On top of this even
the CIO doesn't understand IAM.

------
thevagrant
My limited experience in MS Teams has found a number of features that have no
version history/logs or even basic undelete/archive functionality. It's
unreliable for collaborative work.

Teams Planner - what happens when a Team member deletes tasks or renames
tasks? Good luck finding what happened to the task or who destroyed your
project plan.

Wiki has no clear way to track changes. If someone deletes wiki or pages,
chances are they are gone for good.

Wiki and Planner have plenty of horror stories on the MS Uservoice. Some
requests to fix this basic functionality are over 4 years old (still being
ignored by MS).

------
tdeck
I don't know how many employees KPMG has but many of these chats were probably
destined to be deleted anyway because big companies typically have short
retention windows to limit litigation discovery. The main exception being
material on litigation hold, which makes me cynically wonder what KPMG
"accidentally lost".

------
millzlane
Well at least the users will finally be able to remove the old conversations
clogging up the chat window. That is the best feature the come from them.

[https://microsoftteams.uservoice.com/forums/555103/suggestio...](https://microsoftteams.uservoice.com/forums/555103/suggestions/33535006)

------
ornxka
I have several questions:

Why was this even possible?

Why was this apparently easy enough to do that you could do it by accident?

Why was it impossible to recover from what was clearly an inevitability?

I feel bad for whoever fatfingered the "delete literally fucking everything"
switch but I'm not even sure an individual can actually be said to have been
solely at fault for such a colossal and systemic screwup.

~~~
sushshshsh
The converse of this is “lock everything down so that de facto nobody can make
any changes to it, especially those important new features teed up for the
next sprint"

