Goodhart's law also applies, as in the case of the edoor bolts, Spirit intentionally bypassed safety controls to meet performance metrics.
The Mars Climate Orbiter is another example. While unit conversion was the scapegoat, the real cause of the crash is that when people noticed that there was a problem they were dismissed.
The Andon cord from the Toyota Production System wasn't present due to culture problems.
Same thing with impact scores in software reducing quality and customer value.
If you intentionally or through metrics incentivize cutting corners it will be the cost of quality and safety.
I am glad they called out the culture problem here. This is not something that is fixable under more controls, it requires cultural changes.
> The Mars Climate Orbiter is another example. While unit conversion was the scapegoat, the real cause of the crash is that when people noticed that there was a problem they were dismissed.
A decade later a senior engineer at NASA warned about a piece of foam striking Space Shuttle Columbia and requested they use existing military satellites to check for damage. She was ignored by NASA leadership, and following (coincidentally) a report by Boeing concluding nothing was wrong, another 7 people were killed by a piss-poor safety culture. (https://abcnews.go.com/Technology/story?id=97600&page=1)
Goodhart's law also applies, as in the case of the edoor bolts, Spirit intentionally bypassed safety controls to meet performance metrics.
The Mars Climate Orbiter is another example. While unit conversion was the scapegoat, the real cause of the crash is that when people noticed that there was a problem they were dismissed.
The Andon cord from the Toyota Production System wasn't present due to culture problems.
Same thing with impact scores in software reducing quality and customer value.
If you intentionally or through metrics incentivize cutting corners it will be the cost of quality and safety.
I am glad they called out the culture problem here. This is not something that is fixable under more controls, it requires cultural changes.