I'm very disappointed that although using Studies (Normandy) is briefly mentioned, it is treated as a matter of course. I was really hoping for an explanation of why they felt they could break their promise to users that Studies would only ever be used for, well, studies.
* All Shield Studies must be designed to answer a specific question - Shield is a tool for evaluating ideas and features in the product. If you are not trying to evaluate a proposed feature or idea there are other means of shipping your code.
Shield studies will always respect user privacy in accordance with our data collection policies
* All Shield studies will adhere to the scientific method for answering complex questions - Generate a hypothesis, test, collect and analyze data, validate or refute hypothesis, refine, repeat..
* All Shield studies require a PHD (Product Hypothesis Doc) - A PHD or Product Hypothesis Doc outlines the overarching research question the study is trying to answer. It requires the study author to think critically about the problem and the outcomes long before the study ships.
These guidelines (https://wiki.mozilla.org/Firefox/Shield/Shield_Studies#Guidi...) were created after the beach of trust represented by the Mr Robot cross-promotion as a means to restore confidence. Yet now they have been ignored, and again "Studies" has been used as a convenient backdoor.
Would you have preferred addons remained broken for the up-to 24 hours they would have been through the regular update system?
I'm not trying to defend what happened here, and I acutely feel the breach of trust. But, I also am trying to understand that this was a lose-lose situation - and the choice had to be made extremely quickly and under immense pressure.
I would have preferred they wait 18 more hours (24 hours worst case instead of 6) and avoid abusing Studies, yes.
Convenience and speed won out over following their privacy policy and observing their promises. That's a huge deal to me, given that integrity is why I choose Firefox.
* All Shield Studies must be designed to answer a specific question - Shield is a tool for evaluating ideas and features in the product. If you are not trying to evaluate a proposed feature or idea there are other means of shipping your code. Shield studies will always respect user privacy in accordance with our data collection policies
* All Shield studies will adhere to the scientific method for answering complex questions - Generate a hypothesis, test, collect and analyze data, validate or refute hypothesis, refine, repeat..
* All Shield studies require a PHD (Product Hypothesis Doc) - A PHD or Product Hypothesis Doc outlines the overarching research question the study is trying to answer. It requires the study author to think critically about the problem and the outcomes long before the study ships.
These guidelines (https://wiki.mozilla.org/Firefox/Shield/Shield_Studies#Guidi...) were created after the beach of trust represented by the Mr Robot cross-promotion as a means to restore confidence. Yet now they have been ignored, and again "Studies" has been used as a convenient backdoor.
Edit: it is covered a bit more in this post (https://hacks.mozilla.org/2019/07/add-ons-outage-post-mortem...), but they still don't address that this usage was in direct violation of their own policies and promises.