Reaching into content windows is forbidden in multiprocess Firefox. That alone is going to break tons of addons. This necessitates a redesign. Since a major redesign is necessary anyway, it makes sense to future-proof the architecture so that addons will work in perpetuity. Ultimately, this ends up being friendlier to addon developers, since addons will break once instead of again and again as the architecture evolves.
> Wouldn't it be sufficient to write safer, less exploitable software? Isn't that part of the goal of Rust for example?
A Rust-based browser engine will never replicate XUL and XPCOM, because no browser engine besides Gecko can replicate XUL and XPCOM. That is because XUL and XPCOM are too ill-specified and tied to the exact internals of Gecko. As long as add-ons are written to an unspecified, ad-hoc API, it'll be impossible to change the underlying technology, preventing adoption of things like Rust. In the end, that holds back browser security.