Unfortunately, this would have meant no Firefox Quantum.
As a Firefox dev (I'm still working at Mozilla, although not much on Firefox atm), I have seen many, many occurrences in which I couldn't optimize codepaths, or even in some case fix bugs, because the old extension mechanism made it impossible.
Consider the necessary steps:
1. realize that an internal API is broken;
2. come up with a new non-broken API;
3. port all the internal code using the non-broken API;
4. add a compatibility layer between the broken API and the non-broken API;
5. check all the existing add-ons to find out which ones use the broken API;
6. hope you didn't forget any add-on;
7. attempt to get in touch with all the add-on developers;
8. repeat 7. many, many times, until you are sure that the add-on developers that do not respond have simply abandoned their add-on;
9. negotiate a transition plan with the add-on developer with whom you have managed to get in touch;
10. land the patch that you have written now 3-4 months ago;
11. maintain both the broken API and the non-broken API (and their tests) for ~1 year, until you are reasonably sure that all add-on developers who intend to migrate have done so;
12. maintain (and test) a downgrade path for people who switch between versions of Firefox;
13. finally land your code;
14. realize that you still have accidentally broken some add-ons and people are (rightfully) unhappy because "Firefox broke my add-on";
15. it's 18 months since you wrote your 2-lines patch, you can finally get rid of the dead code and tests and move to something else.
This was one of the reasons for which the Chrome teams managed to be faster and more efficient than the Firefox teams (well, that and a bazillion dollars to hire way more people). The add-on architecture is the main reason for which projects such as multi-processes only landed ~8 years after we had working prototypes and some other performance projects never landed at all.
So, yes, removing the add-on architecture is definitely painful for a number of Firefox users, but I believe that we could not postpone it any further, even if it meant that some useful addons could not be ported immediately. Also, for what it's worth, we have postponed it by something like 7 years already :)
I'm sure you'll hear a zillion complaints, but I really appreciate you folks doing this. I've been using a beta version for the last few months on one of my computers and it's so very much better. I can't wait to have it everywhere.
As long as you're here: I was told that the old extension API was way too broad, locking in a lot of design choices that were not really considered from the "do we want to maintain this for years and years" perspective. And that the new one is much more focused and considered. Is that the case?
> As long as you're here: I was told that the old extension API was way too broad, locking in a lot of design choices that were not really considered from the "do we want to maintain this for years and years" perspective. And that the new one is much more focused and considered. Is that the case?
Definitely. The old extension mechanism was basically "here is the toolkit we are using to build Firefox, come and plug anywhere/replace anything". If my memory serves correctly, at the time, Mozilla Browser/Firefox was (almost) the only browser doing any kind of extensions (I'm not counting M3 and a few experimental/academic browsers), so there was no real precedent on how to do this.
For a time, it worked extremely well. After all, much of today's Firefox is built from add-ons that were progressively integrated in the browser. And then, progressively, we realized that there were drawbacks to this "anything goes" approach, but we couldn't fix things because that would mean breaking thousands of add-ons.
So, after many years trying to postpone the inevitable for the sake of our users, we finally switched to a much better defined API. This WebExtension API is much smaller, much better documented, and does not expose internals-only stuff. Which means that now, we can fix internals-only stuff without breaking the API. Which should make the life of both Firefox developers, add-on developers and users much better :)
> This WebExtension API is much smaller, much better documented, and does not expose internals-only stuff.
It also doesn't expose a lot of stuff that's useful and not tied to Firefox internals in any way.
The browser is one of the most heavily used programs on people's computers. Integrating it with the rest of your system and workflow can have huge payoffs in user experience and productivity. The traditional XUL-based extension system, while not always pretty, allowed for that. WebExtensions are severely lacking here and some of that seems by design.
As an example, I'm still trying to figure out a non-insane way to implement something akin to the It's All Text extension that allows editing text areas on websites using a proper text editor.
I just checked the It's All Text github repo, and their suggested replacement is a thing called GhostText¹, which actually seems a good deal fancier that IAT ever was.
Looking at GhostText and its editor integration scripts, those basically work by running a websocket server on localhost. Implementing access controls seems to be left as an exercise for the reader.
If I understand the native messaging API correctly, nothing in there requires the presence of any networking daemon. The browser just execs a program you can communicate with using JSON messages on stdin/stdout. That is already a lot better than I thought.
I tried that, too. There is no such method at this time that doesn't involve setting up something in between the browser and the editor to marshal between the browser's native messaging API [1] and the editor's desire to operate on text files. I don't think that's insane at all, but the complexity involved did exceed my frankly passing desire to get IAT back - I barely ever use it any more. It shouldn't take a reasonably experienced extension developer more than a day or so to implement, though, I would think.
I believe IE also had extensions (the channel bar in Windows 98, and later toolbars and ActiveX that partly lead to Firefox becoming so popular). It certainly didn't have as much flexibility as XUL though!
I hate to be the person to say this, but perhaps a better approach would have been to simply create a build pipeline that iterated over the addons on AMO to find the ones that use functions from the changed API and bulk send out an email about the changes to the authors (this eliminates steps 5 through 8). The addon developers then get the chance to change it or not. If it's broken oh well (eliminating steps 8 through 15). Making it the responsibility of the FF core team to sit on changes while waiting for a reply is a procedural problem not a problem with XUL/XPCOM. Moving to Web Extensions, which fundamentally make it impossible to access the full file system, permanently breaking many useful addons with no functional way to migrate to FF57 and calling that better is frankly dishonest.
> I hate to be the person to say this, but perhaps a better approach would have been to simply create a build pipeline that iterated over the addons on AMO to find the ones that use functions from the changed API and bulk send out an email about the changes to the authors (this eliminates steps 5 through 8).
This would have automated steps 5 through 8, but not significantly reduced the problem.
> The addon developers then get the chance to change it or not. If it's broken oh well (eliminating steps 8 through 15).
Ah, well, sure, in that case, randomly breaking add-ons all the time would indeed have made our life easier. But everybody else's life would have been much worse, so we decided not to do that :)
> Making it the responsibility of the FF core team to sit on changes while waiting for a reply is a procedural problem not a problem with XUL/XPCOM.
It's a problem of the combination of having no API (i.e. XUL/XPCOM) and not wanting to break user's add-ons.
> Moving to Web Extensions, which fundamentally make it impossible to access the full file system, permanently breaking many useful addons with no functional way to migrate to FF57 and calling that better is frankly dishonest.
Let's just say that we have different priorities. While it's not as powerful, it's better for security, performance, privacy, bugs and future-proofing.
>Ah, well, sure, in that case, randomly breaking add-ons all the time would indeed have made our life easier. But everybody else's life would have been much worse, so we decided not to do that :)
but you did exactly that
>Let's just say that we have different priorities.
seems so, your priority became... trying to speed up hoping that people too dumb to care would switch back from chrome despite it's horrendous and unethical marketing while sacrificing everything that made your browser viable
> Ah, well, sure, in that case, randomly breaking add-ons all the time would indeed have made our life easier. But everybody else's life would have been much worse, so we decided not to do that :)
This is the part that I think you are going to get the most flak for. People are willing to deal with temporary setbacks if the changes can be brought in at some point. Permanently breaking things and calling that better will get the Mozilla Foundation a mountain of angry hate mail from people who committed to the platform.
> Let's just say that we have different priorities. While it's not as powerful, it's better for security, performance, privacy, bugs and future-proofing.
I have no problem if the Mozilla Foundation or the Firefox team has different priorities, but say that rather than telling technical people the reason for the changes is because XUL or XPCOM are somehow so hideous the team had no choice. It smacks of dishonesty when everything that you have described here is a problem with procedure not anything technical.
> I have no problem if the Mozilla Foundation or the Firefox team has different priorities, but say that rather than telling technical people the reason for the changes is because XUL or XPCOM are somehow so hideous the team had no choice. It smacks of dishonesty when everything that you have described here is a problem with procedure not anything technical.
Well, some of our priorities with WebExtensions are (not necessarily in this order):
- stable, documented, future-proof API;
- improving security;
- improving performance;
- improving privacy.
You are, of course, free to consider these things "not anything technical", but they were impossible as long as add-ons weren't based on an API at all.
So, again, while I fully realize that there is a cost, I believe that we're moving from something unsustainable to something sane, which makes it better in the long run.
> I believe that we're moving from something unsustainable to something sane
It is only unsustainable because the FF team chose to make the process more difficult than it had to be. This is how what you are saying sounds:
1. We didn't want to break plugins so we involved addon developers
2. The process takes so long that it takes 18 months to introduce any new code
3. Since 2 was so slow we decided instead we would PERMANENTLY break plugins with no way to ever fix them
Put another way: things were taking too long because of Mozilla's own self-imposed guidelines so the Firefox team had no choice but to PERMANENTLY break addons that will never be fixable by design because the Firefox team was so concerned about temporarily breaking plugins. This is double speak. The predicate (3) contradicts the subject (1).
After it was pointed out how ludicrous this sound the caveat is added that this had to be done in the name of security, performance, and privacy. At what point did security and performance become more important than an open platform and why? Numerous addons exist solely to provide privacy by blocking fingerprinting, stopping redirects, providing control over cross-site requests (RequestPolicy Continued), super-cookie safeguards (BetterPrivacy), and these options are no longer available. How are these privacy enhancing features being added now that the option has been removed since the goal is privacy?
The whole thing is hard to take at face value when everything seems to be self-contradicting (sans performance).
Looks like we're both running out of arguments, since we're both essentially repeating ourselves, so I'm going to admit that I'm not going to manage to convince you :)
I am not trying to be hard on you. I know you likely don't have any real say in the process, but as one of the senior developers at the Mozilla Foundation you have an opportunity to call out shenanigans when you see it.
First we hear the changes are because adding new code took too long because the team didn't want to break addons, yet WebExtensions does just that in ways that are far worse than just temporarily breaking addons.
Second we hear it's about privacy. Yet WebExtensions breaks a large number of privacy plugins that won't be ported. There is also the Cliqz partnership and the October experiment. "In August 2016, Mozilla ... made a strategic investment in Cliqz. Cliqz plans to eventually monetize the software through a program known as Cliqz Offers, which will deliver sponsored offers to users based on their interests and browsing history."[1] "Mozilla is experimenting with including the Cliqz plug-in by default in its open source Firefox browser."[2] The reader can decide for themselves whether or not this is in the interest of privacy.
All that is left then to explain the changes are possibly security and speed. Security I am not so sure about as privacy and security tend to go hand in hand. It would be nice if you could respond to the earlier questions. FF57 is noticeably faster however so that is at least believable.
Whatever the real story is I do appreciate you engaging with us because you have no obligation to be here and you deserve respect for that.
The browser internals aren't userspace. Web Extensions are userspace. The browser internals are more like kernel space, which the Linux kernel breaks, all the time [1].
Indeed, if you provide an API you'd better commit to it. The difference is that before WebExtensions, there was no API. Everybody was just hooking into the internals.
That's not "do not break userspace" – which makes sense – that's "do not change anything, ever" – which is project suicide.
Now, with WebExtensions, there is a difference between the API and the internals, so we can commit to something. And, while there is a cost to this change, that's definitely a much, much, much better base for developers on both side of the API.
Mozilla pitched the Add-on SDK to developers as that kind of API. It was deprecated less than a year ago with no migration path and a half-baked replacement.
Many thanks for this informative post. I’ve read quite a few of the official Mozilla communications (announcements, blog posts, etc.) and so far, this has been the best explanation I’ve read for the impetus to move from XUL-based add-ons to WebExtensions. I write this from the perspective of a user who will lose almost half of their current extensions when they upgrade to 57.
Ignore the naysayers. Thank you for this. I've been dodging legacy add-ons already because they like to interfere with multi-process mode, killing Firefox's performance. Ubuntu's crappy legacy add-on actually nearly made me dismiss Firefox out of hand when I first gave it another try in February.
I'm sure you're absolutely correct in all the above, but I was happy with Firefox 56, then you broke mouse gestures so I stopped using Firefox and switched to Vivaldi. Whether alienating a huge number of formerly content users ends up increasing Firefox marketshare remains to be seen. But I won't be one of them.
As a Firefox dev (I'm still working at Mozilla, although not much on Firefox atm), I have seen many, many occurrences in which I couldn't optimize codepaths, or even in some case fix bugs, because the old extension mechanism made it impossible.
Consider the necessary steps:
1. realize that an internal API is broken;
2. come up with a new non-broken API;
3. port all the internal code using the non-broken API;
4. add a compatibility layer between the broken API and the non-broken API;
5. check all the existing add-ons to find out which ones use the broken API;
6. hope you didn't forget any add-on;
7. attempt to get in touch with all the add-on developers;
8. repeat 7. many, many times, until you are sure that the add-on developers that do not respond have simply abandoned their add-on;
9. negotiate a transition plan with the add-on developer with whom you have managed to get in touch;
10. land the patch that you have written now 3-4 months ago;
11. maintain both the broken API and the non-broken API (and their tests) for ~1 year, until you are reasonably sure that all add-on developers who intend to migrate have done so;
12. maintain (and test) a downgrade path for people who switch between versions of Firefox;
13. finally land your code;
14. realize that you still have accidentally broken some add-ons and people are (rightfully) unhappy because "Firefox broke my add-on";
15. it's 18 months since you wrote your 2-lines patch, you can finally get rid of the dead code and tests and move to something else.
This was one of the reasons for which the Chrome teams managed to be faster and more efficient than the Firefox teams (well, that and a bazillion dollars to hire way more people). The add-on architecture is the main reason for which projects such as multi-processes only landed ~8 years after we had working prototypes and some other performance projects never landed at all.
So, yes, removing the add-on architecture is definitely painful for a number of Firefox users, but I believe that we could not postpone it any further, even if it meant that some useful addons could not be ported immediately. Also, for what it's worth, we have postponed it by something like 7 years already :)