
Web Request and Declarative Net Request: Explaining the impact on Extensions - migueldemoura
https://blog.chromium.org/2019/06/web-request-and-declarative-net-request.html
======
danShumway
> _There’s been a lot of confusion and misconception around both the
> motivations and implications of this change_

Oh, heck off. This is Google's fallback whenever people call them out on
anything -- "you just don't understand the entire picture." It's just their
way of gaslighting critics.

People understand the change perfectly.

a) The privacy arguments are untrue because the Observational capabilities of
the WebRequest API aren't being deprecated. There are no privacy benefits
unless an extension voluntarily ignores that part of the API -- which no one
will do because, as Google says:

> _when extension developers are given the choice between capability and
> security, the vast majority of developers choose capability._

Google makes the argument that 42% of malicious extensions use this API. Well,
guess what they'll still be able to do under Manifest V3?

b) The vast majority of data suggests that ad blocker performance is not a
serious issue on the web right now. Ad blockers pretty much universally help
more than they hurt, so it's not clear why performance is such a big concern.

c) The declarative model is a less powerful alternative, which is the exact
reason why Google is deprecating the blocking capabilities in the first place
-- if they didn't force it, no one would switch (again, see their "why not
both?" argument).

So what don't consumers and developers understand about this? It's an
objectively less powerful API that does very little to improve privacy for the
sake of performance improvements that aren't necessary.

~~~
dfabulich
> _The Observational capabilities of the WebRequest API aren 't being
> deprecated_

They are being severely curtailed.

[https://twitter.com/justinschuh/status/1138889508512866304](https://twitter.com/justinschuh/status/1138889508512866304)

> _Yes, but network requests won 't be observable without permission on the
> relevant hosts. And host permissions are changing significantly in Mv3 (e.g.
> they can't be granted as blanket install time permissions)._

> _The underlying issue with blocking Web Request is that it is effectively a
> UXSS, which could not be bound by origin-based host permissions like the
> rest of the API can._

Google's privacy argument is correct.

~~~
danShumway
I strongly disagree. There is pretty much no chance that every ad blocker and
every other extension isn't going to widely request that new permission,
because without it you can't get the data you need to maintain adblock lists
or do more advanced detection (ie Privacy Badger).

On the current discussion thread, Chrome devs say:

> _The webRequest API 's ability to observe requests is foundational for
> extensions that modify their behavior based on the patterns they observe at
> runtime._

And in this article, they say:

> _when extension developers are given the choice between capability and
> security, the vast majority of developers choose capability._

So the argument that hiding observation requests behind a permission means
that they are "severely curtailed" is just plain wrong. Most extensions
(including ad blockers) will still request these permissions as wildcards
across all hosts, and most users will still grant them as wildcards across all
hosts.

Per-domain permissions for the blocking WebRequest API would be great (that's
a change I would broadly support), and if Chrome devs honestly believed that
per-domain permissions were an adequate user protection against malware,
they'd look for ways to implement per-domain permissions with the blocking
API, rather than deprecating the entire thing.

The Twitter post you link claims that's impossible, I don't see why. That
would be a good thing for someone from Chrome to explain in either the massive
developer thread that's been going on for months, or maybe in the article that
complains about how much "confusion" users and developers have, instead of
asking devs to blindly take their word.

~~~
dfabulich
You can't just request "that new permission" for all web sites ("hosts"), but
only for certain specific hosts, and only with the permission of each host.

> _The Twitter post you link claims that 's impossible, I don't see why._

Your original post claimed that people understand this issue "perfectly." It
turns out there was more to it than you thought.

~~~
danShumway
Check the spec[0] -- Google isn't getting rid of the ability for users to
grant full URL access for extensions, because of course they're not. That
permission is absolutely necessary for a wide variety of extensions, including
ad blockers. They're just implementing changes to how that permission is going
to be granted.

With respect, a lot of us have been following these changes for a long time.
We know about the permission stuff already. And to be clear, the permission
changes are good -- they're just not going to significantly change normal user
behavior in most cases.

> _It turns out there was more to it than you thought._

It turns out that a Google advocate on Twitter posted, "we can't do it the
other way" with no explanation of why it would be a problem. Again, this
exactly is how Google gaslights. They refuse to engage with the wider
developer community, ignore proposals and objections, and then when the wider
community finally gets mad enough to start complaining in mainstream channels,
they start posting on Twitter and blogs that "actually, it's complicated."

If developers don't have the full picture, _give it to them._ The Chrome team
can't complain that developers aren't informed about the technical limitations
they're facing when they're posting arcane updates to Twitter instead of the
official threads that everyone is following.

[0]:
[https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3Nzz...](https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3NzzhHzc-
qnk4w4PX-0XMw8/edit#heading=h.oenw1ekaaubz)

~~~
dfabulich
You say that they give "no explanation of why it would be a problem," in
reference to Justin Schuh's tweet:

> _The underlying issue with blocking Web Request is that it is effectively a
> UXSS, which could not be bound by origin-based host permissions like the
> rest of the API can._

The reasoning here should be obvious. _Observing_ web requests can be useful
with the cooperation/permission of the host you're observing, but it makes no
sense to have an ad blocker that requires permission from the ad-blocked
websites.

If you grant the core premise is that we don't want extensions to be allowed
to observe all web traffic, because this permission will be abused, then the
only logical conclusion is to use a rules engine. That's the only way to
design an ad blocker that doesn't/can't observe all web traffic.

~~~
danShumway
> _it makes no sense to have an ad blocker that requires permission from the
> ad-blocked websites._

I don't follow what you mean by this.

Under the current V3 specification, as currently proposed, Privacy Badger
(assuming they stick it out and keep maintaining a Chrome version) will need
to be granted observation permissions on every single website, adblocked or
not. The way Privacy Badger works is by determining at runtime based on
request observation which domains it should blacklist. This is not going to be
a fringe usecase.

Are you misunderstanding how the proposed permissions system works by some
chance, or am I misunderstanding your argument? It's not the website you
request permissions from -- it's the _user_ you request permissions from for
domains/domain patterns.

Why doesn't it make sense for a user to grant an ad blocker permission to
operate on domains where they want it to block ads?

------
SquareWheel
They've addressed a number of complaints about limitations of the new API:

> _Since the original announcement of the Declarative Net Request API, we have
> added significant functionality to the API as a result of these discussions.
> The Declarative Net Request API now allows for the registration and removal
> of dynamic rules - specified at runtime rather than statically in the
> manifest. We’ve also added the capability to remove common tracking headers,
> such as Referer, Cookie, and Set-Cookie._

> _We are actively exploring other ways to expand this API, including adding
> methods to get feedback about matched rules, and support for richer
> redirects leveraging URL manipulation and regular expressions. Additionally,
> we are currently planning to change the rule limit from maximum of 30k rules
> per extension to a global maximum of 150k rules._

They posted a simpler post to the Security blog also addressing these issues.

[https://security.googleblog.com/2019/06/improving-
security-a...](https://security.googleblog.com/2019/06/improving-security-and-
privacy-for.html)

They lead with the point that most-seemed to get lost in translation, even on
technical sites like HN.

> _No, Chrome isn’t killing ad blockers_

~~~
hlieberman
Killing? No. Handicapping? Yes. And why are we surprised? It’s in their best
interest to do so; expecting a for-profit company to go against its own
monetary best interests forever is naïve, at best.

~~~
dfabulich
The rule limits Google's proposing are more generous than the limits Apple
puts on content blockers. Apple allows only 50K rules. And it turns out 50K
rules is actually fine!

Declarative rules engines are better and preserve privacy better. This is
Google following Apple's lead and doing the right thing.

~~~
buckminster
It does nothing for privacy because extensions will still be able to see the
requests, they just won't be able to block them.

~~~
dfabulich
No, manifest v3 extensions will only be able to observe requests for specific
hosts, with permission from each host.

~~~
buckminster
This isn't mentioned in the public version of the v3 spec (perhaps I'm looking
in the wrong place) or in today's blog post. Do you have a reference?

~~~
magicalist
One of the other comments has a link to this tweet about it:

> _Yes, but network requests won 't be observable without permission on the
> relevant hosts. And host permissions are changing significantly in Mv3 (e.g.
> they can't be granted as blanket install time permissions)._

[https://twitter.com/justinschuh/status/1138889508512866304](https://twitter.com/justinschuh/status/1138889508512866304)

there's also some stuff in the design doc:
[https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3Nzz...](https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3NzzhHzc-
qnk4w4PX-0XMw8/edit#heading=h.oenw1ekaaubz)

Which explains the specific hosts part, but only makes vague mention of how
extensions that users _do_ want to run on every site will get permission to do
so.

~~~
buckminster
Thank you. If Google were trying to be obfuscatory they could hardly do a
better job.

------
snowwolf
This seems like massive spin. Their primary argument doesn’t wash. As far as I
understand it the web request API will still exist and still allow extension
developers to view all request data. They just won’t be able to block the
request and change it (I.e block content from loading)

~~~
SquareWheel
As much as I've defended this change, I agree with you. The only justification
I could see is that it might require an extra permission in an extension's
manifest, but even that I'm not sure of.

If this is part of a longer-term plan to deprecate webRequest and completely
remove that ability, then they should say so now.

~~~
SquareWheel
This might create a confusing link loop, but a dev commented to explain this
rationale now.

[https://twitter.com/justinschuh/status/1138889508512866304](https://twitter.com/justinschuh/status/1138889508512866304)

~~~
snowwolf
So lets take an alternative look at this. The justification is that this
improves privacy, security and performance. For who? I use 2 extensions, an Ad
Blocker (uBlock Origin) and Password Manager.

uBlock Origin has over 10M installs and the author has stated publicly that
they cannot effectively do what they do if this change goes ahead. I can
guarantee that uBlock Origin does more to protect my privacy, security and
performance than this change. How many people have been affected by malicious
extensions? I seriously doubt it's greater than 10M. So to improve the
privacy, security and performance for a few, they are going to make it worse
for the many.

Surely there are better solutions to tackle this problem? Maintain the API but
add in additional checks/vetting for any extensions that use it.

~~~
dfabulich
[https://www.vice.com/en_us/article/zmdxxj/the-hack-
millions-...](https://www.vice.com/en_us/article/zmdxxj/the-hack-millions-of-
people-are-installing-themselves)

> _Last year, hackers phished Chris Pederick, who runs Chrome Web Developer,
> an extension with over one million users. Once they had grabbed Pederick’s
> developer login details, they swapped his extension on the Chrome Web Store
> with their own malicious version, designed to inject adverts into users’
> browsers. Then in September, someone loaded the popular Chrome extension for
> file sharing service MEGA with code that could steal login details for
> Amazon, GitHub, Microsoft, and Google accounts, as well as pinch
> cryptocurrency keys. And a malicious browser extension appears to be behind
> a recent dump of private Facebook messages. Once hackers gain control over
> the extension, through hacking, coercion, or otherwise, they can replace
> code as they see fit._

There's a lot more where that came from.

Aside from hacking the extensions, a couple of years ago there were severe
issues with people buying control of popular extensions and then converting
them into malware.

~~~
snowwolf
So there's a few things to unpack here.

> with their own malicious version, designed to inject adverts into users’
> browsers

Your very first example wouldn't be prevented by these changes.

> 42% of malicious extensions use the Web Request API

So 58% don't event need the Web Request API to do something malicious. So
these changes don't really improve safety at all.

> hacking the extensions, buying control of popular extensions

Both these scenarios need to be addressed at a different level. Things like
enforcing 2-factor for high value extension authors (> 100K installs or
something). Also remember Google operate a walled garden here. No extension
can be published on the store without being vetted. They should be identifying
high profile/value extensions and subjecting them to additional checks.

For me, and I'm sure many people, my biggest threat vector is ads. They
threaten my security (malicious/malware laden ads), my privacy (tracking), and
performance (slow, bloated ads). My ad blocker protects me every day from this
threat vector. Google are now definitively weakening mine and a lot of other
peoples protection in order to improve the security of the few. To me this
does not seem like a reasonable compromise, especially when there are
alternative ways to address their concerns.

And I'm sorry, but I really don't trust the motivations of a company that
makes billions of dollars a year off online advertising, that has publicly
stated that Ad Blocking is a threat to their business model, to then be making
a change that just so happens to cripple Ad Blockers.

------
ChrisCinelli
> In addition to these safety concerns, there are also significant performance
> costs. In most cases, these costs are not from the evaluation of the
> extension script processing events, but rather from everything else
> coordinating the script. That overall performance impact can be very large,
> even for an extension written as performantly as possible where the
> JavaScript execution time is negligible.

Cannot they just instrument the code that intercept the requests and tell the
user when performances are affected? Something like "It seems that extension
xyz is slowing down Chrome. Do you want to disable it? Yes No [] Do not ask
anymore"

------
dang
Related:
[https://news.ycombinator.com/item?id=20167246](https://news.ycombinator.com/item?id=20167246)

------
drenvuk
No. It is incredibly hard to be cordial when you're trying to ruin the web and
say that you're actually helping. I am livid. Because of content blockers the
speed of web browsing has increased substantially. Because of content blockers
users' security is vastly improved. ublock Origin has been doing the right
thing and now you're going to make it harder. With the limits imposed by the
declarative net requests at 150k developers and extensions that are
safeguarding us are being hamstrung.

In addition, because of the Chrome extension team's obsession with using js
workers instead of persistent processes we're going to have to deal with
finding work-arounds for holding important and unencrypted things in memory.
Someone brought up the issue that some extensions will have to create a newly
pinned tab to the top left of the browser because of these inane bullshit
rules removing the background page. Imagine a stack of pinned tabs just so
your extensions can do their jobs. Does that sound dumb enough yet? Sometimes
it's necessary to hold a lot of information in memory because the user
actually wants the extension to do a certain task in a reasonable amount of
time. They are making our jobs harder because they're too stubborn to
understand that extension developers are the ones who create the magic. This
is central governance at its finest.

These rules are stupid as hell.

~~~
dfabulich
The rule limits Google's proposing are more generous than the limits Apple
puts on content blockers. Apple allows only 50K rules. And it turns out 50K
rules is actually fine!

Declarative rules engines are better and preserve privacy better. This is
Google following Apple's lead and doing the right thing.

~~~
mixedCase
> The rule limits Google's proposing are more generous than the limits Apple
> puts on content blockers

And Firefox and most Chromium forks have no limit. Does that mean they have
_infinite_ generosity?

Seriously, how is Apple being bad at something an argument for lowering other
options to the same level? Right now I have ~200K filters enabled on this
machine. EasyList alone is sitting at 90K according to uBlock Origin.

