
Firefox: The Effect of Ad Blocking on User Engagement with the Web [pdf] - __ka
https://research.mozilla.org/files/2018/04/The-Effect-of-Ad-Blocking-on-User-Engagement-with-the-Web.pdf
======
gwern
> In the group that installed an ad blocker, we find significant increases in
> both active time spent in the browser (+28% over control) and the number of
> pages viewed (+15% over control), while seeing no change in the number of
> searches.

I'm struck by the almost numerical identity of their adblock correlate on page
views with the long-term causaleffect on total site traffic I estimated in my
banner ad A/B test ( [https://www.gwern.net/Ads](https://www.gwern.net/Ads) )
of (15% vs my 14%, 13-16% CI). The unit of analysis isn't the same but they're
arguably equivalent.

This is also interestingly parallel to Pandora's giant A/B test: Huang et al
2018's
[https://davidreiley.com/papers/PandoraListenerDemandCurve.pd...](https://davidreiley.com/papers/PandoraListenerDemandCurve.pdf)
"Measuring Consumer Sensitivity to Audio Advertising: A Field Experiment on
Pandora Internet Radio".

~~~
thomastjeffery
It seems to me like this study is looking at the correlation of heavy usage
and ad blocker usage, and jumping to the conclusion that blocking ads is a
_cause_ for using the web more, and not the other way around.

~~~
gwern
It is correlational, yes, but it's consistent with what we expect (surely no
one expects ads to have zero or even positive effects on Internet usage?), the
direction & size of the effect are consistent with the 2 most relevant
randomized experiments (linked in my comment), and they have longitudinal data
which lets them control for a decent number of baseline characteristics like,
to quote from the propensity scoring section:

\- Active hours: Total time the user spent interacting with the browser \-
Session length: Total browser session time \- Number of subsessions: Total
Telemetry reports \- Total tab open events: Number of tabs opened \- Total URI
count: Total number of pages loaded \- Unique URI count: Number of unique
websites (TLD+1) visited \- Address bar searches: Number of searches initiated
from the address bar \- Search bar searches: Number of searches initiated from
the dedicated search bar \- Total Yahoo searches: Number of searches made
using the Yahoo search engine \- Total Google searches: Number of searches
made using the Google search engine \- Total searches: Total number of
searches overall initiated from one of the Firefox search bars \- Number of
bookmarks: Total bookmarks added \- Length of history: Total number of pages
visited

So everyone with an adblocker had normal activity levels at the start
(compared to the non-adblock-user they are 'matched' with by the propensity
scoring), but after installation, then the adblocker people saw large
increases in activity that their non-adblock counterparts, who were web
browsing just as much before hand, did not experience.

This largely rules out the reverse causation of 'browsing -> adblock
installation'. It could still be driven by people forecasting that they will
be doing a lot of web browsing in the future and installing adblock _before_
they begin doing all the extra browsing, but this feels a lot less plausible
to me.

------
nattaylor
If you accept that advertising is necessary to some business models, I hope to
see publishers move ads back to the server because I think eliminating all the
code from the client would drastically improve user experience.

This would make them tougher to block and hinder users ability to control
privacy, but it would also restrict tracking to that specific site.

~~~
dmortin
> If you accept that advertising is necessary to some business models, I hope
> to see publishers move ads back to the server

The problem is the big guys can do this easily, but e.g. someone running a
blog cannot easily serve ads from the server.

Ad networks are about ad auction and showing the highest paying available ads
for a site. Client side ad solutions makes this easy for the little guy,
because ad selection is done on the client by the ad network.

The big players (facebook, etc.) can easily alter their methods or require
subscription if necessary. It's not so easy for the little guy, so a change of
the current model without offering a similarly effective alternative is going
to kill many little sites, blogs, forums, while the big players will keep
thriving regardless.

~~~
Kalium
If the small players can only continue to exist by serving malvertising, some
might opine that they might not have a right to continue to exist with that
business model.

~~~
manigandham
Why must non-server advertising be the same as malvertising? The integration
type is different from the ads that are possible. Server-side integration
doesn't affect what ads are available.

~~~
crtasm
Don't all the ad networks use JavaScript? Has any network never let malicious
code from advertisers slip through? (I guess Google? But I'd prefer not to
have even more people embedding their js everywhere I browse)

~~~
manigandham
Yes, there are networks that have always remained clean. It's a business
problem more than technical.

But the topic was smaller sites running 3rd party networks which necessarily
only integrate via client-side JS. That has nothing to do with what kind of
ads are being run. It's the same exact ad calls to the ad exchanges, just done
via HTTP from the server rather than JS in the browser.

~~~
michaelt
You can't virus-scan an e-mail that doesn't pass through your server. How
would you inspect javascript that doesn't pass through your server?

Or do you have to just get a bit of paper from the ad network saying they
won't make any mistakes, then take their word for it without any way of
auditing their claim?

~~~
manigandham
That's not how it works. The payload is dynamic javascript. Just because you
make some calls on the server-side doesn't mean you can inspect the payload
with any real reliability, and if it was that easy then it would already be
done by the ad exchanges who send the payloads in the first place.

There is an entire inner industry around security and malware for programmatic
ads, with a few companies that are pretty good at it, but it is not something
that can just run as a simple server for any small publisher to deploy.

------
gorhill
> adblocker | either AdBlock Plus (ABP) or uBlock Origin, the two most popular
> ad-blocking Firefox add-ons

Adblock Plus ("ABP") does not block a lot of ads by default, as ABP's
"Acceptable ads" is opt-out -- i.e. it is enabled by default.

I wonder if this was taken into account, and if not surely this affected the
result of measuring "the effect of ad blocking on user engagement with the
web", as you can't categorize ABP's "acceptable ads" as "ad blocking".

~~~
gwern
If many of the 'adblock users' are still being exposed to occasional ads which
are 'acceptable', then presumably the true effect of ads is even worse than
estimated.

------
amelius
> We conclude that ad blocking has a positive impact on user engagement with
> the Web, suggesting that any costs of using ad blockers to users’ browsing
> experience are largely drowned out by the utility that they offer.

Facebook: "This user isn't clicking any ads; let's keep them engaged more with
pointless content until they do."

~~~
rfugger
Facebook: "This user isn't clicking any ads, and will probably never click any
ads. That's fine; their presence and contribution to the platform helps us
attract and retain other users, some of whom will click ads, and so is
inherently valuable to us."

------
itronitron
Companies like WP, NYT, and Ars should conduct similar analyses focused on
their own sites to see if reducing the number of ads (or their size in pixels
or bytes) results in more engagement, and ultimately a larger number of ad
views.

~~~
matt4077
There is absolutely no doubt that all those sites, and many others, are
continuously running such experiments.

There are also new financing methods, paywall implementations, prices,
cooperations, etc. being tested all the time, with some successes and many
failures.

While people here and in general often seem to consider publishers to be
rather stupid, I think there's enough evidence at this point to conclude that
it has become a particularly difficult business, what with almost half of
revenue almost instantly disappearing when classifieds move to the net, and
people generally considering to appear out of thin air because the costs of
copying are zero.

~~~
gwern
> There is absolutely no doubt that all those sites, and many others, are
> continuously running such experiments.

They are running randomized experiments, but they are not necessarily running
the _right_ experiments. If the bottom line is revenue, then attrition may not
be directly measured; if they are testing on per page-view or per session
instead of bucketing users _permanently_ , then they can miss effects too; and
if they are testing by bucketing users simultaneously, they can miss non-
additive site-wide effects. If you look at my comment, I link two publicly-
documented randomized experiments, and both set up/analyze their tests in an
unusual way.

After I published my results, I heard about some parallel results, but not
that many. I heard whispers that Google might (or might not) have internally
run equivalent tests; I heard from Chris Stucchio that an unspecified company
he worked for had run an A/B test with similar results; I heard from a mobile
developer that they had run a similar experiment but found no effect of ads;
Pandora subsequently published their A/B test; and Mozilla has published this.
I also read through most of the 'advertising avoidance' literature in academia
going back 30 years, and found nothing relevant. And that's it.

So, if everyone is always continuously running obviously relevant
experiments... where are they?

~~~
manigandham
Do you mean where are the published results? That's not something publishers
do, so most of the intel will be internal. It's also unlikely that are running
experiments constantly as the basic stuff has been figured out.

Also most of the innovation happens with adtech vendors who try out new ways
to get attention and payments. TrueX and Sourcepoint have been working on some
new stuff, along with the various formats that keep being invented.

~~~
gwern
> Do you mean where are the published results? That's not something publishers
> do

Publishers like Google, Microsoft, and Yahoo do publish many research papers
going in depth on A/B tests - such as the effects of page load time - and I've
benefited from reading them over the years. So yes, it is something they do.

~~~
manigandham
None of those companies are considered publishers, and that seems to be more
generic UX research rather than ad focused.

For example, Washington Post runs experiments specifically with ads as a
digital media/news publisher, but you won't see that research published
anywhere.

~~~
gwern
They publish a ton of stuff and are some of the biggest media properties on
the Internet. Other publishers like the NYT or the BBC also do writeups of A/B
tests and lessons, but they're such small fry I didn't want to mention them.
And if 'page loads harm user retention' is worth publishing, 'ads harm user
retention' definitely is and they are exactly the companies best placed to
find that out.

~~~
manigandham
We're just going in circles here. You asked about relevant experiments as a
reply to comments about media publishers and ads. I'm telling you why you
don't see that research as official papers.

Google, Microsoft (Bing), and Yahoo are not considered media publishers
because they are ad networks first.

If you just dismiss actual media publishers like NYT and BBC as small fry
(when they are some of the biggest) then I'm not sure what you were expecting
as an answer.

Even if you don't discount their sizes, they do not reveal their research
because there's no time or incentive to reveal things like yield management,
which is complex, audience specific, and contains competitive intel. That's
why it can't be officially found, but you can get details if you talk to their
dev teams.

~~~
gwern
> I'm telling you why you don't see that research as official papers.

Which is a strange thing to claim since what you claim never happens, happens
all the time with some of the largest media publishers in the world. Think of
how many pages Google serves via AMP, or Yahoo's front page, or Google News,
quite aside from their companies literally being based on advertising. If they
are not media publishers who are doing experiments of the relevant sort, no
one is and you are blatantly no-true-Scotsmanning.

There are large entities, better positioned than anyone else in the world to
know, who run experiments constantly, and publish the results constantly.

So.

Where are they?

> they do not reveal their research because there's no time or incentive to
> reveal things like yield management, which is complex, audience specific,
> and contains competitive intel.

Congratulations, you have disproved the existence of hundreds or thousands of
papers and blog posts. I must be hallucinating them all.

> That's why it can't be officially found, but you can get details if you talk
> to their dev teams.

Really? Then why, when _I_ talk to them, I never hear any specifics, I always
get told either they don't know or no one's checked, and only once in a blue
moon do I hear something relevant (always without details)? Since you
apparently talk to so many dev teams and they tell you all about the effects
of ads on user retention and total traffic from the many experiments they run
but never ever publish on, perhaps you could summarize them and provide the
details for the rest of us.

~~~
manigandham
>> If they are not media publishers... no one is

No, Google, Microsoft, Yahoo are not considered media publishers by any actual
media publisher; because they are ad networks that do some news aggregation at
best and do not produce original content. There absolutely is a definition
that you are refusing to recognize for some reason.

>> There are large entities..., who run experiments constantly, and publish
the results constantly. So. Where are they?

Again, what are you referring to here? The big companies are not publishers
and what they do is closer to Amazon.com testing page speed rather than NYT
testing ad load. Actual media publishers do _not_ publish results constantly,
if ever.

>> Congratulations, you have disproved the existence of hundreds or thousands
of papers and blog posts. I must be hallucinating them all.

What? How are you asking about the lack of research and then saying there's
research everywhere? Again, what are you talking about here? What media
publisher* is printing heavy insights into their ad yield management?

>> ...when I talk to them, I never hear any specifics, I always get told
either they don't know or no one's checked...

Because it's internal intel. Do you know anyone on the revenue team? They
might be more open to details but won't know the technical background. You can
look at trade publications like Digiday, Adweek, Mediapost, Adexchanger and
others to find details and some rare whitepapers.

>> perhaps you could summarize them and provide the details for the rest of
us.

You need to have specific questions, but here are some details:

Top sites are focused on revenue over UX. Sites have found equilibrium between
ad load and retention with maximum time spent. Users rarely stop visiting
sites, they just come back with adblockers instead, which has led to
strategies like free articles per month and increased direct-sold campaigns
and native ads from brand studios (for pubs that can afford a team). WSJ and
other pubs have stopped the google-search free entry workaround as well.

UX changes include never-ending feeds and auto-changing articles to minimize
bounces, along with less ad slots but more ad refreshes to increase total
impressions. Tech-forward pubs like WaPo have built their own adtech stacks to
show users different formats and ad frequencies depending on how they respond.
Subscribers get less ads until churn is minimized. No ads for subs just leaves
money on the table and even FT.com runs plenty of campaigns. Video has also
invaded everything with 5x higher prices than banners.

Overall pubs have managed to squeeze out the same number of impressions from
just the non-adblocked market, while optimizing paid conversions for
adblockers and capitalizing on their uninterrupted attention to show more
high-touch direct campaigns. Readership is up all around and both segments
make more money than before.

------
asdf1234tx
I don't object to advertising. Highly optimized images can load quickly. I
object like a banshee to the insane amounts of js that is raining down on my
connection. The bad things that people are doing with that js, from crooked,
inept, unprincipled, to down right criminal, well that's just the icing on the
cake...

~~~
fliesblackflags
This is why I block nearly all adverts. I wouldn't object to ads that don't
track me, or autoplay sound/video, or obscure the content, or waste my
bandwidth.

------
abdullahkhalids
Mozilla should work on a micro-payment system that is inbuilt into the
browser. Ultimately that is the only solution to a web that values user
privacy and prevents the psychological toll of ads.

1\. Obviously, the micro-payment system should not be selling my usage data to
anyone. Perhaps some cryptographic techniques can be used to increase trust in
the service.

2\. The price for each page should be low - a cent or less for most articles,
less than 5c for more serious journalism. Maybe 2 cents/day for social media
websites. I would guess for most people in the west 30 dollars/month would be
the max they are willing to spend and for other countries, ~5 dollars/month.

3\. In return for paying for a page there should be 0 off-site advertisements
on the page.

4\. People are going to say - such a business model will not work because too
many decisions. Look at places like South Asia where hundreds of millions of
people use prepaid phone services where they get charged for every call/sms at
very low rates. And they would not have it any other way. I am sure consumers
in the West can be trained similarly.

~~~
Animats
_micro-payment system_

This comes up regularly. All the enthusiasm for micropayments comes from
people who want to collect micropayments. Not from the users who would pay
them.

~~~
privong
> This comes up regularly. All the enthusiasm for micropayments comes from
> people who want to collect micropayments. Not from the users who would pay
> them.

All the enthusiasm for ads only comes from the people who want to show/sell
them. Not from the users who would be subjected to viewing them.

I think it's still a worthwhile discussion. Presumably the micropayments have
fewer (or at least different) unintended consequences. And since in both cases
(ads, micropayments) the side collecting the money is the one pushing them. So
discounting micropayments solely on this basis isn't really a good way to
discriminate.

~~~
jasode
_> All the enthusiasm for ads only comes from the people who want to show/sell
them. Not from the users who would be subjected to viewing them._

Yes, it does seem that way because nobody actually says, _" Please please give
ads because I love them!"_

Instead, the correct way to frame what users really want is to separate _"
stated preferences"_ from _revealed preferences_.[1]

The _stated_ preference, of course, is for users to complain about ads and
loathe them. Another source of stated preferences is marketing surveys and
questionnaires.

But the _revealed_ preference is different because it's the observation of
_what really people do_ instead of what they say they'll do. Most would rather
watch youtube for free rather than pay $9.99 for YoutubeRed. Same for Spotify
free tier with ads rather than $120 a year Spotify Premium. Most would rather
read the 5 free articles with ads from New York Times rather than pay a
subscription.

Another interesting example was Amazon's Kindle e-reader. They had a version
with ads (what they call "with Special Offers") for $114 ... and one without
ads for $139. _More buyers chose the device with ads._ [2]

Did Kindle users want ads?!? Of course not. But they valued the $25 savings
more than the the ad-free experience. Basically, they "do want ads" \-- in a
roundabout way.

The micropayments enthusiasm doesn't really change the consumers' _priority_
of "free with ads" over "pay for zero ads". People really really like the
economics of "free". Unless micropayments is $0.00 -- which would be an
oxymoron -- it will always have that economic handicap to widespread adoption.

 _> Presumably the micropayments have fewer (or at least different) unintended
consequences._

One unintended consequence is that authors and creators end up with _less
revenue_ if they restrict their output to only subscribers making
micropayments.

[1] [https://www.beyondcostplus.com/blog/stated-vs-revealed-
prefe...](https://www.beyondcostplus.com/blog/stated-vs-revealed-preferences-
pricing)

[2]
[https://www.pcmag.com/article2/0,2817,2385398,00.asp](https://www.pcmag.com/article2/0,2817,2385398,00.asp)

~~~
Nursie
These things are very much aggregates and averages.

A significant number of people do subscribe to spotify, and some is buy the
ad-free kindles.

~~~
Nursie
some _of us_ buy the ad-free...

sigh, phone keyboards and a lack of proof-reading.

------
StavrosK
I don't see anyone here pointing out an alternative explanation: It could be
simply that more active users are more likely to install ad blockers.

~~~
faitswulff
They seem to cover that:

> We then compare test group Web usage after installing the ad blocker to
> control group usage over a comparable period, _controlling for baseline
> usage levels._

Emphasis mine.

~~~
StavrosK
Interesting, thanks.

------
jchw
Interesting, but I do want to know if a randomized study produces the same
results. Users who use adblockers are much more likely to be power users.

Mozilla does control for that by separating any add-on from adblockers,
however. So maybe the finding is generally true. This is not very surprising
if true, given the difference in experience across most websites when
adblocking is on.

~~~
jonathansampson
Brave is extending ad-blocking beyond the power-user domain, and into the
general user base. We recently passed 4 million monthly active users, and 10
million Android downloads. With our built-in ad-blocking, the practice is
becoming more ubiquitous among non-technical users. In fact, prior to Brave,
ad-blockers had appeared on more than 600M devices. There's a clear and strong
trend towards more and more blocking/privacy-by-default on the web.

------
downrightmike
More people are voting with their wallets that online advertising is executed
poorly and an annoyance to a threat vector for malware. More people need to
stop the advertising from even coming down their wire. I'm happy with Pi-hole:
[https://pi-hole.net/](https://pi-hole.net/)

~~~
jonathansampson
This is what we do with Brave ([https://brave.com](https://brave.com)). We
stop it from even touching the network; after all, much of the ads and
trackers we download aren't benign. Many are aggressively trying to monitor
our online activity at best, or infect us with malware at worst. Brave blocks
these requests from taking place.

We offer the user an option instead to support online properties (websites,
YouTube channels, Twitch streams, etc.) with pro-rata contributions of
Attention Token (BAT). Brave stakes users with free BAT grants each month to
prime the system.

Understanding that the web runs on advertisements, we're also working on an
alternative advertising model wherein the user gives explicit consent to
participate, matching happens on-device via machine learning, and revenues are
shared with the end-user (as much as 70%).

------
jamesb93
I think I would just rather have advertisers barrage me and I'll continue to
block them.

~~~
jonathansampson
Something has to be done about the sustainability problem though, otherwise
you'll have fewer and fewer sites to visit in the future.

Brave is working on an alternative digital advertising platform based on
explicit user buy-in. If the user does not consent, the user does not see
these third party ads and trackers.

As part of an incentive to get user consent, Brave's model pays the user for
their attention (as much as 70% of ad revenue). After all, attention is a
limited commodity, and you should get paid for yours.

In the Brave model, ads are matched locally, on the user's device via machine
learning. This reduces complexity, thus reducing opportunity for fraud and
abuse. As such, we expect to see more revenue for the content creators
themselves (in addition to the users).

------
RestlessMind
> We find that installing ad blocking extensions substantially increases both
> active time spent in the browser and the number of pages viewed. This
> empirical evidence supports the position of ad blocking supporters and
> refutes the claim that ad blocking will diminish user engagement with the
> Web.

This gives me echos of "thrift paradox" \- kind of behavior which is great for
you as long as most of the other folks behave otherwise. If a majority of
users continue viewing ads and thus provide monetary returns for content
creators, then the minority users of ad blockers get a better experience and
hence they are more engaged with the web.

On the other hand, if everyone starts using ad blockers, then how are content
creators going to survive? Probably, a bunch of copycats and click-baiters
deserve to perish. But this can also have unintended consequences of niche
blogs vanishing and/or paywalls going up and/or survival of only those with
some benefactor (Washington Post) or cross-subsidizing business empire
(Murdoch). Do we want that kind of future?

~~~
zzzcpan
Small independent creators cannot rely on ads for income already and fund some
other way. And so almost all of the websites, except for some thousands of top
ones. But those can probably find a way to survive another way.

Blocking ads may not actually affect anyone beyond online advertising
corporations.

~~~
jonathansampson
Content creators have been hurting for some time, and it doesn't just impact
those who have seen diminished returns. Many have been unable to monetize
their content at all (which we see quite often on YouTube, etc).

We built Brave Payments (now Brave Rewards) into the Brave browser to _re-
monetize_ the web. Each month we give out BAT grants to our desktop users,
which fund their favorite online properties (sites, streams, channels, etc).

Brave is also standing up an alternative ad-model which requires explicit user
consent. If a user participates, ad-matching happens locally, on the user's
device. This protects privacy, cuts out middle-men, reduces fraud, and more.
Furthermore, the users are compensated for their attention (up to 70% of ad
revenue), which adds to the sustainability of the web.

------
TekMol
I smell a Correlation-Is-not-Causation fallacy here.

~~~
jonathansampson
Could you elaborate as to why you believe there is no causation here? Those
who use ad-blockers do so for the explicit reason of making the web more
tolerable; a more tolerable web is therefore more desirable to browse.

~~~
thomastjeffery
This is proving just as clearly that the more you use a web browser, the more
likely it is that you will block ads.

The only _evidence_ here is correlation.

~~~
gwern
> This is proving just as clearly that the more you use a web browser, the
> more likely it is that you will block ads.

No. That's the point of using propensity matching in a longitudinal dataset.
What this 'proves' is that 'when two new Firefox users who use a web browser
an equal amount (as defined by a number of variables like time-in-browser or
total-TLDs or total-URLs) are compared, one of whom installs adblock and one
who does not, afterwards, the adblock user subsequently increases their web
browser use compared to the non-adblock user'. As they put it:

> Restricting to new users who installed Firefox during a specific time
> period, we select a test group of users who installed an ad blocker, and a
> control group that did not. The selection is performed using a matching
> technique applied to baseline user features, which allows us to infer a
> degree of causality from the observed effects. We then compare test group
> Web usage after installing the ad blocker to control group usage over a
> comparable period, controlling for baseline usage levels. This is sometimes
> referred to as analysis of covariance, and is closely related to difference-
> in-difference analysis...This is achieved using matching. Each test group
> member is paired with a similar member of the control group, where
> similarity is determined based on features observed during the baseline
> period. This ensures that the groups start off “more or less the same”, e.g.
> that one group is not biased towards more engaged users...Taking into
> account the matching and adjustments made to ensure comparability between
> the groups, we find that installing an ad blocker does in fact have a strong
> effect on average Web engagement. Users who installed an ad blocker were
> active in the browser for around 28% more time on average, and loaded 15%
> more pages (URIs), controlling for baseline activity.

(Which of course still doesn't prove causation, because there is 1 remaining
possibility: some third factor caused both the adblock installation and then
later in time the increased web browsing. But it doesn't prove what you say it
does.)

------
tgsovlerkhgsel
TL;DR: _We find that installing ad blocking extensions substantially increases
both active time spent in the browser and the number of pages viewed._

(also known as "water is wet")

~~~
bordercases
Water is wet, but to what degree?

