
Firefox 59 to strip path information from referrer values for 3rd parties - jhatax
https://blog.mozilla.org/security/2018/01/31/preventing-data-leaks-by-stripping-path-information-in-http-referrers/
======
sologoub
As someone that digs in this data for a living, personally strongly believe
this should be on by default for all browsing. The example they give is an
insanely bad design for healthcare.gov and I would absolutely not want to find
that type of data in my analytics telemetry.

It is very useful to know where your traffic is coming from, but that’s
usually viewed at a higher level than the querystring params being shown. In
some cases, this may restrict you from knowing which article the person was
reading on the given site before clicking through to yours, but if that’s so
important, there are other ways to instrument source tracking.

~~~
godelski
As someone that doesn't dig into this data for a living, why would you not
want this on non-private sessions (first question I had when reading this)?
I'm curious why Mozilla wouldn't want to protect this data leaking in even a
normal session.

~~~
groovecoder
Disclaimer: I'm the Firefox engineer who wrote the patch and the post.

We did a user research study measuring website breakage under various privacy
protections:

[https://blog.mozilla.org/data/2018/01/26/improving-
privacy-w...](https://blog.mozilla.org/data/2018/01/26/improving-privacy-
without-breaking-the-web/)

tl;dr - strict-origin-when-cross-origin was one of the protections with the
lowest amount of breakage. Entering Private Browsing is a clear, strong signal
that the user wants more privacy, so we started by implementing this
protection in Private Browsing.

However, note that some advertisers demand that AdTech vendors must not serve
their ads on certain kinds of pages. (e.g.,
[https://support.google.com/adsense/answer/1348688?hl=en&topi...](https://support.google.com/adsense/answer/1348688?hl=en&topic=1271507&rd=1))
Many of those agreements require full referrers to be able to audit the ad
inventory.

So there are some concerns and trade-offs to make in this space.

~~~
sleavey
Why don't websites just remove ad code from pages they don't want ads to
appear on, instead of telling the ad server to not display it on such pages by
using the referrer information? Seems like a convoluted way to solve the
problem.

~~~
fiddlerwoaroof
No, the advertiser doesn’t want their ad to appear on a certain publisher’s
web page (e.g. banks often can get in trouble if their ads are associated with
certain kinds of content).

~~~
sleavey
Ah ok. But why then doesn't the ad's embed code just contain the URL of the
page it's on in the query string / POST data? The (website that shows the
ad's) server knows what page it is providing.

~~~
andrewaylett
I suspect it does? But that's spoofable by the ad aggregator in a way that the
Referer header isn't. So much of ad design is (for better or occasionally for
worse) defending against bad actors.

------
weinzierl
> To help prevent third party data leakage _while browsing privately_ ,
> Firefox Private Browsing Mode will remove path information from referrers
> sent to third parties starting in Firefox 59.

Emphasis mine.

It only does it in private mode. I experimented with the referrer options
mentioned in the article with mixed success. Not sending the referer header
breaks some sites and often in a non-obvious way.

EDIT: referrer header -> referer header

~~~
franciscop
Just curious, did it break any popular website?

~~~
Feniks
It will break forums and imageboards in my experience. Perhaps due to CAPTCHA.

~~~
Zancarius
I think it's due to dubious "security" implementations, including at least one
that was (?) present in a rather notorious PHP message board software package
[1]. I'd be surprised if this survived all these years later, but then I've
also seen some custom session handling code in my travels that did something
similar even as recently as 2-3 years ago.

Some image boards do it to prevent casual hot-linking, as my sibling poster
notes.

[1]
[https://tracker.phpbb.com/browse/PHPBB3-8396](https://tracker.phpbb.com/browse/PHPBB3-8396)

------
jacquesm
This one of the reasons I tend to flip completely whenever I see healthcare
providers and their suppliers run google analytics tags _inside_ their logged
in areas (yes, this really happens). Besides the questionable value of having
such tracking inside the logged in areas (it's healthcare, they are not going
to worry about their conversion rates) such information should simply never
leave the premises. Better still if they didn't do this in private mode but
always. Private mode is still associated with doing something sneaky, rather
than that it should be the default.

Happy to see FF do the right thing here and I'm really curious if Google will
follow suit. Microsoft and Apple have an opportunity here to show they care
about end user privacy than Google.

~~~
gregknicholson
> it's healthcare, they are not going to worry about their conversion rates

Monitoring conversion rates can be used to find out whether people are
actually able to use your web service. The goal of a “conversion” doesn't have
to be a sale.

But I agree that if you're going to do this sort of tracking, it definitely
needs to be private.

I hope there's a court case soon where the court rules that sending a whole
load of business-sensitive data to Google, Microsoft and Apple actually _does_
breach a non-disclosure agreement.

~~~
jacquesm
There will definitely be such a case in the EU, that GDPR has some pretty
impressive fangs and either a healthcare provider, an ISP or an insurance
company is going to make an excellent example.

Coming to a courtroom somewhere in Europe in 2019.

------
x775
In about:config, setting 'network.http.sendRefererHeader' to 0 (default is 2)
will stop the referer header from being sent, and the document.referrer from
being set. See
[http://kb.mozillazine.org/Network.http.sendRefererHeader](http://kb.mozillazine.org/Network.http.sendRefererHeader)
for more information.

~~~
randomString1
There are several referer settings.

[https://raw.githubusercontent.com/pyllyukko/user.js/master/u...](https://raw.githubusercontent.com/pyllyukko/user.js/master/user.js)

user_pref("network.http.referer.userControlPolicy", x);

// TODO:
[https://github.com/pyllyukko/user.js/issues/94](https://github.com/pyllyukko/user.js/issues/94),
commented-out XOriginPolicy/XOriginTrimmingPolicy = 2 prefs

user_pref("network.http.referer.spoofSource", x);

user_pref("network.http.referer.XOriginPolicy", x);

~~~
x775
That is a very informative source, thank you!

------
saagarjha
> [https://www](https://www). healthcare.gov/see-
> plans/85601/results/?county=04019&age=40&smoker=1&pregnant=1&zip=85601&state=AZ&income=35000

For a moment I thought this was an example to make a point…

------
Tepix
Whoah, TIL that

> EFF researchers discovered this leak of personal health data from
> healthcare.gov to DoubleClick

It blows my mind that a site such as healthcare.gov would include 3rd party
trackers. You guys in the US really don't care about privacy at all.

~~~
rrcaptain
This is largely the part of independent contractors and subcontractors.
Contracts go to the companies good at winning contracts, not necessarily the
best company to do the job. The contractor takes an obscene profit for
providing no value and then subcontracts the project to various subcontractors
who may or may not employ actually qualified and skilled engineers.

The government employees managing the contract typically do not have the
expertise to evaluate the project or write proper specs. The HealthCare.gov
contract was a mess of incompatible buzzwords.

The engineers have no vested interest in the project as they're only there to
complete that contract and they're so many levels removed from the government
agency that no one actually knows who they are so it won't reflect poorly on
them when everything comes out poorly.

Because their career doesn't really depend on the success of the project, as
the government gets blamed for contractor failures while contractors get the
credit for success, they don't really need to do more than meet the specs. A
better way to do this would be to expand the number of engineers within the
government through groups like 18F and USDS, and give preference to them over
private industry.

Private contractors rarely work, but even when they do it's only when you have
expenses that the government doesn't need (such as contracting a machine shop
or car manufacturer to build something with their preexisting infrastructure).
In software though, your only expenses are really your engineers and the cloud
(as no one needs to run their own data center). The only thing subcontractors
can do that the government can't is pay their employees more than the GS
scale. However because the contract is supposed to be cheaper than the
government just hiring employees themselves (as industry has "profit motive")
they're going to have to cheap out elsewhere, either by hiring fewer
developers or neglecting parts of the development.

All of this is solvable by Congress, simply boost pay flexibility, but there's
no political motive to fix it as all of the contractors are political donors.
As a result, government software sucks.

~~~
belorn
> (as no one needs to run their own data center).

That can be extraordinarily expensive once it leaks out that classified
government data is in the hand of uncertified third-party cloud in some other
nation, and you have to rush and pay twice or three times more in order for
the contract to be changed and now have local certified supplier. This is what
happened here in Sweden in equivalent departments for the DMV, which later
implicated a further 40 different government department which used the same
practice.

When the cost go up by 200%-300%, suddenly the idea of running your own data
center sounds much cheaper. It ended up being the highest single cost the
departments had, excluding salaries and rent. you can get quite a nice data
center for those billions.

[https://simple.wikipedia.org/wiki/Swedish_Transport_Agency%2...](https://simple.wikipedia.org/wiki/Swedish_Transport_Agency%27s_law-
breaking_of_security_for_computer_system)

~~~
rrcaptain
The example you bring up in Sweden is exactly what I'm talking __against __.
Stuff was outsourced to other companies for the lowest bidder, who were not
held remotely as accountable as the government.

>IBM took over the agency's IT operations, and "IBM used subcontractors
abroad, making sensitive information and an entire database of Swedish
drivers’ licences accessible by foreign technicians who did not have the usual
security clearance".

IBM used subcontractors, which is the profit maximizing stuff I'm talking
about. When you pass stuff off to a for-profit corporation, they're going to
do what they can to maximize profit, even if it screws the government over,
because people will blame the government, not them.

>When the cost go up by 200%-300%, suddenly the idea of running your own data
center sounds much cheaper. It ended up being the highest single cost the
departments had, excluding salaries and rent. you can get quite a nice data
center for those billions.

The costs never were lower though. They just looked lower on paper because the
bill was less. But they weren't actually getting what they paid for.

There are already cloud providers certified for government use (at least in
the US). But you don't need to pay a company to pay some other company.
Government employees can do that fine.

------
staunch
If Mozilla genuinely prioritized its users' interest it would block ads and
tracking networks, which are the major way people's private information is
leaked and also a primary vector for hacking.

And yet for some mysterious reason Firefox hasn't broken ranks with Google by
incorporating ad blocking. Even though its an obvious major feature and
Firefox is losing marketshare every year.

We know why Google won't prioritize the interests of Chrome users but why is
the only major independent browser seemingly corrupt in the same way?

Mozilla should be helping society by pushing it past an era of internet
advertising and the clearly terrible clickbait-fake-news culture it creates.
And yet, it does not.

Is Google using the money it pays Mozilla to "discourage" Firefox from going
forward with ad blocking? As a concerned citizen, I sent an email to
antitrust.complaints@usdoj.gov requesting an investigation. Anyone with
insider info should send it there.

~~~
boomboomsubban
If Firefox disabled ads by default, a large number of sites would likely block
Firefox or set up angry messages telling the user to switch browsers. Yes, you
could bypass that, but most people wouldn't bother. Add in the logistical
issues, and the fact that some of the things blocked are beneficial, then I
see no reason to expect the conspiracy you suspect.

~~~
staunch
If a major service blocks Firefox users it could lead to a massive boycott.
Users could rally around Mozilla if it stopped selling out their interests.

More people use ad-blockers than Firefox has users. The best way to attract
more users is to make the best possible browser. That means incorporating ad-
blocking as users have loudly demanded for a decade.

~~~
yorwba
> If a major service blocks Firefox users it could lead to a massive boycott.

... of Firefox.

Most people don't care as much about their browser choice as about the ability
to access those major services. _I_ would continue using Firefox, but only
because I don't care about most of the popular sites. The majority would
switch browsers in a heartbeat.

------
alkonaut
#1 this should be on by default. I might be missing something, but do sites
really need the referrer? What would break if the browser sent the same page
as referrer, or google.com/ or something similar? Is there any value in the
referrer to the _client_? The host can use it for a whole range of reasons -
but apart from helping the host, what is the immediate benefit to the client?

#2 Won't this be possible to bypass simply by encoding more in the domain part
of the url than in parameters? So you switch from a.b.tld/foo?p=123 to
123.a.b.tld/foo ?

~~~
rovek
You're right about subdomains but they are stripping both query _and_ path, so
123.a.b.tld/foo would become 123.a.b.tld. It's fair to assume most misplaced
sensitive data will be in the path or query, rather than the subdomain.

~~~
alkonaut
I didn't even think about the subdomain until I saw the image that is in the
article here - which literally looks like it added an arbitrary number in the
subdomain, duplicating the parameter number! When I saw the image I thought
the article was actually about blocking some shady way of leaking that
doubleclick had invented.

------
jusob
Interesting, this is what the Referrer-Policy header is supposed to do, site
by site. It make sense to enable it private browsing mode, though... and then
you'll see how many sites break because they use the Referrer as some kind of
authentication mechanism (yes, seen in practice multiple times).

------
Justsignedup
I mean we already have firefox plugins to permanently block the referrer.
Which is great. But I applaud Mozilla for going privacy-first in a consumer
package. I hope that eventually Mozilla will focus entirely on privacy and
make good anti-tracking, anti-ads, anti-referrer, anti-cryptomining all
default packages.

------
makecheck
E-commerce checkout codes, etc. are the only reasonable form of referral. In
other words, if I give you something voluntarily that tells you where I came
from, fine; otherwise, why do we have so much auto-leaking built into
protocols?

~~~
eitland
Was very handy to find other people linking to your site, often people with
similar sites to yours.

Also I remember someone I know got an email that a page ge was linking to was
about to move. I guess this was only possible because of the referer header.

~~~
fiddlerwoaroof
It’s, nice, for example, to see where your GitHub repositories were referred
to

~~~
masklinn
Or which sites & forums were hotlinking your images, to decide who'd get horse
porn.

------
joelthelion
Are there any reasons not to get rid of referers altogether?

~~~
TheCoreh
They are useful to prevent the hotlinking of images/video from third party
domains, which can incur in significant bandwidth costs for smaller websites.

~~~
koolba
That would continue to work fine with a default same origin allowance. A
separate domain would require a referral policy.

~~~
pawelk
Origin policy is a client thing. Referrer is accessible to the server. It's
like using robots.txt vs. HTTP Auth to protect your content.

~~~
nothrabannosir
Referrer is also a client thing. Referrer based hotlink protection implicitly
trusts the visitors' UAs. Same for Origin; it just offloads the actual check
to the visitor. No big difference.

HTTP Auth, on the other hand, is actual password protection which cannot be
spoofed by a malicious client. Very different from robots.txt.

------
executesorder66
You can use the smart-referer addon [0][1] to send a custom referrer string[1]

[0][https://addons.mozilla.org/en-US/firefox/addon/smart-
referer...](https://addons.mozilla.org/en-US/firefox/addon/smart-referer/)

[1][https://github.com/meh/smart-referer](https://github.com/meh/smart-
referer)

[2][https://imgur.com/a/1HZQK](https://imgur.com/a/1HZQK)

------
newman314
For those who are curious, here's some more detail on the various options.

[https://feeding.cloud.geek.nz/posts/tweaking-referrer-for-
pr...](https://feeding.cloud.geek.nz/posts/tweaking-referrer-for-privacy-in-
firefox/)

Basically, set the following:

    
    
      network.http.referer.(XOriginPolicy|XOriginTrimmingPolicy|trimmingPolicy) to 2
      network.http.referer.spoofSource to true
      network.http.sendRefererHeader to 0
      network.sendSecureXSiteReferrer to false

------
compsciphd
Wouldn't this make it obvious that the user is browsing in private mode? While
I get that might be preferable to leaking information, its also not an ideal
solution either.

~~~
groovecoder
Disclaimer: I'm the Firefox engineer who wrote the patch and the blog post.

I'm very interested in this thread. Other replies here are correct - there are
many ways that sites try to detect private browsing, and many ways they can
get it right or wrong.

How do people feel about the "stealth" design goal of private browsing? Should
it be a goal? What about a hide-in-a-big-crowd tactic? (E.g., how Tor tries to
make all its users look identical.)

~~~
alkonaut
> How do people feel about the "stealth" design goal of private browsing?

I think this shouldn't just be a goal for private browsing, this should be a
goal for browsing period.

Shouldn't the default be to just send the top level domain (if anything) of
the source site as soon as you go somewhere else? The next site can't possibly
use the complete url of the referring site for any (non-shady) purpose?

------
codedokode
Sites like healthcare.gov or banks should not include third party ads or
analytics scripts. Referrer is not the only way to leak information from the
page.

------
dewiz
For the scope of requesting a document, there is no need of referrer nor
useragent.

A lot of features/apps/websites have been built around the assumption that
this information is sent, but it would be nice to start dropping it by
default.

------
jokoon
The weird thing about browsers that tries to protect your privacy is how many
website will get broken because of this.

I managed to disable cookies by default using cookie whitelist, and I counted
many website that broke down.

I applaud firefox for daring to break website for the sake of privacy, but I'm
waiting for websites to react.

Firefox should be even more strict regarding privacy: ask the user if he want
to set a cookie, never save history etc.

I'm using the extension that compartmentalize website usage on firefox, and
this should be made default.

~~~
Too
Early versions of IE asked every time if the site was allowed to set cookies.
Guess how fast people got used to clicking "yes and don't ask again".

------
kemitche
Color me stupid, but I thought all major browsers already stripped referrer
info when navigating from HTTPS? The examples used don't make sense to me if
that's true.

~~~
groovecoder
The default user agent policy is no-referrer-when-downgrade, which strips the
referrer header going from HTTPS pages to HTTP resources.

Firefox 59 PBM now implements strict-origin-when-cross-origin by default,
which trims the path off the referrer value of ALL 3rd-party requests.

------
rowyourboat
TIL that is an option at all. It's there in earlier versions, too, all they
did was change the default behavior for Private Browsing mode

------
marco1
Setting `network.http.referer.XOriginPolicy` to `1` in Firefox’s
`about:config` is actually a pretty reasonable choice for _all_ browsing, and
balances privacy with preventing your favorite sites from breaking.

------
jasonlotito
Flagged this as the headline here and the headline on the blog post do not
match, and the one here is not accurate and misrepresents the post.

~~~
njsubedi
Also it should tell that this behavior is only expected in "Private Browsing".

------
ecthiender
Why is this being only implemented in private browsing mode and not in the
normal mode? IMO, this should be the default.

~~~
vanderZwan
My guess is: they are letting the privacy nuts (and I am one) figure out the
kinks in the system first before unleashing it on the general public

------
rkagerer
This might be a dumb question but does whether a site uses SSL have any impact
on browser behavior concerning query strings?

Seems counterproductive my browser is taking so much care to encrypt my
querystrings then leaking them to any host from which the site I'm visiting
happens to pull content.

------
pasbesoin
Or, you could have chosen to not break the RefControl extension, that did this
and a lot more.

~~~
CorpusCalcium
See [https://addons.mozilla.org/en-GB/firefox/addon/smart-
referer...](https://addons.mozilla.org/en-GB/firefox/addon/smart-referer/)

~~~
pasbesoin
Thanks, but this doesn't appear to offer equivalent features. RefControl
allowed a default action overlaid by siste (pattern) specific overrides.

One could block refer(r)er altogether, and then adjust on a site/resource
basis as needed.

There is one current web extension compatible extension that purports to do
this, but when I tried it, it didn't want to cooperate with my configuration,
despite adjustments. Further, it sucked Discus comments into its local
configuration dialog/page, something that I find... sucks.

Finally, it wasn't open-source and didn't have a well-known provinance. All
this didn't leave me feeling too confident in it.

P.S. uMatrix is supposed to provide layers of referrer control, but I haven't
made the effort yet to switch over to it including switching some of my other
points of configuration to use it instead.

------
DarronWyke
I disable referers for all browsers. Firefox has them off completely, with
Referer Control using a random one for Chrome. I only enable them for sites
that absolutely need them (and that I need to use).

It's simply good data hygiene and privacy.

------
mehrdadn
Uhm, wouldn't this give an indication as to whether the user is in private
mode?

~~~
Nicksil
This was addressed in an earlier reply

[https://news.ycombinator.com/item?id=16286653](https://news.ycombinator.com/item?id=16286653)

~~~
mehrdadn
Thanks!

------
ospider
This feature should be easily implemented by a extension for Google Chrome
using the webRequest API

~~~
Sylos
Well, yeah, the feature itself is not a particular accomplishment, really it's
been in Firefox for years, you just had to enable it with some about:config-
flag.

The big news is them enabling it by default in some fashion (that is when
you're in Private Browsing), meaning that all users now have this, not just
the 0.1% who understand referrers (and have not forgotten to enable this the
last time they installed Firefox).

As a power user who knew about this, you might not particularly care, but for
users in general it's great, while it pisses off webpage owners.

Then again, even as a power user it's impossible for you to know about all of
these sort of config options, so you might care to use a browser which tries
to help its users out while having to keep an eye on not pissing off webpage
owners too much, rather than a browser that tries to maximize revenue for
webpage owners while trying its best to hide all the ways it infringes privacy
from its users.

------
therealmarv
Any other browser doing this? Safari?

------
spondyl
For those confused as to why half the comments have "misspelled" referrer,
here's an interesting bit of history:

The misspelling of referrer originated in the original proposal by computer
scientist Phillip Hallam-Baker to incorporate the field into the HTTP
specification. The misspelling was set in stone by the time of its
incorporation into the Request for Comments standards document RFC 1945;
document co-author Roy Fielding has remarked that neither "referrer" nor the
misspelling "referer" were recognized by the standard Unix spell checker of
the period.

[https://en.wikipedia.org/wiki/HTTP_referer#Etymology](https://en.wikipedia.org/wiki/HTTP_referer#Etymology)

~~~
seszett
> _Phillip Hallam-Baker_

Funny how the misspelling of a double consonant comes from someone who has a
misspelt given name with an extra double consonant!

~~~
ketralnis
I don't think you can ever describe a person's name as spelled correctly or
incorrectly. It's spelled how they spell it. It's their name, not a
dictionary's. Variations in spelling are perfectly natural and for names in
particular are incredibly common

~~~
seszett
I'm not "attacking" their parents who chose the name or anything like that,
and any spelling is fine for a name, but there is a clear etymology to the
name "Philip" that comes from "philos" and "hippos", someone who loves horses,
and an indisputably historically correct way to spell it.

That a misspelling has become particularly common (or like for my own name,
much more common than the historically correct spelling) doesn't make it
anymore correctly spelt than "referer" in my opinion.

But if you disagree with the term "misspelling", I can formulate it another
way: let's say that it's funny how the creative modern spelling "referer"
instead of the historical "referrer" comes from someone who has a creatively
spelt name "Phillip" where "Philip" was historically more common, and that
both differ from the historical spelling on a double consonant. It's a much
more awkward sentence though for such a trivial, passing remark.

~~~
danso
I think it's being argued that describing a name variation as a "misspelling"
\-- as in, a _mistake_ \-- is incorrect. What does the etymology or historical
popularity of "Philip" have to do with it? We don't know that he has that name
because he truly loves horses or because his parents attempted to honor a king
of Macedonia.

I know this is treading into the classic prescriptive vs. descriptive
linguistic debate, but the reason why we can call "referer" a misspelling --
rather than a creative decision - is because the original authors seem to
admit that it was unintentional. Fewer folks would be calling it a misspelling
if the authors had meant to do it, e.g. to avoid a name collision with some
other attribute named "referrer" or to honor a colleague named "Referer".

