
How Simple Analytics calculates unique visits without cookies or fingerprinting - AdriaanvRossum
https://docs.simpleanalytics.com/uniques
======
AdriaanvRossum
Author here. Just to shine some light on the problem we are having and how
this solves that problem. We get more and more customers that are required by
law to have an active opt-in for cookies and fingerprinting. They basically
can't use those techniques before a visitor actually ticks a checkbox (which
has to unchecked by default).

This results in a lot of missed data. They can't run Google Analytics for the
visitors that don't check this box. For them it's a big issue because they
can't see how their visitors are behaving before checking that box.

Simple Analytics wants to fix this issue by not requiring consent. Customers
can use our tool for every visitor that lands on their website.

One other thing that is a common misconception is that unique visits with
cookies are accurate. They also have flaws:

\- What if a user uses a different browser

\- What if a user uses a different device

\- What if a user blocks (third-party) cookies

The same goes for using IP based fingerprinting. All techniques have flaws.
Although they are flawed they are used to get a number. It's not an accurate
number but it's a number. That's also the case with using our technique. It's
less accurate then using cookies, yes, but it's a number you can use. You can
compare it to a previous period, you can see the unique views of a page, etc.

If you don't need to comply with privacy regulations you can use all the tools
you want. We just focus on the companies who do care about those regulations
and give them the big picture they are now missing.

~~~
digitalengineer
> They can't run Google Analytics for the visitors that don't check this box

I don’t understand. They can. Just anonymize the ip. You can track all their
activities. Am I missing something?

~~~
AdriaanvRossum
Google Analytics tracks your visitors with a cookie [1]. You need consent for
placing that cookie under the GDPR and other privacy regulations.

[1]
[https://developers.google.com/analytics/devguides/collection...](https://developers.google.com/analytics/devguides/collection/analyticsjs/cookie-
usage)

~~~
eli
I don’t think GDPR necessarily says that one must get consent for every
cookie.

~~~
tyingq
Google analytics does use first party cookies for the site stats, but it also
s̶e̶t̶s̶ [edit] can set a third party DoubleClick cookie that tracks users
across sites. So that anonymizing the IP doesn't anonymize the user.

There's also optional functionality that can track users via the first party
cookie. Passing login id, for example, to GA.

~~~
founderling

        it also sets a third party DoubleClick cookie
    

I have never witnessed that. Can you link to a site that uses _only_ Google
Analytics where that happens?

Maybe you confused Analytics and Adsense?

~~~
tyingq
_" For customers that are using Google Analytics' Display Advertiser features,
such as remarketing, a third-party DoubleClick cookie is used in addition to
the other cookies"_

So, I suppose I should have said "can set", though remarketing is very common.

[https://developers.google.com/analytics/devguides/collection...](https://developers.google.com/analytics/devguides/collection/analyticsjs/cookie-
usage)

~~~
founderling
I would think that enabling the "Display advertisers features" only makes
sense for webmasters who also use Google Adsense. And then their visitors get
bombarded with all kinds of cookies anyhow. So I don't think it applies to the
use case at hand. Where a webmaster wants to implement statistics without
violating GDPR.

~~~
tyingq
It's used for Adwords. That doesn't change anything on your site. You can
remarket only on Google searches, for example...no AdSense involved.

------
pdkl95
> This referrer is very useful to figure out where traffic is coming from.

This is exactly why sending the referrer header is _insane_ [1]. The header
betrays the user's privacy _by design_.

Remove it ASAP. If your app breaks, that's unfortunate but necessary.

[1]
[https://news.ycombinator.com/item?id=20468115](https://news.ycombinator.com/item?id=20468115)

~~~
liquidise
Necessary for whom? Privacy purists seem to be surprised that website and app
owners want/need analytics to meaningfully improve their app's usability. I
praise SA's privacy oriented approach as a meaningful alternative to GA.

This sort of overly-critical hyperbole doesn't help the privacy conversation.
If follow SA loses a feature it likely needs to remain competitive, and
customers would have another reason to go back to Google Analytics. Hardly a
necessary privacy win a my book.

~~~
TeMPOraL
> _Privacy purists seem to be surprised that website and app owners want /need
> analytics to meaningfully improve their app's usability._

Both "privacy purists" and many other people see this as _expediency_ at best,
and a lie at worst (i.e. once marketing gets wind of the tracking built for
development purposes; see e.g. recent Gitlab fiasco).

------
maweki
As I understand it, this is the method Webalizer has been using to calculate
unique visitors for over two decades now. It is mindboggling that we have come
full circle and this is now a business model again.

~~~
rwmj
It's one of the methods we used when doing logfile analytics in the early
2000s. At the time it just appeared obvious to me. One problem with it (which
is why we combined it with other techniques too) is that some browsers even
then did not send Referer headers reliably, neither between pages on the same
site nor between third party sites, so what they say in the article about
"direct visit" is not reliable.

~~~
avian
> some browsers even then did not send Referer headers reliably

Apart from https/http thing mentioned elsewhere in this thread, there's also
Referrer-Policy [1]. I don't know how wide-spread its use is, but I find that
most requests I get have no referrer header these days.

[1] [https://developer.mozilla.org/en-
US/docs/Web/Security/Refere...](https://developer.mozilla.org/en-
US/docs/Web/Security/Referer_header:_privacy_and_security_concerns#How_can_we_fix_this)

------
marcusjt
There's nothing "unique" about those visits, just using the referrer isn't
enough to make them unique.

~~~
glitcher
The author's comment post discusses how the analytics are going to be flawed,
but is emphasizing that the trends in the numbers over time can be very
useful, even when we know the exact numbers are not precise.

I think it's a great trade-off to respect user privacy while removing annoying
popup garbage from the user's screen about consent. Aren't we all sick and
tired of these consent buttons everywhere online now???

------
__ka
I think a more robust (and privacy-preserving) way to calculate uniques ( per
time-frame as well) is to make use of the local storage / indexDB in the
visitor's browser.

This is how it would work:

\- Visitor X lands on your website.

\- With some JavaScript you check local storage if `dailyUnique` for today's
date is set.

\- if yes, send `visit` signal

\- if not set, send `unique visit` signal and set `dailyUnique` to local
storage.

You can apply this to any analytics use case. Its is private and does not rely
on referrers. We have been using this goal-attainment approach to do analytics
at my company for quite some years.

~~~
AdriaanvRossum
Author here. Our tool is GDPR compliant out of the box. We can't use local
storage (it's legal wise the same as a cookie). Although your suggestion will
provide more accurate stats, it's a trade-off we are willing to accept.

~~~
bouncycastle
How about referring from a https site? AFAIK, browsers don't send a referrer
from s https site.

~~~
michaelbuckbee
If the referring site is http and the site being referred to it won't send the
referrer header.

http --> https = no referrer header

https --> https = referrer header

~~~
sm4rk0
Actually:

https --> http = no referrer header

------
dennisy
Very excited by the title, but very disappointed by the content...

This is not true unique visit counting. Which can be achieved using non unique
values being set in browser storage, which is privacy friendly in my eyes.

~~~
AdriaanvRossum
I agree with you. It's privacy friendly to have a cookie stored on the browser
with a boolean only. Unfortunately it's not allowed by GDPR without asking for
consent first. Would be open to other ways without cookies and fingerprinting.
I think there is no other way.

~~~
grsmvg
So even if it’s obvious that there is no unique ID in there, you still need to
show the notice?

We really need to lobby for a smarter law.

------
raverbashing
I'm just glad someone is focusing on obeying the spirit of the law rather than
tacking an ironic "we value your privacy" popup to the website and continue
selling my data.

~~~
fhars
Yeah, I always read these popups as “we value your privacy, therefore we take
it away from you and sell it to the highest bidder” and hit the back button.

------
buremba
A user can land your page from multiple referrers (or the same) more than once
and some search engines including Google will hide the referrer information so
the calculation will be off IMO.

The regulations are not strict if you're not identifying the individual people
from your visitor/user data. If you're collecting user event data into your
servers and can prove that you're using the data just for the analytics
purposes (i.e. providing a better user experience to your users) there is
nothing you can afraid of using cookies.

That may not be the case for third party analytics providers though because if
you're using Google Analytics, you're responsible from how Google are using
your customer data.

------
grodes
The same user will trigger many unique visits if coming from, for example,
google search

~~~
AdriaanvRossum
This is true and that's a trade-off for not using fingerprinting or cookies.

------
WoefullyInept
It really doesn't solve any problems surrounding web analytics, user privacy
or user experience. I think it shows the creators lack of industry experience
in ecommerce, advertising and user tracking. It's like someone has sprinkled a
flawed 20 year old method of tracking in glitter and saas'd it.

I think there is a very small subset of website owners that would be willing
to pay for this. Then again he only needs to get a few people that aren't
sales focused and are deluded enough to think this is the pinnacle of
'privacy'.

------
narad
I am wondering how they will try to compensate when the browser is actively
blocking referrer information. In that case the data will be polluted like the
website has received more unique visitors, but it is not the actual.

In earlier times, this kind of tracking was the norm. Big Analytic companies
tried to distill this information and add value to the existing data to make
sense. So the cookies and fingerprinting were added to distill that data.

~~~
donohoe
Given you want discount internal site traffic from being unique, you could add
a query param to URLs as the user navigates.

------
founderling
This article uses the terms "visit" and "pageview" in unusualy ways.

They give the example that when a user clicks through from yourwebsite.com/ to
yourwebsite.com/page then the first is a unique visit while the second is a
non-unique visit.

I would neither call a "visit" but rather a "pageview".

In my book, the visit is what started from the first pageview on
yourwebsite.com site and continues to the last pageview.

~~~
AdriaanvRossum
Author here. It's technically more correct to call those visits "unique page
views" and "page views". I think visit is a very simple word to understand for
most people. My tool is targeted at people who want a simple tool, that's why
I want to keep the wording also simple.

~~~
Terretta
How we used these words 20 years ago, when server logs were cool:

Uniques are visitors not pageviews. Hits, page views, and sessions are
decreasingly granular content stats, not people stats. Visits and visitors are
people stats. Sessions tend to be the matrix of page views (containing
multiple hits) per visitor visit.

------
jypepin
The article says

> When a user lands on your website without visiting another website (direct
> visit) we will record it as a non unique visit

But then the image below says the contrary

> This will be tracked as a unique visit as the current website name is
> different from referer (not available).

Which is it?

Anyhow, this seems like this approach unfortunately makes the "unique visits"
data completely wrong, since users might tend to visit your website multiple
times a day.

~~~
AdriaanvRossum
Thank you. It was a typo, it's now fixed.

Unique visits data is never 100% accurate. What if a user switches from device
or browser?

~~~
thaumasiotes
Well, your site could require authentication. 100% accurate unique visits data
isn't really a good reason to do that, but it is a side effect if you already
do.

------
kimsant
PWA 1px image on top with cache-first show single visits. A cookie without a
cookie

~~~
retrobox
Creative - I love this solution

~~~
kimsant
with cache expire time set to midnight you also can get daily users

------
tiku
You could just add a unique value to every link between pages?

~~~
AdriaanvRossum
This way a user is still unique when he/she closes the tab and reopens, right?

~~~
kbsletten
What if they share the link on social? All the new traffic would seem to be
non-unique.

------
skizm
> Other analytics businesses use this technique (for example based on IP
> address). This seems privacy friendly but is considered fingerprinting. For
> which you need consent.

Do you need consent to count unique IPs that visit your site in the UK under
GDPR? Aren't there laws that say unique IPs do not correspond directly to an
individual (in the context of piracy), and yet for simple website analytics
you count IP as uniquely identifying a user? Am I required by law to change
the default config of apache or nginx to not log the IPs? Or is it enough if I
promise not to analyze the logs?

------
jgalt212
GDPR is just a mess. The bureaucrats who wrote it don't understand it, neither
do the enforcers, or the folks under enforcement.

Brussels should just shut itself down.

