
Lies, damn lies, and front-end tracking - AlexeyMK
http://alexeymk.com/2020/07/14/lies-damn-list-and-front-end-tracking.html
======
ve55
>The less JavaScript (especially third-party) you have on your landing pages,
the better. It’s a better customer experience, and it improves your page’s
conversion and quality score.

Really wish I read this more. And not just for the landing page.

~~~
messe
I'm surprised we haven't run into this before now. JS performance on android
has been stagnant for a while now, yet we still see the same awful JS-heavy
applications.

This doesn't bode well for the end of Moore's law. That is Moore's law in the
performance sense, i.e. guaranteed future single-core performance increases
[even if small], not Moore's law as in the the doubling transistor density.

~~~
phendrenad2
Companies that keep their JS footprint small will have faster-loading pages,
and that translates to non-trivial increase in profits. Sadly, people running
businesses don't understand this, and tech teams certainly aren't going to
voluntarily make their jobs harder without word coming down from above. So do
your company a favor - educate those in charge about how much page load time
matters.

~~~
smart_jackal
Unfortunately, Amazon S3 and Akamai CDNs will come to their rescue in still
making their pages load faster. As a plebeian, all my efforts at keeping my
pages light will go in vain as I simply can't afford the CDN infrastructure of
these tech giants.

~~~
MaxBarraclough
Don't today's bloated, slow-loading pages already use these CDNs?

------
m463
> inform Google/Facebook via their server-side APIs when feasible, instead of
> trying to load a pixel

I was wondering when this was going to be the norm.

Can't block third parties for privacy if the first party talks to them behind
the scenes.

(If I understand what this means)

~~~
amluto
I imagine that the vast majority of users of Facebook SDKs are not actually
motivated to inform Facebook about anything. They’re using the SDK’s to enable
login, likes, etc. If this moved to the server side, they might only inform
Facebook about actions that involve Facebook.

edit: for the specific case of conversion tracking as mentioned in the
article, I would hope that only referred by Facebook/Google would be reported.

~~~
Nextgrid
A good portion of the SDK usage in mobile apps is not nefarious. They're
embedding it for advertising conversion tracking which would respect the
"limit ad tracking" setting in your device's settings.

The problem is that beyond whatever functionality the developer of the app
intended to use the SDK for, it also does its own thing of stalking the user
even if the app doesn't end up using any SDK features at all.

~~~
amluto
Exactly my point. The third-party app is not nefarious, but the SDK is. A
server side API would have a better chance of doing the useful bits without
the nefarious parts.

I doubt many competent server operators would be willing to run Facebook-
supplied _binaries_ on their precious servers, especially given Facebook’s
track record of SDK crashes. (Not to mention PCI requirements, etc.)

------
dntrkv
We ran into the same exact issue at my previous employer. It's scary how
similar it was to your situation given we also moved our site over to Next.js
and we were a competitor to Opendoor. It also wasn't the first time I've run
into bad metrics in a legacy app making it appear to convert better than a new
version.

------
dccoolgai
Sshhhh. So many people will lose so many bullshit jobs if this gets discussed
widely. There are entire tertiary industries built on top of front-end
tracking and most people with a modicum of analytical ability know it's
turtles all the way down.

~~~
jkdfslgjklgdf
What do you mean by "turtles"?

~~~
dccoolgai
A turn of phrase:

[https://en.m.wikiquote.org/wiki/Turtles_all_the_way_down](https://en.m.wikiquote.org/wiki/Turtles_all_the_way_down)

In this context, I'm referring to the "trust us, it works" attitude that
people who sell/provide analytics on front-end tracking use to justify their
effort/expense. Based on experience, when you point out obvious problems doing
analytics with JS, they keep repeating the mantra "but you _need_ front-end
analytics" infinitely.

------
ravivyas
The headline, the problem and the solutions are all different things.

\- Front end tracking did not lie, the people tracking it were not aware. If
opendoor was spending on Ads, the marketing team would be the first to see a
disparity between the clicks on an ad network and pageviews on their end.

\- As such this is also a reason why you need to check your server hit logs
with your Analytics.

\- Where is the author is right it Frontend numbers being incorrect largely
due to blockers

\- His understanding of bounces is also incorrect. Bounce is when there is no
second "interaction event", many marketing folks fake-fix the bounce rate, by
sending a scroll event as an interaction event.

\- I always tell people that analytics numbers are "signals", not "metrics",
they are not accurate enough to be called "metrics"

~~~
d0gbread
If these numbers were significantly changed by a server side page view, I'd
bet their page view wasn't firing any earlier than the top or bottom of the
body tag, or maybe even on window ready. Anyone that's spent any real time
outside of dropping in a snippet understands the implications of users
navigating before hits fire, and it sounds like the team just didn't have that
experience.

Like you, I also work to make sure people understand that client side data is
not necessarily accurate, but still refer to it as metrics and KPIs, just with
the caviot that we should bank on consistancy over accuracy, keeping our
analysis geared towards behavior and trends versus most business people's
immediate desire to take it to an operational/business intelligence type
place.

------
XCSme
I don't think this is an issue with the on-site analytics, but with the
quality of the traffic and website performance. If you make sure that:

1) Your site loads in under 3-4 seconds for any user.

2) The user is interested enough to wait 3-4 seconds until the page loads.

Then most issues will be solved.

The problem with ads in many cases is that the traffic they send is of very
low quality or just bots. In the end you already know from your ads provider
how many users they say they sent and you should always use that when
calculating ad conversion rate.

Also note that using Cloudflare will count as bounced users who never actually
even tried to load your page (bots, crawlers, scrapers, all HTTP requests).

~~~
dogfoods
It's amazing that 3-4 seconds is considered a reasonable target in 2020 when 8
seconds was a target on freaking dialup. It's like having a Ferrari and
considering outrunning a horse a success.

~~~
ianamartin
No one thinks that's a reasonable target. That's insane. I don't know who that
person is or what he's in charge of, but that's the fastest way to being a
dead company.

100 ms max on backend processing, 500 ms max on first time to interaction.
Client connection speeds matter, obviously. 3-4 seconds is get your fired ass
out of here in my world.

~~~
XCSme
3-4 seconds includes the DNS resolving, downloading HTML, CSS, JS, media,
parsing them, creating the DOM, layout, calculating styles, rendering, etc.
It's very hard even for a static imageless site to load in under 1 second for
low-end mobile device.

That being said, for good UX the target should definitely be under 1 second to
interactice, but even on desktop, if your users can't wait 3 seconds after
they click the ad, they are not actually interested at all in your product.

People are also used with waiting a bit after they change sites/applications
when they initiate the transition, so transition feels faster than it is. It
starts feeling slow when you are already on the blank page, having time to
look at the loading spinner.

I'm not saying slow is good, but I highly doubt that you can make any site
have 500ms TTI for any user (they could easily have 150ms ping to your closest
server). I assume that TTI means having access to all website functionality,
not some simplified placeholder UI.

~~~
ianamartin
If users are used to waiting, it's because people like you have trained them
to be. You list all the things that go into rendering a usable UI as though we
don't all know that. Users probably don't, and that sounds like a lot. It
sounds like the kind of excuse for shitty load times that PMs use when people
do complain about slow websites.

And you can doubly tell how much of a PM you are by assuming that everyone is
accessing your product by ad and how much you don't care if they are
interested after 3-4 seconds. That's a lot of money you're leaving on the
table.

Low-end mobile devices are not the right metric for TTI.

Most people don't realize how fast the web can be because they don't bother to
benchmark until they are well into the tooling choices and feature development
and maybe someone somewhere decides to complain about speed.

A functional Pyramid app with dynamic templates using Mako, and some db
queries via SQLAlchemy can fully load in under 100ms.

The bottom line is that speed is a feature. It's not just one feature; it's
your most important feature if you want users. It's more important than
developer productivity; it's more important than any other feature. You don't
know this because you work in a b2b/enterprise space where users don't have a
choice. Their managers make the choice. But if you ever work directly on
consumer products you'll find this out really quickly. The difference between
~.5 sec TTI and 4 sec TTI is millions of revenue per minute in eCommerce.

But even though it's b2b doesn't mean it has to be bad and suck people's time
away from them. You have the option to learn something and change your
priorities. But you won't.

~~~
XCSme
Wow, a personal attack based on assumptions.

I just want to say that I'm not a PM and I don't even work in a corporation. I
have worked my entire career focusing on performance (learned programming for
and during algorithmic contests were every millisecond really matters, made
open-source contributions that vastly improved performance, worked in the
gaming space where even users care a lot about performance, and working in the
analytics space, where I created a high performance, lightweight alternative
to some heavy and data demanding alternatives).

I respect your opinion, but my comment was based on experience, knowledge of
human behavior and analytics and A/B tests ran on millions of users. I do
agree that it might differ from product to product, but in the majority of
cases you just get to a point where it is "fast enough", users no longer
notice and spending more time on optimizations takes too much effort for the
benefits it brings.

------
EdJiang
Wouldn't the B variant show higher session count? If your A/B testing tool
doesn't detect imbalances in cohort size I would imagine you have a bigger
problem, since it's easy to accidentally measure the A and B groups
differently.

~~~
jart
Important takeaway is they rewrote their website from scratch before even
having the A/B testing. Probably one of the best arguments I've ever seen
against writing bloated website code. We now know that software can hit a
point where it's so slow, that it produces false metrics about its own
slowness. Imagine how many people are still out there, who didn't write
website B, and are thinking, oh my bounce rate is fine, I don't need to invest
money in performance, since the numbers are telling me people don't care.
Folks who trust numbers at face value, don't always know what they don't know.

~~~
AlexeyMK
author here

> We explicitly only changed the infra which served our landing pages, and
> kept the content - the HTML/CSS/JS - identical. Once the new infra was shown
> to work, we would begin to experiment with the website itself.

\- [http://alexeymk.com/2020/07/14/lies-damn-list-and-front-
end-...](http://alexeymk.com/2020/07/14/lies-damn-list-and-front-end-
tracking.html#fn:1)

------
pachico
Although interesting story, it seems the main issue was that the author never
sanitized their analytics in the first place and blindly relied on data that
was never confirmed before. I've seen this as a quite common phenomenon. No
piece of software does magic and they all require certain certain conditions
to work as expected. In client JavaScript, these conditions are simply harder
to grasp but they can be studied.

------
ddevault
There's a fourth reason not to do this that isn't mentioned in the article:
_tracking is unethical_. Don't do it.

~~~
kirillzubovsky
Parker Thomson, aka StartupLJackson, had a great view on this. Would you
rather see a ton of ads that are completely irrelevant to you, or see ads that
you actually need?

Tracking that makes your life better is useful. If Opendoor knows you are
looking for a new house in Arizona, and they can show you just the right house
before it's sold to someone else, isn't that fantastic?

~~~
IggleSniggle
Honestly? I'd rather see more ads that are irrelevant. Ads are already an
attempt at manipulation. I would feel better knowing that everyone else is
seeing the same shit I'm seeing. That I'm not being pandered to. That there
are things that are valuable to others that I have no personal value in.

That's part of living in a diverse world.

I don't know about you, but I don't want a myopic world for myself. The
perfect ad targeting system removes my agency; decides what I want and can
afford on my behalf; makes my world significantly smaller.

~~~
d0gbread
And I'd rather see more search results that aren't tuned to me. Down with
algorithms! Down with personalization!

I'm surprised how much hate there is on HN - the entire future of technology
feels like it's heavy on data, personalization, and prediction, but a lot of
comments read like a parent or grandparent that was targeted and is loading up
their shotgun.

The technology isn't the enemy, some of the use is. And in some cases, we need
legislation to catch up, like it gas started to for things like loot boxes /
gambling in kid's games.

------
malisper
Can someone explain to me why third party JS has such a big impact on load
time? I thought browsers deprioritize third party JS and load it after all
first party assests have been downloaded and the site has been rendered. If
that's the case why does third party JS still have such a big impact on page
performance?

~~~
defjosiah
You can (usually accidentally) end up with a render blocking script tag. In
this article example it was client-side optimizely.

The parsing and execution of third-party javascript is definitely non-trivial
if you profile it, especially on lower end devices.

Finally, browser download priority requires async and defer attributes on
scripts (usually), or other clever ways of deferring loading.

~~~
malisper
> You can (usually accidentally) end up with a render blocking script tag. In
> this article example it was client-side optimizely.

This makes sense for a tool like Optimizely which has to block rendering.

> The parsing and execution of third-party javascript is definitely non-
> trivial if you profile it, especially on lower end devices.

Given that this is supposed to happen after the page has already loaded, does
this matter? If the thing you are optimizing for is page load time, and it
takes a few 100ms after the page has already loaded to run the JS, does that
actually negatively impact the user experience?

> Finally, browser download priority requires async and defer attributes on
> scripts (usually), or other clever ways of deferring loading.

When you add third party JS, shouldn't this be taken care of for you? Scripts
for adding third party JS should already make use of async and defer as
needed. What's a case where you would need to actually think about async and
defer when making use of third party JS?

~~~
defjosiah
This type of thing usually ends up being “death by a thousand cuts”. async +
defer help out, but they do still incur parsing and evaluation (iirc, before
document onload event) overhead. If you delay loading until you’re sure the
page is interactice, you’ll end up loading third-party pretty late (which
impacts metrics, and usually isn’t out of the box supported).

On a standard product-ey site with retargeting ads, user tracking, etc. this
third-party slowdown is significant.

All of this is exacerbated on lower-end devices, and non-WiFi Internet.

------
itisit
*damned

~~~
npteljes
And the manager peeked in, not peaked in.

~~~
AlexeyMK
Thanks! Updated on both fronts.

