
Ask HN: How does email tracking work? - markwaldron
From my understanding, Sidekick uses a &quot;tracking pixel&quot; that reports back to their servers when loaded, then sends the user a notification. Are there any guides out there on how to program something of this nature? Or are there any open source projects that may provide a more in-depth look into how email tracking works?<p>Any more information would be greatly appreciated.
======
davismwfl
I have designed a few of these over the years and worked on others
implementations. The common one I see the most is taking a 1x1 transparent
image in the email with query string parameters to provide the specific data.
Usually something like a this img src="/domain,com/imagename.png?i=UID. Also
sometimes we will use the company logo as the tracking image, simply because
it doesn't look like a "tracking pixel" by default.

The other way that I just did was basically the same thing, but the image
doesn't really exist, like img-uid.png where uid is the uid I care about for
tracking purposes. Then in my webserver setup I pass the details to a server
side component to manage the tracking and return a generic 1x1 transparent
image. There are lots of ways to do that, but it protects you for those cases
where an email client strips query string params on img src's.

Of course if the user doesn't allow the images to load you get no tracking, so
that is a limitation but most systems have that limitation. So you should also
have link tracking etc.

~~~
markwaldron
Awesome and very informative! Thanks for the explanation. I'm starting to get
a better grip on this. I found a few posts online explaining how to do this
with PHP and GD. In each of those instances, do you create a unique image for
each user or each email?

~~~
dangrossman
Let's say you assign each subscriber in your database a number. You send a
mailing out to the list. In the mail to the first subscriber, you include this
tag:

<img
src="[http://www.example.com/track.php?id=1">](http://www.example.com/track.php?id=1">)

In the mail to the second subscriber, you include this tag:

<img
src="[http://www.example.com/track.php?id=2">](http://www.example.com/track.php?id=2">)

And so on. There needs only be one file on your server to track opens no
matter how many people you mailed: track.php.

That one file reads the "id" parameter from the request to know which
subscriber opened the mail, and logs that in your database. It then creates a
1x1 transparent image, in memory, using GD, and outputs that to the browser
that loaded the image tag.

The image never existed on disk, and nothing needed to be created in advance
of sending a mail.

~~~
atomical
GD seems like overkill if the same 1x1 pixel is being generated over and over
again.

~~~
davismwfl
As a point, at least in my implementation, I actually use a byte stream for
the 1x1 that is cached and sent to the browser. We do this so that there is no
disk read ever for it. We also don't use php/gd, our code is mostly node.js
and a little golang with nginx as our front end.

It may seem a little overkill, but I found doing it this way made a pretty
significant impact when you have a reasonably decent volume of tracking going
on. A good http server of course would cache the image of course, but we just
found this to be easier and more consistent for our implementation.

------
zhte415
Most email clients block these by default, at least in a business setting. My
advice is to check your audience first.

Bounce-back report tracking could be useful if you're checking emails are
delivered. I've used this when emails that have to be delivered are delivered.
This was mainly for compliance reasons - whether they're read or not, or end
up in whatever user folder, was of little importance. That these emails are
delivered successfully was paramount. The bounce-back reporting showed where
customer contact failed and secondary measures were necessary to get back in
contact.

Your use-case may be different, but if anyone reading this is in a similar
situation to ensure delivery but not readership, implement a simple server-
side delivery/non-delivery report.

------
detaro
Basic version is anything that can look at a logfile of a webserver. Each mail
gets a specific URL for the image, HTTP request to there turns up in the log.

Or you put some kind of server side app on the server that tracks the request
in its own log.

~~~
markwaldron
Is the HTTP request for the image or for a blank page? I'm just guessing here,
but I guess a new file is created each time an email is created and then a
reference to who the email is sent to is stored in a DB somewhere?

~~~
detaro
something that isn't an image might show up on the page, even if only 1x1 px
big, so it's "cleaner" to send an empty gif or something. Doesn't have to be a
new file for everybody though -> one could simply redirect everything to one
file, or put the metadata in an URL parameter.

------
iqonik
It is normally an image like others have said, however, Google's caching
inside gmail is causing some problems:

[https://blog.mailchimp.com/how-gmails-image-caching-
affects-...](https://blog.mailchimp.com/how-gmails-image-caching-affects-open-
tracking/)

So I would say it is not 100% anymore, but, no alternative approach is out
there either.

~~~
Grynn
Google merely prevents you from collecting information about the IP address &
device. It still lets you know who opened the email. Since the request comes
from Google's servers instead of the client device (or browser) you get
whatever HTTP headers google thinks are appropriate (and of course the IP
address of a Google server)

AFAIK Google does not pre-load images (for example, even if no one opened the
email Google could request the image). This would destroy open tracking since
you'd get a lot of false positives.

Since "who" opened is already encoded in the image URL, you still get this
info.

Anyhew, there's a lot more people do with these tracking images - for example
see KickDynamic.com. You can change the image at time of open (to say reflect
the user's location, which you deduce from the IP). This can't be done if it's
gmail, since you don't have the client's IP.

It's kind of strange that Apple doesn't block (or anonymise) images though
they're so hot about privacy at the moment.

