
Mining my mailbox for top email service providers - kehers
https://obem.be/2020/02/18/mining-my-mailbox-for-top-email-service-providers.html
======
TacticalTable
It's interesting that this is missing a lot of major enterprise marketing
email providers.

The author clearly isn't subscribing to things like Best Buy's weekly deal
email, or Target cart reminders.

~~~
runlevel1
A few years back, one of our (Twilio SendGrid's) developer evangelists created
an open source Chrome extension to show the icon of the ESP who sent a message
in Gmail.

The last big Gmail UI update broke it, but most of the ESP identification
strategies probably still work: [https://github.com/nquinlan/Email-
Intelligence/blob/master/c...](https://github.com/nquinlan/Email-
Intelligence/blob/master/content.js)

~~~
pingec
This is a brilliant idea, if only it were updated for the latest gmail UI

~~~
mbrameld
[https://www.inboxsdk.com/](https://www.inboxsdk.com/) lets you pretty easily
manipulate the Gmail UI if you want to have a crack at it.

~~~
alooPotato
We (Streak) made the InboxSDK. Great for projects like this because we
autoupdate so you can write the extension against our high level api and not
have to worry about gmail changes going forward.

Let me know if we can help.

------
amirathi
Deliverability is a big marketing/selling point for all these providers &
important selection criteria for their customers. So it'd be very interesting
to see what % of emails for each provider landed in Inbox vs
Promotions/Spam/Junk folder.

~~~
oefrha
> So it'd be very interesting to see what % of emails for each provider landed
> in Inbox vs Promotions/Spam/Junk folder.

Are those percentages really a good measure of deliverability? I mean, a
significant portion of “one million dollars for you” scams in my Gmail spam
folder are from @gmail.com addresses; that doesn’t say anything about
gmail.com to gmail.com deliverability.

~~~
thaumaturgy
Gmail is somewhat a special case because email services are kinda forced to
accept inbound Gmail, even though it's quite a large source of scams and
phishing attempts today. But for others, they'll still readily get blacklisted
for a little while here and there.

I was thinking about spam as I read the post too, since this is a problem I
get to deal with almost daily. Marketo has a terrible signal-to-noise ratio
and it can often end up blackholed on my network with nary a user noticing.
Sendgrid is my next least-favorite email-as-a-service, and in fact I happened
to have this article open in one tab and in another tab I had this morning's
automated reports from some of my spam software showing that a pile of inbound
messages from Sendgrid had tried to land at a list of nonexistent mailboxes --
someone was rolling through "andy@", "barbara@", "vijay@", "kenneth@", etc.

I have had very little trouble with Mailgun in this regard, so far, but that
may only be because the spammers haven't fully adopted them yet.

As a sysadmin on the receiving side, I really loathe these services, they make
my job significantly harder and more time consuming than it used to be.

~~~
fireattack
>Gmail is somewhat a special case.. forced to accept inbound Gmail

Unless I misunderstood, the person you replied to literally says he has emails
from @gmail.com throwing in his Gmail spam filter. So not sure why you mention
Gmail is special here (regardless if it's true or not).

------
ksahin
I know the results are biased given it's a sample from OP inbox but I'm
surprised not so see [https://postmarkapp.com/](https://postmarkapp.com/)
here. I thought they were pretty popular.

~~~
jamestomasino
They're just for transactional emails, not for marketing ones. They wouldn't
show very much based on how he's measuring.

~~~
ksahin
It seems this post is about transactional emails, not marketing ones. CF the
first paragraph!

------
simias
I must admit that I was a bit shocked when I realized that the author had used
javascript for this task. At first I assumed that he was scripting some
webmail interface but then I saw that he was using 'emailjs-imap-client'.

I understand that people use what they're familiar with and there seems to be
a vibrant ecosystem for JS outside of the browser but it'll never not be weird
seeing it take over what would once have been written in Perl or Python,
especially since for a long time all I heard about Javascript was "well yeah
it sucks but it's not like I have a choice if I want to script in the
browser".

Software is eating the world, and the web is eating software.

~~~
quonn
Modern JavaScript/TypeScript is fine. Especially when compared to Perl, of all
languages.

~~~
AnIdiotOnTheNet
I disagree. JavaScript is one of those languages who's default failure mode
seems to be to pretend everything worked and just produce nonsensical results.

~~~
pxndxx
You're thinking of PHP. Modern Javascript has good error handling and is a
pretty nice language to use.

~~~
tjoff
I know that PHP gets a lot of flak but come on. Compared to javascript PHP is
a marvel. I recommend you to try it.

------
saagarjha
Semi-related: emails can often give away more information than you might
intend. At one of my previous employers, it was quite easy to get a general
idea of how internal development was progressing “from the outside” by keeping
track of who was testing which version of a prerelease OS via proxy of the
information it would leave in the emails they sent.

~~~
technion
I've identified companies as connected to each other by enumerating domains on
an azure tenancy.

------
lettergram
There’s some really interesting things you can pull out of your email. Having
done some analysis myself it’s actually fairly easy to export data from Gmail
to do an analysis.

This is from a few years ago, but I bet it’ll still do the trick:

[https://austingwalters.com/analyzing-email-
data/](https://austingwalters.com/analyzing-email-data/)

Everything is in python:

[https://github.com/lettergram/Email_Analysis](https://github.com/lettergram/Email_Analysis)

Regardless, I encourage everyone to do an analysis of their emails. Cut out
the junk, set reminders on when to check, etc. it’s both enlightening and
potentially horrifying to discover trends.

------
roseway4
The methodology used excludes a number of services using SparkPost. For e.g.
the excerpt below of an Intercom header. It's a challenging problem to solve
given the complexity of email delivery and appreciate this was a fun project
to take on.

Return-Path: <bounces+XXXX@XXXX.intercom-mail.com> Received: from
mta-216-35.sparkpostmail.com (mta-216-35.sparkpostmail.com. [147.253.216.35])
by mx.google.com with ESMTPS id t13si7000210pgg.534.2020.02.03.08.37.30 for
<XXXXXX> (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Mon, 03 Feb 2020 08:37:30 -0800 (PST)

------
dredmorbius
This website presents as a 23-chacter-wide skinny column in mobile vew.

~~~
_Anima_
On my mobile/brave browser, it was even less than that. Opened on issue on his
Github :-)

------
bognition
Very interesting thanks for sharing. initially I was surprised that HubSpot
didn’t make the list, until I realized that there is a nice intersection here
with what you observed and the target businesses of the Email services.

For example MailChimp Targets small B2C companies while Hubspot targets small
B2B business.

I’m guessing the op isn’t a small B2B business hence the lack of hubspot in
the list.

~~~
kehers
Yes, for the mailbox I ran it on. I'll like to see someone do this on a
"business" mailbox

------
thih9
Nitpicking: in the final chart with merged providers, the 'mandrillapp.com'
section and 'mailchimp' section are still shown as separate.

Is this intentional? I assumed the opposite, given that:

> Mandrillapp.com is the transactional email service for Mailchimp. It used to
> be a standalone service until it became deeply integrated into Mailchimp.

~~~
kehers
Yes it's intentionally as Mandrill is still used solely for transactional
emails and Mailchimp for marketing. Same reason I didn't merge Mailgun and
Mailjet.

~~~
thih9
Thanks for the explanation and for the article. That was a fun read!

------
hombre_fatal
[Nitpick] Instead of this:

    
    
        ;(async function () {
          try {
            // All your code
          } catch (e) {
            console.log(e)
          }
        })()
    

Just do this:

    
    
        ;(async function () {
          // All your code
        })().catch(console.error)
    

Overuse of try/catch is probably the #1 mistake I see when people use
async/await.

~~~
mrweasel
Either way, I find the syntax of both to be extremely ugly. Arguably this may
simply be me not getting JavaScript.

Starting a line with ; is just plain weird and the last bit })() seems like
German, where all the stuff that you technically not need to understand the
sentence is at the end.

Maybe someone can explain with JavaScript developers love anonymous functions
so much. Why not just name your function and make the whole thing a bit more
readable.

~~~
akamaozu
Not all JavaScript devs.

Love the language, hate the popularity of anonymous functions and non-
descriptive variable names.

My production code as a sample. [http://designbymobi.us/wp-
content/uploads/2020/02/rabbitmq-c...](http://designbymobi.us/wp-
content/uploads/2020/02/rabbitmq-customer-rpc-api.jpg)

~~~
mrweasel
Your code sort of have the same issue. Why not define a function:

function someName(p1, p2) {}

and then when you need it for something, like a handler, just say:

handler: someName

~~~
akamaozu
Hoped to show that JavaScript can be easier to read than you described.

Obviously wrong. My mistake.

------
hellofunk
Wow those pie charts are almost impossible to read! I've never seen such a
small font in a chart that readers are expected to understand easily.

------
wjossey
I had actually just tweeted about my woes dealing with email going to spam for
no good reason from my GSuite account.

[https://twitter.com/dustywes/status/1228733404826333185?s=21](https://twitter.com/dustywes/status/1228733404826333185?s=21)

I’m planning on doing a more thorough write-up on my solution (it’s
convoluted), but my tl;dr is that email seems woefully broken at the moment.
Hope my experience can be helpful for others dealing with the same issue.

------
bfishadow
That’s really nice analysis. Mailchimp is the biggest one as I guessed. Don’t
know if you count the spam folder since some marketing emails could be
automatically sent there.

~~~
frereubu
Not quite sure how you got that from the graphs. Top three:

Sendgrid: 30.2%

SES: 16.8%

Mailchimp: 16.8%

~~~
dane-pgp
Seeing Amazon's SES with such a large share is no surprise, but it is a little
disheartening knowing about this limitation:

"Amazon SES only supports open tracking over HTTP domains."

[https://docs.aws.amazon.com/ses/latest/DeveloperGuide/config...](https://docs.aws.amazon.com/ses/latest/DeveloperGuide/configure-
custom-open-click-domains.html)

Yes, big webmail providers fetch the tracking pixels for you and serve them to
your browser over HTTPS, but not every mail provider does that, sometimes for
good reason, leading to problems like this:

[https://protonmail.com/support/knowledge-base/connection-
not...](https://protonmail.com/support/knowledge-base/connection-not-secure-
warning/)

------
londons_explore
I'm wondering the legality of this...

The emails in your mailbox aren't yours to do with as you please - they're all
still owned by their original sender, who has given you an implicit license to
read them, and store them for later re-reading.

You don't really have rights to datamine, aggregate, train AI on, post stats
about, etc. the data in those emails. Its a violation of the rights of the
original sender to do it without permission.

~~~
scambier
Are you serious? Unless those emails contain work protected by copyright,
they're not "owned" by their senders. And I doubt that metadata falls in this
category.

