
Gmail spam-filters Stripe security messages as well - nh2
https://github.com/nh2/gmail-spamfilters-paypal-security-messages#update-gmail-spamfilters-stripe-as-well
======
nh2
This is a follow-up to Gmail putting PayPal classifying PayPal security
notification emails as Spam
([https://news.ycombinator.com/item?id=19099887](https://news.ycombinator.com/item?id=19099887)),
and Google stating that it's a problem with Gmail and that they've put in a
temporary fix specific for PayPal.

Just now, 2 months later, the same issue happened with a security login email
from Stripe.

~~~
laurentl
Honestly, the first time I saw a Paypal email (something along the line of
“check out your account activity”) I thought for sure that it was a phishing
attempt. Not one link in their emails point to PayPal, they all point to stg
like paypal-communications.com. You know, the type of domain name a phisher
would come up with. Even the link talking about security doesn’t point to the
PayPal website! That, and asking me to check my recent account activity when I
know I haven’t used PayPal (and PayPal should also know that), is basically
PayPal training their customers to fall for scams.

~~~
scrollaway
Paypal has ridiculously low standards for their official emails.

[https://twitter.com/Adys/status/943346017608585216](https://twitter.com/Adys/status/943346017608585216)

[https://twitter.com/Adys/status/943513254655156224](https://twitter.com/Adys/status/943513254655156224)

------
fooblat
If you are using Gmail and you are not checking the spam folder at least once
a week there is a good chance you are missing legit emails.

At least, this has been my experience for a while now.

edit: typo

~~~
mda
I have almost never found anything of interest in spam for many years. Maybe
none, i can't remember a single instance. In the early years, yes there had
been a few.

~~~
int_19h
I was in the same boat (and checked periodically just in case), but I
recommend that you re-check. In this past year, something must have changed
substantially, because there are a lot more false positives. I just went and
looked, and about 1/3 of what was there wasn't spam, but stuff like e.g.
Kickstarter campaign notifications, and even a car manufacturer defect notice.

------
awinter-py
I mean a lot of companies are mixing spam and critical messaging -- square is
an example (every place I paid with square is now sending me marketing).

It kind of sucks that we still use email for password reset and billing stuff.
It's like the digital equivalent of certified mail with most of the same
problems.

~~~
shereadsthenews
This is the problem. Both of these companies are filthy spammers and are using
the same domains and networks for transactional mail as for spam. They've
created their own reputation issues. The fact that someone from Gmail spam ops
kindly offered to reach in and manually tweak the reputation for PayPal
doesn't mean this is a problem with Gmail.

------
jetrink
I just checked my spam folder and there is an email from YouTube in there. (It
is a notification that I subscribed to telling me that one of my favorite
channels posted a new video.)

~~~
_ikke_
Checked my spam filter, and there was only spam in there :-)

------
cmurf
I'm confused. How is it in 2019 that financial services companies, in
particular ostensibly internet savvy companies, aren't using DMARC? How in the
world can authenticated legitimate emails ever end up in spam?

And surely by now Google spam filtering has a pretty good idea what emails can
be categorized as "financial institution" likely, and if they aren't
conforming to an agreed upon DMARC policy to put those in spam?

Or even better, a mandatory agreed upon tag that indicates the content is
'financial institution account security notification' related (i.e. not
marketing spam from that institution), but any emails that use that tag but
don't also use the agreed upon DMARC policy always go to spam.

~~~
unilynx
Why should DMARC (unless combined with a whitelist) be a non-spam signal ?
Spammers would quickly learn to set up the proper DNS records.

~~~
progval
And they do. For a time I got a large amount of spam from dozens of cheap
domains registered at Namecheap (which sold .bid domains for 0.99€).
Additionally, each had a valid DKIM signature.

I notified Namecheap, including the DKIM signatures, but they told me they
couldn't do anything about most of them because "they are hosted with another
company". ¯\\_(ツ)_/¯

Though if the registrar deleted quickly these domains AND didn't make them so
ridiculously cheap, maybe it would become too expansive for spammers.

~~~
lucb1e
I also don't really get why we don't apply spam filtering retroactively a few
seconds/minutes after it arrived. At first you don't know the domain yet, but
at some point you can go "oh, this domain now reached more than 1000 different
inboxes for the first time, and of the 50 users who saw the message, 45%*
marked it as spam. Must be spam, let's move these and future messages to spam
folders for all who haven't downloaded/opened the message yet." There is so
much spam that is (nearly) identical and reaches a large percentage of people,
but it's just left in the inbox for years by outlook/gmail/etc., even if I
don't log into the account for months at a time.

If the spammers can only reach an initial small sample and the domain is next
to useless after that, even 99ct domains should not be worth it.

* Or whatever is a normal number. I know lots of people just leave the message as 'read' and don't bother marking it. I don't know how many users do this. Maybe one could also keep track of users who regularly mark something as spam and only count the percentage among those.

~~~
jcranmer
Spam filtering already relies on throttling. If you're a domain with unknown
reputation, you're generally forbidden [by the recipient] from sending a lot
of email until a reputation is established.

~~~
dane-pgp
This suggests that a weakness of the global email system is being exploited by
spammers: receiving mail servers aren't gossiping enough information about new
mail sending domains.

Presumably mail server operators are reporting obvious spammers to
(centralised) blacklists, but it would perhaps be possible to better tune a
heuristic (and increase the cost to spammers) by sharing information on the
number of non-spam messages received.

This could actually be done in a provable and relatively privacy-preserving
way, if sending mail servers included signatures of the hashes of the emails
they were sending. Every email that was received by a domain of unknown
reputation could have its hash+sig sent to a public distributed log somewhere.

If this was combined with some sort of good-behaviour bond that domain
registrars required (for domains that send email) and which was paid back
after a reputation was established, it would make cheap domains much more
expensive for spammers.

~~~
joshuamorton
This is tricky though. You don't want to make these heuristics public, or
spammers will just access them to switch domains, and if you say "you can't
see our reputation list unless you're Gmail, Yahoo, or outlook", that can
start to look a lot like collusion.

~~~
dane-pgp
You're right, but my hope is that forcing spammers to switch domains will
increase their costs to the point that spamming is no longer financially
viable.

If the cost of domains is already such a significant expenditure that they
need to look for sub-one dollar registrations, then requiring, say, a $10 bond
on all domains with an MX record might erase their profit margins completely.

(There is a question of what constitutes "good behaviour" and whether that can
be gamed by having spam domains reporting each other as sending legitimate
email, but if these ratings are public then people can choose which ratings to
trust. Domain age would probably be a good heuristic there too.)

------
tlogan
I can confirm that there are some big problems with Google spam filter. Even
emails from Google G Suite support ended up marked as spam. That started
happening like 4 months ago.

We had a problem with following emails:

\- all emails from sns.amazonaws.com are in spam

\- all Stripe emails about "Suspected fraudulent payment on your Stripe
account" are in spam

They all say "Similar messages were used to steal people's personal
information. Avoid clicking links, downloading attachments, or replying with
personal information." Anyway, if you depend on G Suite email to receive
important notifications from PayPal, Stripe, or Amazon you will need to
monitor your spam folder. It seems like there are some keywords (invoice,
payment - I really do not know.) which will trigger spam filter.

------
mesaframe
Situation is worse with Outlook. They'll mark mails from Microsoft itself as
spam.

~~~
aendruk
This is good. Several of the messages from Microsoft that I've received in my
Office 365 mailbox _are_ spam.

~~~
vimslayer
This. So many people in this thread seem to think that an email needs to be
from a Nigerian Prince to be spam.

------
aloukissas
Just the other day, I posted a similar question [1] here, after discovering
that even gmail-to-gmail messages I send to my close friends get categorized
as spam. There has been a significant regression in Gmail's spam filter
recently (both false positives + negatives).

[1]
[https://news.ycombinator.com/item?id=19500357](https://news.ycombinator.com/item?id=19500357)

~~~
metildaa
Spammers are using Gmail for mass mailing now fyi.

------
mike-cardwell
Google recently filtered some emails into spam for us. Those emails were sent
by Google Docs, in response to us adding comments to our own private doc. This
was all within the same organization.

I've also repeatedly seen them suddenly spam filter individual messages in
long threads of conversation. Like they weren't taking into account that a
message was part of a two way conversation at all, even though they have In-
Reply-To and Reference headers linking them to those non spam filtered
conversations.

They also randomly spam filter transactional emails that we send from our
servers that are _from_ our own domain, _to_ the same domain as hosted by
GMail, that is fully validated by SPF, DKIM and DMARC. That should never
happen. We clearly trust our own domain, so if mail from that domain is
authenticated, let the bloody email through 100% of the time!

It's like they spent all their time working on how to blacklist email, and
zero time working on when to whitelist it.

The most important part of a spam filter is the whitelisting.

------
vbezhenar
I guess Stripe is big enough to notice for them. But imagine tiny website. Is
there any chance to reach gmail developers in case of similar failure? Gmail
just makes e-mail an unreliable transport.

~~~
massaman_yams
It's not the best solution in cases of clear false positives, but they do
provide a help document outlining what kind of practices result in reduced
risk of these kinds of problems. (It's oriented toward high volume senders,
but many of the recommendations apply at any scale.)

[https://support.google.com/mail/answer/81126?hl=en](https://support.google.com/mail/answer/81126?hl=en)

~~~
sjagoe
Their postmaster tools don't show anything up at low volumes. I'm a freelance
consultant, and my mail volume is very low. Usually setting up contracts, and
sending out invoices. Not much else. My mail volume is too low to show up
there.

I don't send out bulk or promotional emails, I use SPF and DKIM, and I've
still seen a lot go to spam. My mail is hosted by FastMail, not a home server.

------
sahillavingia
This is a huge problem with Gmail. They've been filtering legitimate receipts
and things from Gumroad in the last couple of months, and I've heard similar
things from founders of several other companies.

Please check your Spam folder, and mark the legitimate emails as such.

Hopefully it'll help the folks (or AI) over at Gmail get better!

------
loeg
I recently got a paper statement from my bank (I'm signed up for online
statements only) because the bank claimed my gmail inbox rejected its
notification email. (I checked spam, and it didn't land there either.)

~~~
tgsovlerkhgsel
There's an ever-growing list of stuff you need to set up to be considered a
legitimate mail server. It started long ago with a valid reverse DNS entry,
and now you need DKIM, SPF and a bunch of other stuff:
[https://support.google.com/mail/answer/81126?hl=en](https://support.google.com/mail/answer/81126?hl=en)

Given that banks seem unable to send e-mail that doesn't look like phishing
(using non-https links and often separate domains), I don't have high
confidence that they can get the basics right.

~~~
loeg
In contrast, I've received every one of their past notifications. Earlier bank
notifications all show DMARC/DKIM/SPF "PASS" ratings. Who knows.

------
sjagoe
I hate GMail and G Suite spam filtering.

I am an freelance consultant, and a number of times I have seen my business
email (hosted at FastMail) go to spam in a Google hosted mailbox. As a single
consultant, I don't send a huge volume of mail because a number of my clients
set me up with their internal email for business communication. The mail that
I do send is invoices, and setting up contract signing; that sort of thing.
The mails are always expected communication. I have SPF and DKIM configured
and validated by FastMail. Mail somehow still goes to spam.

I set up with google's postmaster tools, but it shows nothing because I don't
send a big enough volume of emails for it to care.

I set up dmarc to get reports when mail is delivered under my domain to a
GMail or G Suite hosted address. Every single dmarc report (it's easy for me
to keep on top of them because my mail volume is that low) is "spf: pass;
dkim: pass".

I really don't like that the only way to get this looked at is through
publicly shaming Google like this; I have not seen any way I can talk to
somebody at Google to solve the problem. FastMail support said they were
unable to help me solve it either, suggesting I need to send mail into the
Google garden and have the recipient click "not spam" until their filter
learns.

~~~
tlogan
I think it is not up to "SPF and DKIM configured". I believe is because of you
have some "bad" keyword invoice, money, etc. in the body of the email causing
this.

I assume that is the problem because when I reported to Google support about
"why this Stripe email is in spam" and I provide them with email body. Then
reply from Google G Suite support went to spam (since it had email body in
it).

And of course the fact that you use FastMail does not help.

------
strogonoff
For a few years now I have Gmail’s spam filter disabled completely on my
accounts.

I’m not sure you can do that with a setting, but there is a workaround where
you create a filter that excludes messages matching some random UUID, and tick
the “Never send to spam” action.

So far to me the trade-off has been worth it (YMMV). Not a huge burden in
terms of junk mail influx in my inbox, but one does have to be on alert for
high-quality phishing when reading mail (check headers, for example).

~~~
nh2
That is exactly what I do (and what results in the "This message was not sent
to spam because of a filter you created" message in my screenshot): Create a
filter that maches all and selects "Never send to spam" as action.

------
mehrdadn
Is this a Gmail issue or a G Suite issue? You'd think they're similar but they
haven't been in my experience. G Suite spam filtering has always sucked for me
and others I know. I've gotten completely legitimate _replies_ classified as
spam from completely normal and legitimate people _in the same domain_ as far
back as 2013, if not earlier. In the case of someone else I know this happened
to, it was an extremely critical email that got classified as spam.

P.S. If any Gmail folks are here: There's a bug in Gmail/G Suite that might as
well be equivalent to marking an email as spam, since it prevents you from
seeing emails: last time I checked, the "Important" section of Priority Inbox
seemed to sort based on the timestamp of the most recent _important_ email in
the conversation, not the most recent email in the conversation. So if a
subsequent email comes that isn't Important, the conversation doesn't pop up
to the top -- so if it's not on your screen already, you don't see it as
unread.

~~~
shereadsthenews
A lot of large enterprises using GSuite route their mail through external
systems that provide various dubious services and which often screw up the
spam classification by having dirty VIPs or being open spam relays etc.
Perhaps your organization was infected by such a thing. Otherwise having a
threaded message or a sender in your address book is a virtually guaranteed
saving throw against spam classification.

~~~
mehrdadn
You made me go back and check the email I had in mind. I'm not sure to be
honest. What I do see is that it soft-failed SPF:

> Authentication-Results: mx.google.com; spf=softfail (google.com: domain of
> transitioning {user@domain} does not designate {IP} as permitted sender)

So I guess you can use that to blame the domain owner, but I feel like the
fact that it was a reply to my own email should've been enough to allow it
through? Moreover, there are other emails around that time that _weren 't_
replies but still softfailed SPF, and they were still let through.

~~~
shereadsthenews
Your domain needs to:

    
    
      include:_spf.google.com 
    

... in order to send outbound mail from Gmail effectively. Many people don’t
realize that Gmail sends mail to itself over SMTP, not some special magic
protocol, and it doesn’t exempt itself from SPF checks and whatnot.

~~~
mehrdadn
They seem to have put up that SPF record now, so I can only presume they
weren't aware of this at the time.

------
Animats
It's a Google product. Of course it limits your communication with Google
competitors. If you used Google Pay, like you're supposed to, it would work
fine. It's a free email system. They have to make money off you somehow. What
did you expect?

~~~
nh2
As per other comments in this thread, it seems to classify emails from Google
products as Spam as well.

Also for a minor correction (not that I think this matters):

It's not free, it's paid for, because it's a G Suite account.

------
phantom_oracle
Googles draconian algorithms aside, there are legitimate security concerns
with "security alert" messages.

Here are some of them:

1\. The likelihood of you clicking and opening such an email is higher than
regular emails

2\. Phishers and scammers know this (point 1 above)

3\. Malicious content is embedded in such emails

4\. It is very effective, even against tech-savvy individuals

~~~
renholder
> _Malicious content is embedded in such emails_

Isn't that what content scanning is supposed to check for? I mean, it's not
like they're _not_ scanning the email body for keyword extrapolation, anyways,
right?

To be fair, though, Stripe _should_ change their SPF record to hard-fail
(-all).

;; ANSWER SECTION:

stripe.com. 600 IN TXT "v=spf1 ip4:173.193.210.51/32 ip4:166.78.69.60/32
ip4:198.2.180.60/32 ip4:13.111.2.227/32 include:spf1.stripe.com
include:greenhouse-outbound-mail.stripe.com ~all"

stripe.com. 600 IN TXT "docusign=4a93db58-af07-4632-a881-b569d41a6c57"

------
cavisne
The problem with both domains is they can be used for actual spam, like fake
invoices/receipts.

~~~
RIMR
With a proper SPF record, we can verify whether or not they came from the
right servers. Google verified that PayPal's e-mails were legitimate.

You can't spoof those addresses convincingly unless you compromise a
legitimate mail server listed in the SPF, or hijack BGP to impersonate a
legitimate mail server in the SPF.

Google is filtering known trustworthy senders for dubious reasons.

~~~
brazzledazzle
That’s not what they mean. They mean the services can be used in a
“legitimate” way that doesn’t involve hijacking to spam people.

------
speeq
My Gmail spam folder has emails from Google Cloud in it which I find amusing
because it states the following:

> Why is this message in spam? It is similar to messages that were identified
> as spam in the past.

Oh really Google? You've identified your own emails as spam? Well then!

~~~
kyrra
It's more like people are cooking spam when they get the emails.

------
sneak
I run my own mailserver (not an open relay, long-term ip, et c) and gmail
always puts my messages in spam.

It’s sad and annoying.

