EDIT: Found it. https://news.ycombinator.com/item?id=9855030
However, previous to setting all that up gmail would commonly mark my messages as spam. I haven't tested it in the last few days.
Because, as an individual, you won't qualify for their FBL service and "mysteriously" you'll have weeks of everyone saying you end up in their spam folder.
I'm not against Gmail, just like I'm not against Outlook.com or Yahoo mail or something. It's just that providing tools only work for players in a power position (i.e. Google) who can afford to ignore small players (i.e. me), and what's more, this further strengthens their power position: the better they can detect spam so more people will start using it (the postmaster tools are there to help people prove they are good, thus helping Gmail distinguish).
This was years ago but I guess that person was not alone.
I don't know about prior messages counting against you, given what I've seen it seems to makes sense. Without insider info we can only speculate.
You know how Google treats its free customers with utter contempt? I can assure you, they treat paying customers with the same contempt.
Those of us that are running mail servers are really not loving Google at the moment.
It's a tad terrifying.
Some networks are whitelisted for practical purposes. Google's one of them.
If necessary, I could provide a list of the subject lines of the emails that have been reported. A YC company, Zenefits, is one offender that comes immediately to mind.
But not until Monday. I'm going camping.
Example.com SEO Issues
Tough Times With Example.com? Needs Attention!!?
Get ranked higher: Example.com
Google optimization for Example.com
Get ranked higher : example.com
Website Audit Report to increase website traffic
Example.com - audit report now available
Give a glamorous new look to your website
Web Design Proposal example.com
Organic SEO Promotion For Example.com
"Re: Zenefits + Problem"
"Higher Targeted Traffic: Associatedtechs.Com"
"Re:re: UL,CE,ETL Split-core Current Transformer(0.333V or 1A/5A output) ,Rogowski coil,hall AC-DC transducer"
"Poor support processes could be costing you customers"
"Get ranked higher:"
"Higher Targeted Traffic: Associatedtechs.Com"
"Re: Zenefits + Problem"
"Re: Zenefits + Problem"
"Get ranked higher: associatedtechs.com"
"Google Update for: associatedtechs.com"
"How to increase your website traffic and generating leads??"
"Zenefits + Problem"
"Mobile Apps Development"
"Digital marketing proposal- www.associatedtechs.com"
It seems entirely likely that Google is weighting whether something comes from a private mail server very heavily, and Linus, being who he is, gets a lot of email from private servers.
Not saying this isn't a problem, but it probably doesn't affect 99% of email users.
So, with a conservative one billion email users, only 10 million users see this problem.
Honestly, if Google took support even half-seriously (or: they considered eg: users of gmail their customers, rather than just their advertisers) -- these kind of issues wouldn't be so bad.
I do think it's just a question of time before Google relegates itself to irrelevance through a strictly inferior product though.
I find about 2 a week that have been marked as Spam and shouldn't have been. Your experience is not everyones (nor is mine).
This helped us:
I have also been having any messages that are replies to messages I send being diverted to my spam folder randomly. It is happening on both my personal gmail account and on a business account.
It seems lately Google has really been less focused on the core components that made them successful in the first place. I have found their search results seem to be returning more spam sites than before. My vote is to get Matt Cutts to come back and start cleaning up spam again :)
Unfortunately that's not the definition of spam.
However I have the same problems Linus has so I am not excusing gmail one bit in this.
I've seen that happen too. And only recently.
Yahoo's spam filter also does this.
It was probably something along the lines of "Well consumers are what matters and they all use major services".
Assuming non-gmail servers are spammers is a pretty epic fail.
(There's been improvement - for example, recently I received mail from someone I'd corresponded with in the past, and it wasn't initially marked as spam. Gmail used to be more aggressive than that, such that it would be marked as spam unless it was a direct reply to an email I had sent.)
(...for extra irony, that recent message was "I'm stuck in England and can't get home without a few thousand dollars". Her account had been hacked.)
I wondr what other forms of "gibberish" Gmail classifys as spam? GPG encrypted mail? Mail containing public ssl keys or CSRs? ANy foreign language not regulalrly hard in Bro-ville, South Bay?
I saw his original post on G+, and of course immediately went and checked my gmail spam folder.... and... no false positives at all, 100% correctly identified spam.
There's not really a great solution to that at the moment - either you technically violate RFCs by having your your discussion group software modify some headers, or you deal with other kinds of breakage.
Doing header rewrites is effective for reducing FPs due to DMARC, but adoption is far from universal - off the top of my head I'm not even sure if Mailman supports that at the moment.
Here's an example: you have an address @google.com, which has a DMARC policy of 'quarantine'. You send a message from this address to a discussion group, which in the process, resends your message from a non-google server, thus failing DMARC.
Google's DMARC policy says that if an ISP receives a message from a @google.com From address and the message fails DMARC, that ISP should place the message in the spam folder.
So it boils down to: does a list operator change the From address in distribution group mail to use a list address they own in order to pass DMARC, or do you deal with the filtering consequences of failing DMARC for many domains?
Here's an example from a Google email engineer's recent post to the Mailop list, which is running Mailman software.
spf=neutral (google.com: 2001:41c8:51:83:feff:ff:fe00:a0b is neither permitted nor denied by best guess record for domain of email@example.com) firstname.lastname@example.org;
dkim=neutral (body hash did not verify) email@example.com;
dmarc=fail (p=REJECT dis=NONE) header.from=google.com
Funny thing that happened to me, a mail from Google was marked as Spam. This was a long time ago, and it was from a mailing list, but apart from that, it was a legitimate mail.
DMARC allows a domain to say "email that claims to come from my domain must be signed by me. If it doesn't, burn it with fire, no exceptions". So Gmail is only following the instructions laid out by the sending domain.
This is helping to make the email ecosystem a lot more robust by ending the problem of From forging. Ordinary users rarely realise that the From header is otherwise meaningless so phishing them can be very easy.
However it does not play well with mailman's default settings, and a lot of mailing list admins refuse to help the email ecosystem become more secure (whilst often PGP signing their own mails, doh). So DMARC creates a lot of noise in the technical community from people who have to/want to use mailman based lists.
I would venture a guess that many or most of them are coming from similar people, or at very least similar domains, or even the same mail server.
this email is sent by google when logging in google account from a new machine. they tag their own email as spam ...
Your Google Account firstname.lastname@example.org was just used to sign in from Chrome on Windows.
Don't recognize this activity?
Review your recently used devices now.
Why are we sending this? We take security very seriously and we want to keep you in the loop on important actions in your account.
We were unable to determine whether you have used this browser or device with your account before. This can happen when you sign in for the first time on a new computer, phone or browser, when you use your browser's incognito or private browsing mode or clear your cookies, or when somebody else is accessing your account.
The Google Accounts team
"Your honor, clearly you can see that our services are so impartial as to flag our own email..."
Something tells me that probably wouldn't work.
If I were on the jury, I'd definitely buy it. Their counsel would have to really bone up the argument that it was such equal treatment, they were flagging their own messages. And the prosecution would have to really stretch things, probably entering into conspiratorial territory in order to make a case.
Then again, I've found myself perplexed by jury decisions on tech-related cases more than once. Although having sat on a jury, I can see how such decisions might be made.
I realize that everything takes time to implement, and developer time is not infinite, but this one seems like pretty low-hanging fruit.
Good job, Google!!
those emails were sent from no-reply@accounts.I.google.com
Why is this message in Spam? It has a from address in accounts.l.google.com but has failed accounts.l.google.com's required tests for authentication.
what are required tests for authentication?
Microsoft could own business email again just by answering the phone.
It's likely that he'll actually catch a Googler's attention, but for many of us, user feedback is not an option.
@jacquesm's http://jacquesmattheij.com/ham-or-spam-gmail-not-to-be-trust... is another recent instance — but again, there's no call to action.
Gmail is great for some people, but I prefer having more control, and I highly recommend https://FastMail.com if Gmail is failing to meet your needs.
Source: I am an LF employee.
how do you got a work there?
I'm starting to think that Notspam signals have no effect at an individual level. Either that or the button is simply a placebo.
Fortunately, the false positives for personal correspondence from individuals are still extremely rare, at least for me.
I find this trend of "follow the majority" quite disturbing - it's as if they're implicitly saying that everyone should think the same way and punishing those who don't follow. What's spam to me may not be spam to you, and vice-versa.
Then again, having a personalised spam filter for each user would probably consume a huge amount of resources...
As mentioned further up, some scoring works well for many users, but not for all, such as marking eg: Russian/Chinese/Not-spoken-here-by-most language as spam.
I really see no reason for why Google should be so bad at classifying email as they apparently are.
There's a spectrum, and if a given sender looks considerably worse than average, they're more likely to get filtered.
If anything, if a newsletter is getting filtered, it's more likely to be the marketing manager's fault - perhaps they don't adequately monitor deliverability, or they don't test their content, or they don't use activity segmentation... etc.
That's what I do for the Lua Mailing list.
If I've subscribed to a mailing list or newsletter and there isn't a one-click unsubscribe I'll click the Spam button to get it out of my inbox instead of going through their procedure.
One-Click Unsubscribe is paramount for mailing lists and newsletters not getting marked as Spam.
This may be an anxiety issue on my part. Fair enough. I will generally continue to do it.
My address is my first name + last initial (neither of which are all that uncommon), and this is made much worse by Gmails idiotic ignoring of periods in addresses. There is a dude in Denver, CO who is absolutely convinced his e-mail address is email@example.com. It isn't. I'm really sick of getting his AT&T and car insurance e-mails.
I wish there was a better way to deal with this type of situation other than constantly sending "please fix your address book" emails. Email is a broken system.
Gmail.com is certainly broken in the sense that they want to cram 10 billion users into a single domain. It's ridiculous marketing/brand-motivated UX failure.
Since forever most mail services had a few vanity-domains, so people could get firstname.lastname@example.org. But no, Google doesn't want to provide email, they want to provide "Google Mail".
Apologies for the rant, but I can't stand it when big companies create problems through stupidity.
I meant that email is broken in the sense that when some stranger mistakenly thinks that your email belongs to them, and continues to give it out or sign it up for mailing lists, you have absolutely no recourse. If you have an email address that like mine is easily mistaken for other ones you get incorrectly-addressed personal emails many times a day. There is no way to find the actual intended recipient or get in contact with that person to say "hey you seem to be confusd, stop using my email address". And there is no really good way to filter those emails, since after all they are coming to your correct address.
I think this is the kind of problem that's difficult to appreciate unless it happens to you frequently.
The problem with email is that anyone can email you if they have your address.. thats why we have so much spam. I dont know what the solution is, but it would be much better if the recipient had to opt-in to the conversation somehow as well.
I believe he means that if email@example.com was already taken, Google should offer firstname.lastname@example.org rather than email@example.com. In fact they shouldn't even show that as an option.
Since Google Is now a domain registrar they could create the new domains on the fly.
Then there wouldn't be namespace collisions.
As for there being "no recourse" -- apart from spam, that's just wrong. It's much faster to reply with a "This is not your Smith"-mail, than it is to write a "return to sender" on an envelope. Same thing for getting phone calls from a different timezone etc.
[ed: I do agree that it's a bit more difficult with people that don't know their own address -- still think it should be quicker to reach their contacts via email than via comparable means.]
Right, and I've sent literally hundreds of those emails. They almost never do any good, because while one person may fix your address in their contacts list, the original person who gave out the faulty address is still out there, unaware that they're giving out bad info. I always ask if the email sender can tell the intended recipient about this when they figure out the right address, but that rarely works. Anyway, I know this is a very specific problem that only affects a small fraction of people, but it's extremely annoying.
I dont really see how allowing other domains would help.. that just shifts the issue to the domain string instead of the user string. I guess it gives people more options. But one of the main benefits of gmail addresses is that it's so common. Everybody knows it, so nobody ever misspells the 'gmail' part at least.
It is annoying however especially since most of the spam in my spam folder is addressed to her through my email address.
TBH I hardly use personal e-mail these days, it's basically a bucket that receipts and confirmation gets dumped in to, in which case search works well enough. Most actual conversation is done via Facebook or IM, etc.
I think it's because I'm beginning to resent the idea of every website and its dog requiring that I have a user account before I'm allowed to even browse the content.
I know I get a fair amount of unsolicited marketing list messages that do have an Unsubscribe link, which I click, but I also mark as spam because I never subscribed to it in the first place (of course, I'm also not using Google Mail, I'm using FastMail with a personalized SpamAssassin filter, but I assume it will still influence the global default SpamAssassin filter).
Yup: Gmail is marking comments originating from Google+ and written by legitimate users as spam.
Even better would be if you could have different handling for different levels, like black-holing or auto-trashing the absolutely-definitely spam, making it easier to occasionally scan the regular spam box. I get something over 1000 spam emails per day, so it's just not feasible to give even a cursory look over them to find the false positives. I can't even imagine what it would be like for someone like Linus.
Unfortunately, that would draw attention to the fact that the spam filter isn't perfect, and would require users to make choices with tradeoffs, so I can't imagine it's a very attractive option for Google.
It seems to very easily flag discussion about .exes as spam, it's really disappointing. It's been several years and the filters haven't improved, despite me religiously flagging spam/not spam in those lists.
In the end I just gave up and set up filters to specifically prevent marking incoming emails on those lists as spam. It misses the odd linkedin invitation, but it's not like it was catching it before...
This is why I never mark mailing list email as spam, even if it is in fact spam.
If so, maybe gmail thinks "this sender is spammy". That's the mailing list.
Notice the comments from Sri Somanchi. He's listed as the product manager here: http://gmailblog.blogspot.com/2015/07/the-mail-you-want-not-...
1. Everything became "spam"
2. They got to a point where they believed they were the standard
3. Nobody could do anything about it
I can think of many places where this same situation has played out. Its yet to work long term without disastrous results after a reign of technical darkness. That doesn't seem to stop people from thinking it won't happen to them.
Its fine to aggressively fight spam. If you choose to error on the side of false positives then its in your own interest to provide reasonable recourse. If not, you've left a very large gap that somebody else will come in and fill. Just as google did.
From: [...] Baby <[...]firstname.lastname@example.org>
Subject: HELLO HANDSOME
Body: HOW ARE YOU DOING
What's even worse they rejected email to postmaster@. I know you can adjust the spam filter sensitivity somewhere in Google Apps but come on, you should not reject any mail to postmaster by default.
Luckily it's pretty easy to scan the folder for valuable messages.
However, having to do that is clearly not ideal.
Had a wedding RSVP get flagged as spam.
Of course gmail deletes spam more than 30 days old, so how does it happen than an obscure educational non-profit gets over 4000 spam messages a month? Gmail must be a huge spam magnet, but still a mystery how those messages find their way into this spam bucket. (Unless in the past somebody had abused the account and the email address is on a thousand spammers lists...)
In any case hard to be certain what criteria the spam filter uses to declare a piece to be "spam". Not all the misclassified emails were sent from "private" servers, it would be useful if it was more clearly specified.
I had legitimate emails bouncing because the mailing list had put me over the maximum number of emails that a free Google account can recieve in a day. I didn't even know there was such a limit until I hit it.
I now run my own mail server and have none of these or the other problems outlined here.
The mail simply does not reach my suppliers'/partners' inbox, and as a result we're all losing time and patience with this
The really funny thing? Some of those people I work with are @google.com.
(And yeah, my corporate domain is clean, SPF'd up to the wazoo, etc.)
I wonder what Google's internal metrics show...
When I get a second email from a recruiter that previously has been marked as unsubscribed then I click that 'mark as spam' button.
I've been doing this for eight years, since I'm quite happy with my work situation and have actively tried to remove myself from their databases over that time. However my CV still seems to be floating around, even though I've also deleted it from every online job portal I was signed up to.
At some point most recruiters need to feel that pain, because they just don't listen.
As they should be, because they are. They have terrible practices and I have never met one I would want to do business with.
The false positive rate is defined as (the number of false positives over (false positives + true negatives)). However, Linus quite obviously calculated the 20% as (false positives over positives), i.e. (200 / 1000). If Linus happened to have 200,000 true negatives (i.e., non-spam messages that were not flagged as spam), a number that I'm making up because he did not disclose one, then his false positive rate would be (200 / 200,200) ~= 0.1%.
Think about it... whether his email address was harvested by all the spambots in the world or none at all would have no effect on the fact that out of 200,200 legit messages, 200 were incorrectly flagged. This is why the false positive formula doesn't even include true positives! The 800 true positives (actual spam messages) don't matter to the formula. Therefore, neither does the 1000 total (true+false) positives. Don't divide by it.
I've enabled every possible thing to make it work (DKIM, SPF, who the hell knows) and mails forwarded to my members with gmail (or Google domain hosted) accounts are always getting the forwarded email in their spam folders.
I really wonder how these preferences are reflected if someone hasn't been using spam filters much. I get maybe at most a dozen spam emails escaped a year, and the rest of the emails go through (all mailing lists... I am on a lot of mailing lists) so I get about 50+ emails per day from just DL. So I barely ever need to mark something as spam, or even more something to another folder, so how does Google know what's my preference? It sounds like people who don't actively mark spams are less likely affected...
At least they're nowhere near as bad as Outlook. I have one of my domains on their free Live Domains (grandfathered plan, can't get it for free anymore, similar to google apps) - and 90% of my emails end up in spam, even if they're from a reputable company with sane mail setups such as Digital Ocean, Github or even Google.
To make it worse, with Outlook you can't turn off the spam filter, and it's known that Microsoft sometimes SILENTLY drops emails for various reasons so they never even make it to your spam box...
Sadly I've yet to find any decent replacement mail service for my domains that's free (or very cheap) and of decent quality.
A Googler on HN was kind enough to get in touch with me and help bring the problem to the attention of the right people at Google, who fixed things on their end and made a few suggestions on my end. Presumably, Linus Torvalds will get about 1000x as much Google love as I did.
Good idea to check often though: I just discovered a few LiberWriter customers in my own spam filter... :-/
Speculation of course but it does point out the problem with Google becoming too big.
I still run Outlook 2010 with Paul Graham's SpamBayes plugin. It's far from perfect -- a whitelist would sure be nice, and how hard can it be to automatically refrain from marking anything as spam if I've emailed that sender before? -- but the fact that I can train it myself and adjust its classification parameters covers a multitude of sins.
I've resisted a lot of peer pressure to use server-side email over the years, both commercial and FOSS. I fully expect to be using SpamBayes in 10 years, probably with a wheezing, clunking copy of Outlook in a VM.
There seems to be a fair bit of spam sent to the LKML, I don't know whether there's been more lately, but perhaps the large amount of email sent to many people for the LKML and the fact there's a decent amount of real spam sent there, combined with a more aggressive setting is an explanation?
The lesson I learned is we still need to review the spam box once every day or two.
I also can't stand when I get added to a repo and start getting messages from every single notification.
If anyone is interested they can read it here: https://www.linkedin.com/pulse/20141202174235-36852258--spam...
This is an individual who, in response to broken version control systems wrote his own version control system which he uses, primarily, to control versions of the operating system kernel that he wrote.
How is it possible that the above described person is moving his wrist around going clickity-click with a mouse just to read an email ?
With alpine over ssh I can use email on a bad 2G cellular connection with no hassle at all while google won't even load the inbox.
Gmail is not my primary email, but still I do get important emails.
In our company , ( we are a hosting email solution ) lot of time and human resources are spent to monitor and work with the work produced by the spam filters. I guess it is too soon for this kind of solution to be sostituite with AI.