It's big enough that when someone complains that a message sent wasn't received, the intended recipient will say, "I never have problems with my Gmail account. It must be you." And the sender has to switch to Gmail to reliably communicate with the outside world.
I wish this was just paranoia, but we've seen multiple discussions on HN about Google programs and policies that alter the internet in ways that only benefit Big G. It's like we're heading back to the days when people didn't know the difference between AOL and "the internet."
It's simply not true we have no incentive to fix this. Here are a few:
Firstly, Gmail's success is entirely predicated on the health of the global email ecosystem. Gmail does not, inherently, have any network effects (unlike FB, messengers, any other comms tool of Gmail's scale). Email itself, of course has a huge network effect, and that is because you can email anyone in the world, regardless of what email system they use. It's because email is open. If we lose an open, healthy ecosystem with many providers, we'll destroy the base we stand on.
Secondly, we care deeply about having positive relationships with developers and all our users. I can tell you it definitely makes me sad to see articles like this. There are going to be false positives, we will make mistakes, but we certainly care a lot about fixing issues like this when we hear about them.
I agree Postmaster tools has been underinvested in and we could do much better there.
SPF, DKIM, reverse DNS, no blacklists, no open relay, longtime ownership of IPs, etc etc. Using various mail testers returns a 10/10 deliverability score.
And yet, messages sent to Gmail always go into the spam folder, or are never delivered at all. These are everyday regular messages, I have never used mailing lists or sent bulk automated messages.
The issue is, there is no recourse, no fix, no acknowledgement of the problem with false positives. There is no tool available to me to understand or correct the "problem". Hint: this comes across as Gmail not giving a shit.
Gmail has a responsibility to be more accountable, even if these problems are unintentional, because Gmail is such an enormous node in a federated network.
> If we lose an open, healthy ecosystem with many providers, we'll destroy the base we stand on.
Correct. Gmail is contributing to the erosion of email reliability. Please course correct.
- Legitimate class action notices related to Amazon purchases.
- Email coming from addresses to which I had already sent email. (!)
- Email from my landlord.
- Email coming from Google itself.
Based on the contents of my spam folder, which I have to check fairly often because of the extreme overaggressiveness, I would be vastly better off if nothing ever got filtered at all. 
>> There are going to be false positives, we will make mistakes, but we certainly care a lot about fixing issues like this when we hear about them.
This doesn't sound honest, or at least not complete. People have been complaining about this for years. I have personally been complaining about this for years. The loss of obviously legitimate email is completely outrageous.
It doesn't look intentional (look at that fourth category!), but it certainly doesn't look like anyone is trying to address the problem.
 Yes, if spam filtering was disabled, more spam might get sent.
That didn’t stop me from repeatedly getting junk from every other record label my email was sold to. It was an endless procession of shit I never subscribed to.
When either provider decides your small email server is sending spam (eg: sending an email with an attachment, or any kind of form email like a daily report) you won't get through to user inboxes, and instead you'll be routed to spam, or for Outlook.com hosted addresses they will accept mail from your server and send it to /dev/null. Gmail's process is bad, but Microsoft has decided to accept emails and throw them away (which is ridiculous).
I never had a problem with gmail for business regarding spam. I regularly receive mails from smaller businesses (some of them hosting their own mail server) and never had a complaint from anyone yet. Since i can also be contacted via phone i'd know.
On the other hand, MS Exchange constantly delivers obvious spam mails and (quite seldomly, but still) swallows legitimate mail.
Anectodal, i know. And disclaimer: the behavior depicted in the article is as bad as it gets, if everything is as described.
Yep. I missed an invitation to a Google-hosted event at a conference I attended because the email (from an @google.com address, no less) got caught in Gmail's spam filter.
(afaik in denver water boils at 95C)
50 of them blast twitter and it seems the world is collapsing.
The world is resilient.
That is assuming by error you mean missing the vein. If error is defined as a fatal complication, then 1/1000 is terrifying.
Google does what's best for Google - how could there be any discussion of that fact in 2019.
If there are 5 million people and 50,000 experience problems, that's fine?
Isaac Asimov's comments about world population increase involved something about this; the more people there are, the more each individual is dehumanised and rendered irrelevant (my paraphrasing).
When all rounded up it isn't even a single penny on the balance sheet. The owners of these businesses literally never even know from the their only view into the companies.
I have no idea why I'm being downvoted on this. Hackers can't do math or what?
On the other hand, once you train it a bit, it is mostly remarkable good. For me, switching from fastmail.fm (which was pretty good itself) to Gmail gave me a big improvement in spam control.
Shame on AT&T for not validating their customer's email address.
I too get the same type of spam from AT&T.
The only way Google would have to identify that this message was not for you would be to get the subscriber information from AT&T and cross-reference it with name and address information they had for you - and even then most of the time they'd probably be wrong (e.g. if the email is coming to you but the account is actually in a family member's name).
It doesn't make any sense since they are emails from Google, they are emails I even have a filter applied to so that a label is applied to them. Yes I can adjust the filter and choose "never send to spam" but the messages will still show a warning on them saying "This message was not sent to spam because of a filter you have applied".
Sure false positives makes sense, but I don't get how the majority of what is in my spam folder would be emails sent by Google.
It makes a lot of sense... people use the spam button as a lazy man's unsubscribe. Youtube adding the bell button, making mail opt in is probably a response to that.
I actually give Google a lot of cred for not simply white-listing its own domains. Though spammers would probably find ways to abuse it and make them look bad anyway.
This is the small mail server crux right here. If you’re a small mail server and a few of your emails have been spam binned instead of unsubscribed, it would likely lead to your whole server getting shit canned.
* Available for decades.
* Far from being top notch technology.
* Sometimes of course it's literally mean outdated. Like if you run older CentOS or Debian with decade-old packages.
So it's doesn't mean SpamAssassin is bad, but it's very far from state-of-the-art ML technologies that Google might have.
You'd think the various governments would put more effort into computer security. They appear not to care, though.
Edit: Gah. Now I checked my work Gmail spam folder. There was an email from one of my users there. (and nothing else)
Now I'm considering migrating away from Gmail, at least for work related things.
If email coming from google itself got special treatment there would be masses with pitchforks complaining about that.
You've made a lot of good points, but I don't think that's one of them.
It feels like Google no longer has any incentive to follow the rules, and they feel that they are going to be the ones to make the new rules. The rest of us end up having to implement workarounds.
I disagree. I think in the absence of that point, it would have been hard to say this:
> It doesn't look intentional (look at that fourth category!)
But also, I think special treatment for trusted actors is a completely appropriate way to handle email delivery, and I also think it's appropriate for gmail to trust themselves to be sending legitimate mail. Blocking their own email makes them look totally incompetent. They absolutely should whitelist themselves. And they should have a way for you to be whitelisted too, if you want to send email.
That is understandable. It is hard to validate if an email is authentic. SMTP has no authentication built in. Gmail can't just blindly accept all emails from addresses that you have already sent an email to.
Look at this example:
Anyone can connect to relay.example.com and pretend to be firstname.lastname@example.org.
All those "I hacked your email and send you a message from you account" I don't get, because I have a DMARC policy that says if you don't pass SPF/DKIM then you get rejected. So try as the spammer might to connect to my mailserver and pretend to be me, they can't, because my mailserver sees they're not authenticated, and the mailserver they're sending from isn't in my SPF records, isn't signing the message with my DKIM key and therefore it gets rejected at the SMTP level.
The article discusses mail not being accepted by google/gmail in the first place.
Messages they think you won't want to receive are what the spam folder is for.
The amount of spam that would be delivered if they didn't discriminate AT ALL is enormous.
They have to read all this feedback and discriminate better.
Yes there is. They don't want to carry traffic from anybody from the major email blacklists. If a mail server is on a real, very transparently-managed blacklist, no large provider should be accepting their smtp traffic.
To be fair, sometimes Google does send spam.
You'd think Google had an incentive to get that delivered.
But it still beats no filtering.
There are layers of filtering beyond what appears in your spam folder, layers that block obvious spam long before it gets anywhere near your account. If every email ever sent to your address wound up in your spam folder you'd beg for filtering.
Even after running it for years, Spamassassin never marked a legitimate mail as spam, so i'm pretty sure that if i wasn't too lazy to configure it to move it to a spam folder, it'd work fine. Stuff did pass through it (at a ratio of one every four or so) but i was fine with deleting those.
What i'm trying to say is that from personal experience, i'd be fine with a spam filter that errs on the side of not marking stuff for spam and me deleting whatever goes through manually. Having to see a bit of spam mail is small cost for losing mail i'm actually interested in.
Thing is, most of my messages get delivered. Then suddenly they don't.
Google "giving a shit" means responding to hostmasters about delivery problems, and they just don't.
Please note this, Gmail PM.
I changed jobs as well, and now work is using a MSFT based hosted mail service, and I am getting delay messages.
Seriously, GOOG, MSFT, and others broke mail. This is not an improvement.
I've not looked into speaking with MSFT mail folks about their breakage yet. With GOOG, you have really no mechanism of reaching out to someone there and getting attention for the problem they are causing.
This is the much bigger problem with GOOG actually, in case any googley people are reading this. They just don't get customer service. At all. It is near impossible to be able to report a real problem across the spectrum of their services. Unless you are one of their bigger customers, you don't have access to even a telephone support number. Their online help is a crapshoot, with you getting useful information less than 50% of the time.
So where I am now is with locked down, long time existing domain mail servers, which send maybe 5-10 outbound messages per month, that suddenly and inexplicably, have a bad reputation. Well, no they don't have a bad reputation, they can send email just fine to other services.
It is my belief that this is intentional and I would love to be corrected if it’s not.
Large providers tended to have a world view dividing all senders into two categories:
1) Bulk senders who are clearly mailing the same spam to a list of a billion addresses
2) Non-commercial individual hosts which should be sending five messages a day or less in total.
It felt like there was a huge missing third category for transactional emailers nobody wanted to acknowledge. They are probably difficult to score fairly. A hundred "Order details" emails are going to have the same level of randomness/templatedness as the old Viagra spam which had a random block of Project Gutenberg text pasted at the end to trip up filter math. You're not going to have a clear history of "this address bounced twice, let's stop sending newsletters" when most of your messages are to first time customers or once-every-few-years return ones. A lot of the messages will look generic because they use default shopping cart templates.
To the extent they provided sender guidance, it was focused around use case 1) -- sign up for feedback loops and deal with greylisting (because people really love waiting 18 hours for an acknowledgement)
Anyway, Gmail is getting some heat in this thread and rightfully so. We should however not forget that Microsoft and Yahoo are just as bad if not worse in this respect.
I had set up some commercial accounts on Zoho (my own e-mail is on Zoho) but they asked me to migrate to Google, even though it was more expensive.
I no longer use Gmail myself but half of my users relay some aliases to their main Gmail account. No problems with that, except my servers continuously get rate limited by Google:
Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 rate limited. Please visit 421-4.7.0 https://support.google.com/mail/?p=UnsolicitedRateLimitError to 421 4.7.0 review our Bulk Email Senders Guidelines.
This leads to family phoning me all the time to say my server is broken when it is just Google throttling everyone's emails. I frequently have to check Postfix queues and clear some obvious spam or just pointless Facebook update emails which seems to be the majority... (Yes your email admin can read your email...)
The only way to improve is to constantly remind my family to not sign up to all crap, use not so common aliases, and try to keep tightening my anti-spam configurations. But we really are not talking about a lot of emails. Probably less than a hundred per day spread across 5-ish end accounts of which 95% is probably legit. Yet Google is treating me as some totally open relay. (╯°□°)╯︵ ┻━┻
But it does feel like they are treating nearly all minor relays as spam relays. We can not block 100% of spam before we relay onwards to Gmail as that would mean too many false positives emails get blocked, but most try to block as much as possible.
In my case, my servers probably block 99% of spam, but some will get relayed, and most of those ends up being handled by Gmail's even better Bayes scoring and filtered to end users' spam folders.
I don't know what the automatic threshold limit is to be on their naughty list, but it must be very low, as in double digits per day.
It is not to be set up correctly.
Now in your case, you failed to mention DMARC records. ALL of the big cloud email services have required that for years. Your SMTP server checklist is straight out of 2005.
This entire thread reminds me of the guy who pulled out of a desk drawer an Analog Startac cell phone and blamed AT&T for it not working. Email has moved on. It not Google's fault that people are configuring SMTP servers like it 2005.
> error SPF must allow Google servers to send mail on behalf of your domain.
So you must let Google spoof your domain? That seems crazy.
ProtonMail has the same error:
For domains that are not using Google Apps, its results are meaningless.
I know because I haven't set up DMARC for my email domain, and my email still gets accepted by everyone.
I believe it helps that I accept on TLS with a valid certificate, and I used Google's postmaster site verification.
I _think_ this is because G doesn't want people gaming the system.
> or correct the "problem".
I regularly fish mail from my spam bin. From mailing lists, and other important stuff -- and indeed Google's own mail!
One thing you can do is to get people to add you to their contacts list.
It's a hard problem. There is one "solution" you probably do not want. Have Google (and other companies) give their imprimatur to certain mail senders.
9 out of 10 threads marked as spam were false negatives.
That's a 90% failure rate.
Since money was involved, I paid attention and followed up
Our finance person found many emails in Spam that were important and should not be there
On the other hand, my personal Spam folder is certainly full of crap I never want to see. But now I don’t trust the system, so I have to scan them anyway
I just happen to have set up an email server and encountered the same problems with Google as described in the article. I own the IP since quite some time, it is not on any black list, reverse DNS is set up etc. but Google rejects email as spam.
And this even happens when the gmail account has added the sender in his address book and has send the first email to which I replied - thus there is a message id that should already be known on Gmail's side.
Use your AI to put email into the spam folder. Refusing it outright is a case for the European Commission which hopefully will slap you another few billions of fine onto the wrist until you remember to play nicely with the other kids.
There is no excuse to refuse SPF, DKIM, reverse DNS, proper MX, no blacklist, sender in recipient address book and reply-to msgId email.
I wish Google had a way you could tell it "messages from these people/domains are never spam (or I'll deal with it myself)".
I use gmail and they know who is sending. I got an email today with a gmail flag saying “this user sent from a different email address previously”
I doubt this affects things when messages are rejected at the protocol level, but if your problem is emails showing up in the spam folder, this might fix it.
What Google is likely doing is checking the domain of the originating IP in the SMTP "envelope", but that also gets tricky with outsourced email services or internal IPs.
I do not use Gmail, so these are just wild guesses, but I do run my own mail server and frequently get my email not showing up for people.
Problem with SMTP bounces is that it may take a week for the final bounce to show up in my inbox (because again, that's how SMTP protocol is designed, to expect nodes to be down and retry a number of times).
Isn’t this the exact thing that DKIM is designed to fix?
So yes, DKIM will help a receiving server know for sure, but a receiving server still needs to accept emails from non-DKIM-enabled servers, and perhaps that's why whitelisting didn't work for the parent.
I prefer getting a bounce than ending in the spam folder by far. Lots of relatives on Gmail never look into the spam folder so I never know if they get to see my email or not and I end up having to ping them on eg. WhatsApp.
At least with a bounce, I know they did not get my email.
If the message is ending up in your spam folder, it's harder to shift that blame off of Gmail's shoulders.
My mother would just simply ignore the spam and often times spam catches majority of the phishing e-mails too. So it's a double edge sword educating and conditioning the users to review the spam folder. Why is it the end user's job to determine what is spam and what is legitimate ?
Who else could determine that? I could hire you to send me emails about Viagra sales. They would not be unsolicited, because I specifically asked you to. They wouldn't be commercials, either, so they wouldn't be Spam. An automatic filter can't determine if it's spam or not, it can only take an educated guess.
Yes, I'm yet another person with the same problem as the author of the article - running my own mail server for years, only I send mail from it, very low volume, everything set up properly, etc. The difference is that my emails go to spam, rather than being rejected by the SMTP servers, which to me is even worse, since I never know whether an email I sent to GMail has been delivered or not. Since GSuite is so popular now, I never know this for any unfamiliar domain, unless I do an MX lookup.
You say all the right words, but Google's actions on this issue (or lack thereof) speak much louder. It's very difficult to believe that the situation will improve and many comments here reflect this skepticism.
I think it also hurts Google themselves. There is a reason they have trouble competing with AWS. Why should developers trust Google that there will be a person on the other side helping them out when things go wrong? And they always do, at worst possible time. Amazon, for all its problems, has excellent customer service.
So your parent's attempt to fix this is a PR stunt at best. If they (/Google) care about this, they should fix the problems in the process:
- define, hardcode and publish rules that will lead to successfull e-mail delivery (SPF, DKIM, history,...)
- establish a gmail technical customer support service
And they should stop with preferential treatment of those who shame them publicly. Or do they want all of us to start doing the same?
That's the only way companies care in the 21st century, 3 weeks on an email thread, one tweet and suddenly everyone cares.
An increase in public shamings probably means that public shamings become less news- or interest-worthy. So an increase in shamings might not have as much effect as you might expect.
(FWIW, I don’t seem to have this problem in a significant way, but that might be because I have DMARC set up.)
A big part of the problem with Google taking everything over is that only in very limited situations -- the service formerly known as YouTube Red, for instance -- are we given the ability to actually conduct business with them as a paying customer. Conventionally, Google users don't even rise to the status of sharecroppers, since we're the "crops" being sold to advertisers. They expect us to depend on them for the everyday conduct of our personal lives and careers, yet the only way to appeal for help is to start a shitstorm on Twitter or HN and hope somebody notices.
Well, it's a hackneyed worn-out cliché. Without the users there's no viable product. That's what makes them the customers and that's why they need adequate customer support. The fact that you pay for the product with ad impressions rather than dollars doesn't change anything about that dynamic.
No, that's what makes them "users." The customers are the advertisers who actually pay money to Google. You can rest assured that they don't have to post a cri de coeur on social media to get a response from Google when something goes wrong.
IMO, a company that does its level best to act like vital public infrastructure needs to be held to standards appropriate to vital public infrastructure. If that's a controversial point of view, then so be it.
I understand your idea. But in my opinion this is a useless and unconventional definition of the word 'customer'. The customer is the one who receives a service in exchange for compensation. In this case the service is gmail and the compensation is ad impressions. The fact that Google can sell those ad impressions is secondary to that dynamic. If there were no users willing to provide that compensation in exchange for access to gmail, then there'd be no product, and there'd be no secondary market for selling those advertisements. Meanwhile even with no advertisers there would still be a product, and there would still be other avenues for monetizing it.
> You can rest assured that they don't have to post a cri de coeur on social media to get a response from Google when something goes wrong.
Are you sure about that? A quick google for "adwords support experiences" gives quite a number of telling stories to the contrary. Besides, there are several orders of magnitude more users than there are advertisers, so obviously it's going to be much easier for users' issues to get lost in the noise. And obviously no advertising agencies are going to be getting any sympathy by complaining on Twitter.
This has existed for years:
If your site passes and is not on a blacklist, your email gets delivered.
It is not Google support, it is just an employee giving some insight on his own time.
What is wrong with that?
This is a very interesting idea that I completely missed.
Oh really? Then what is the mechanism by which an affected user can report an issue like this and receive individual, accountable response? Because that is what a company that cares about fixing issues does.
As far as I can tell, the only redress available for any issue like this with Google is 1) be a big enough name like Jamie Zawinksi (he has been posting about struggles with this exact issue for several months) that your blog post gets attention, or 2) hope your blog post makes the front page on a site like HN.
They are largely unaccountable and there's no real way to get a hold of a human being unless you shame them on social media and it gets enough traction.
The difficulty in this case is the people operating SMTP servers are not paying customers, the free Gmail users are not paying customers and the GSuite paying customers don't think it's their problem (after all, the spam filters works quite well these days and lots of people are on Gmail, making the problem very small in the receiving end of this situation).
It seems an external force will have to push Google in the right direction (of having systems in place to deal with the exceptions). If that's going to be public outcry, legislation, etc... We will see.
On the automation front, it seems Gmail could do a better job at tracking sender reputation over time (i.e. you haven't send spam in a while, we'll be more lenient with our spam rules).
Based on the number of hacker news posts from people who pay them money and are locked out of their accounts by automated systems with no recourse I'd say that's utterly false.
Maybe you meant to say "if you pay them enough money" which is true. But most people don't have that much money to pay them.
Have you had direct experience with Google support? My experience has been pretty regular but maybe I'm an outlier.
The process took a while (about a week) but we did get the account back.
Things have changed a little bit since Google just presented a wall for support requests to bounce off.
Doubt. We're paying ~$10k/month on ads. The Ads API is rate limited, but it's either broken or has arcane logic, so about every 4-6 weeks, the usual workload will trigger it and we're shut out for 12-24 hours. Can talk to ad support whenever I want, but all they are good for is optimizing campaigns and explaining the UI. No option to get to technical support, no option for ads support to escalate.
Of course, we might not be a large enough customer, but it certainly isn't about "paying".
It has become a bit of a rite of passage when somebody new arrives, they see the error ticket for the first time and are eager to dig into it and solve a long standing problem. It's like a trust fall, only the lesson is not to trust, and you learn that lesson by falling.
I may not be giving them money directly, but my use of their products contributes to their billions in ad revenue. Without us free users providing eyeballs for their ad network they would be significantly less rich.
It's upsetting that the default response to complaints about Google's terrible customer service is "if you don't pay them you aren't a customer." Most of their value comes from us, it most certainly isn't a one-way relationship.
It feels like Google hopes that you just give up on self-hosting your own email and simply move over to Gsuite. It certainly feels that way. Especially when it feels like Google/Gmail/Gsuite are commonly recommended nowadays.
I'm inclined to think Gmail's approach to this problem is fundamentally flawed, because Gmail has been by far the worst at binning my server's emails. I don't have this problem with Hotmail, AOL, Yahoo, etc., etc. Now, feel free to argue that their approaches are too lenient, but if I have to choose between false negatives and false positives in my spam filter, I absolutely know which one I'd choose.
This is wrong. The current situation is "if you want to reliably contact gmail users you need a gmail address". This is no different than Facebook or messengers.
And that may actually be the incentive for Gmail being such a bad player compared to everybody else.
I assure you there is a lot of email coming from Outlook/Exchange.
I suspect that is not super relevant to this conversation, because there are a lot of signals that Google can use to see that we’re a big fish, and therefore automatically tread more lightly on the SMTP bouncing. We’re running Exchange, for one thing, which is not cheap or easy to use. We’re in the IP address of a big enterprise ISP. We have other enterprise services running on adjacent IPs, like OWA and websites.
Small personal email servers have none of those sort of “ambient” signals; they’re probably running open source email server software on a single IP coming from a consumer ISP or general data center IP space.
So the question might be why Google seems to react more to those ambient signals than other email providers. Because we have no issue at all getting personal emails into a Gmail inbox.
But technically, it works outside of Gmail.
Microsoft and Verizon are hardly vulnerable timid players.
Ideally, also a client that handles all this automatically, i.e. chooses the right sender account depending on the recipient, so that I don't have to think about it.
Just like Pidgin was a common solution for AIM and ICQ, this would be a common solution for e-mail and GMail.
It's only ever Gmail that sends mail to spam. Every test I run on that email marks it as clean (generally spam assassin)
Gmail is certainly NOT a part of the open email system you mention, but is a constant thorn in open communication.
Ironically, I use Gapps for one domain and because of my spf settings with "include:_ghs.google.com" I get every cat and his dog trying to send as users from my domain, which thankfully end up in that domain's gapps spam. (hint: let us use geo located includes, like _ghs-us.google.com or _ghs-au.google.com, so that there's a smaller include list!)
This is so deeply disingenuous and at odds with so many peoples experience it’s absurd. Or satire?
Saying this from firsthand experience as a customer of Google's B2B offerings, Google earned its bad reputation on the enterprise side.
Saying this after a good number of years as a PM: when you spend 40-50 hours a week thinking about your product, it's really easy to get lost in the weeds and forget that your users see your product from a totally different perspective than you do. I see this all the time because it's very, very difficult to avoid this problem.
My partner's auncle was also an ex government official from the previous government.
He was in charge of the country's office for regulation work in rural areas.
Up to that point, he was a complete believer that the government he was part of, was there to really change things and put the country in the right track.
When the news about that other guy's 8 mill broke out, he was completelly and uterly devastated.
I think the same can be applied to a lot of the people that works in Google regarding the way they perceive the company.
I just recently read in the news that there has been some retaliation exherted on googlers because of organizing some walkouts.
To think that your company just does good because you try to do good, and not take a look at what is really going around in the world related to what the company you work for is "a little" naive.
Also, I think that right now companies like Google and FB should be treated and reasoned about like states. To think that they will not abuse their power in one way or the other - just like countries do - is also "a little" naive.
That companies this big ought to be reasoned about just like Nation States is a very interesting idea.
But my point was more directed towrds understanding how companies this big are starting to behave.
I do think antitrust is a way to look at it, but the anti trust laws in the US I think are difficult to point towards Google for instance (Probably the same about European laws)
> we aught to have a discussion about whether nation-state size companies are really a desirable feature
I think it goes a way longer than that. Obviously private companies are subjected to a lot less scrutiny than states, but states do miss behave and do not run into the risk of eternal intervention, unless they really fuck up, and even then.
> I have this 8 million from the church, but I need to get it out of the country in order to spend it ...
Disclosure: I've been self-hosting for 5 years.
Then I looked at what was in my Spam folder. I haven't looked into it for a long time before, because, well, Google has trained me that their spam filtering "just works".
I now try to check it at least once a week. The amount of false positives was staggering, and some of it was important.
The worst thing about this blocking is that you often times don't get to see what's wrong. So you're running around in circles, trying to use various mx tools on the internet to find the problem.
This is not just gmail issue. Though with many smaller providers you'll at least get something like "your blocked by XYZ blacklist".
I would much rather get this than silent discard. At least when I've seen things in the logs in the past (yes, I do read my server logs), I have been able to follow up and fix them. I get the sense that Google is even going beyond using 550 and discarding email silently. I just did a grep through my logs to see if this message came up and found it nowhere, yet I have users on my mailing lists claiming they don't get emails and they've checked their spam folders. They're on GMail.
Or you've grown large enough that the base no longer matters.
we care deeply about having positive relationships with developers and all our users
it definitely makes me sad to see articles like this
There are going to be false positives, we will make mistakes
we could do much better there.
Most of this reads like it was written by Facebook's PR department after yet another scandal.
That depends on what you define as "health".
> Email itself, of course has a huge network effect, and that is because you can email anyone in the world, regardless of what email system they use
If that's the case, then why does Gmail system explicitly thwart this objective, as described by this very article?
> If we lose an open, healthy ecosystem with many providers, we'll destroy the base we stand on
And that's exactly what Gmail is doing when it does what is described in the article.
> we care deeply about having positive relationships with developers and all our users
You personally might, but your company does not. What your company cares about is advertising revenue. If your company really cared about users, it would figure out a way to let them pay directly for your company's services (not just Gmail but search, maps, etc.) so you could see directly from users how valuable those services were, instead of having any benefit to users be a side effect of trying to capture their eyeballs for advertisers.
It's too bad that Google has no incentive to allow a premium/ad-free experience as that implies having ads is a negative thing, which basically undermines their entire business model.
I would love it if more of the Internet were based on payments and micro-payments but consumers have voted decisively against that.
I try to sell people on the idea of paying for news but extremely few are willing even for something that critical.
No, they haven't; they've been given no opportunity to vote. Such an opportunity would be, for example, Google offering its basic search service for pay to users. Of course they would have to offer some incentive, some benefit that free users (more precisely, ad-supported users) don't get, but we already know how that works: the obvious benefit is no ads. I would certainly pay for an ad-free Google.
I have paid them for years for photo storage. Looking for somewhere to move it in case their AI suddenly decides it doesn't like me (I've already went through a reCaptcha party after trying to dig up some documentation for some special error messages that I felt should be out there somewhere.)
> I would love it if more of the Internet were based on payments and micro-payments but consumers have voted decisively against that.
Didn't have a chance at all to vote.
They could have earned a lot more on than than than they do from me not clicking on the dumbest ads I know about.
Then they would have - they are a for-profit company.
My assertion that consumers are not willing to pay if they can get something 'free' (ie. with targetted ads) is hardly new or controversial.
How many news sources do you pay for? I pay for my news and take a very keen interest in its quality, but the media outlets have an extremely difficult time convincing enough people to care enough to pay even for their news - the most important thing of all.
> Then they would have - they are a for-profit company.
I doubt they ever considered me as a person, only as male 25-65 => show ads for scammy dating sites.
> How many news sources do you pay for?
2 newspapers, +used blendle a while ago, ready to pay for more when I can pay pr read.
I also have paid other things like an tech/art channel on a video site etc.
(I guess I'm not the only HNer that does this?)
> but the media outlets have an extremely difficult time convincing enough people to care enough to pay even for their news - the most important thing of all.
Around here that might be because my choice is either a two hour drive to somewhere that sells that paper printed, -or to sign up for an auto-renewing subscription.
I could of course sign up and cancel but I already has too much on my plate (more than two kids, trying to be active in my communities etc.)
Google initially federated with XMPP, until you had critical mass and shut out the rest of the Jabber/XMPP ecosystem.
Of course now you seem intent on killing your _own_ messaging product so it's hard to divine any self-consistent intent here.
And there's the problem. The only effective way for OP to contact someone who likely has the ability to fix it was to write an article, post it on HN, get upvotes, and hope you happen upon it.
That's not a viable solution for everyone who runs a mail server Gmail falsely identifies as a source of spam. You don't have a good way to hear about issues like this.
Can you bring back the old web UI?
I experience obnoxious interaction and interface bugs in the new one on an hourly basis, it has horrendous interaction latency for many operations that used to be snappy, it looks worse, and has basically nothing new I want.
Actually, can you bring back the Gmail of like 10 years ago? Basically every change since then has been negative for me.
I have 20 years of XP in this industry and build large gov services, and I am astonished that Google allows itself such a failure.
If you want, I will make a gif of Gmail unresponsiveness in default Chrome browser on 50MB connection - taking more then a minute to load and then some more to stop glitching. Then I will show you ProtonMail, Zimbra etc. having no problem whatsoever.
Its embarrassing and affects all their cloud tools in some measure. Google was once a good company. Today, most of the IT people I know or work with try to replace anything google.
Above post from the google employee which "cares" makes me angry - what do you think m8 ? That we are squirrels ? Shame on you. If you have some moral decency that you propagate here, go and work for somebody non-evil, take as many people you can with you and write a blog about it so anybody can know.
Not to mention it always returns to default view, so you need to click html view each time.
I know its not politically correct and I am sorry that some people will have their feelings hurt, but lets pretend that we speak to grown ups for a moment and not 13 year old teenage girls: the GMail is gigantic pile of shit and due to the fact that personal or other non-shitty mails will be blocked because they do not fit Google agenda, there is really no easy way out of this lock-in. Making it euphemistic and diplomatic counts in my world as dishonesty so if you do so, go freck yourself too because you are contributing to destruction of this nice planet we live in.
At least I have freedom to express and say here that Google can go fuck themselves (minuses are welcome, counting morons is my interpretation), and please Google
PMs, take those words as my default response any time when Google Something asks me on a phone 'Hey, you were at XYZ, how was it?' during those 3 seconds I keep GPS on. Please make default answer - go fuck yourself, thats how.
That's ... unconvincing to the point where it sounds like PR spin. How does Gmail not benefit from crappy email service outside of gmail? How do more gmail users not accentuate this?
>Secondly, we care deeply about having positive relationships with developers and all our users.
Frankly, talk is cheap.
I don't mean to be rude, but your response is either (a) completely missing the point or (b) disingenuous. Assuming the former, can you substantiate your claims?
I have a contra story. One of no delivery issues to Gmail or Hotmail, none, zero, nada. I've run a private email server (friends/family/small business/private mailing lists) for two decades. It's kept pace with every possible factor for reliable delivery - SPF/DKIM/DMARC/ARC, valid client SSL, IPv6, correct PTRs, DNSSEC etc and have no sketchy affiliates. In that time the IPv4 address changed exactly once and has never been RBL'd. Our mail gets delivered AOK.
And yet, even though I think my compliance level is good, I still feel like the blind man groping an elephant. I'm hoping I'm perceiving things correctly; I have no idea if I'm missing something. It helps that I'm an old-school ISP engineering inmate and contributor to well-known MTAs and MDAs, but few folks are lucky enough to have exposure to so SMTP radiation.
My take on the Postmaster Tools is that they've been created entirely to serve Google's purposes, and thereby serve no-one well because (as you point out) it's ecosystem engagement that makes a difference. If you sincerely have an incentive to improve, there is an awful lot of work to do there. It's okay to push the burden of compliance back to the sender, but the Postmaster tools offer only the most rudimentary levers to pull and provide almost no useful information, particularly for smaller/indy senders.
The message that comes through is that Google only really gives a shit about other large scale entities and struggles to see other points of view. This stands in quite stark contrast to Google's effort level over HTTP certificates and webmaster tools.
"Gmail does not, inherently, have any network effects" - you do and you can't help it.
I've been doing email for quite a while (20 odd years) and I don't find Gmail refusing my customer's email for silly reasons too often.
I have had some odd rejections from Gmail (int al) and no-one to talk to. You opine that Postmaster tools are under-invested in but actually miss the real point:
I thought it's great that I have so rarely stuff in my Gmail spam folder that I don't even have to check it regularly until I nearly missed a very important email. Apparently I was lucky that ot didn't get outright rejected.
I rather have to deal with some spam than to miss important, or even regular, emails.
I get why gmail isn't transparent about all their spam-filtering mechanisms. But when personal email to gmail users is being regularly classified as spam, that produces a serious usability problem for non-gmail users.
It absolutely does not. It is so large now that what the parent poster said is the real case. It has such dominance that people are pressured to switch to Gmail or Google apps to get stuff reliably delivered to the massive percentage of email users.
This is exactly like an IE6 PM coming on extolling the virtues of open standards and how they are critical to the success of internet explorer.
Seriously think about it (assuming you are even posting in good faith). If Gmail's success was "entirely predicated on the health", do you really think Postmaster tools and other interoperability efforts would have so little investment?
That simply is not relevant anymore if 99% of email is on Gmail.
I want to echo the exact same issues as the original author. Running my own email servers since the 90s I recently just “gave up” - no single indication about spam issues. All green on several tests. But Google rejects mail. Zero help or tools from google to get off their filter or whatever it is that makes these kind of decisions.
Friends and family all have Gmail accounts for those important emails that Must go through. Enough said.
I’ve moved my email to another provider and stopped self hosting.
He says elsewhere that more email is sent via Exchange servers than GMail.
It's sheer chance that you happened upon this HN post. Google needs to have an open, healthy customer service infrastructure in place to hear about, track, resolve, and follow up on reported issues. This is antithetical to "The Google Way". That is how you get posts like this.
Gmail used to be a small, light, fast and simple email service. Frankly, you should be ashamed of what Gmail has become.
To sum it up: sorry, but what you're saying here doesn't sound like it's true, and if it is let me tell you that you're failing at it, miserably. The only reason I didn't move away from gmail is that it's just a lot of work. But the time will come, eventually.
That's the box I use for my personal email. Out of principle. I believe in a distributed/federated net. It just means that I can't expect that emails I send actually get delivered. Sigh.
My experience for business has been that you simply have to bite the bullet and be on outlook.com or gmail.com. Or you can't expect that regular email always arrives. What everyone in the corporate world expects of course.
I've heard this story from everyone I know that operates their own mailserver. No hyperbole. I would be tremendously happy if Google would spend some serious (and visible) effort on this. Thanks Paul.
That's not true at all. I've noticed when sending from Gmail to Gmail it always gets delivered and quickly. I've noticed with other email service people, even ones using big providers like Yahoo, it's really hit or miss if they can receive from me, send to me, and it takes a long time to show up in my email. So there's a really strong network effect to use Gmail because Gmail doesn't work well outside Gmail. So much so that I keep a bunch of address (personal/Gmail, work, school, my personal web site and email server) and try to pick the one that works best based on who I'm sending to.
Really, it's not about incentives; it's about perception. Just like you can't act at Google's scale on your own, you can't perceive the effects of Google's actions on your own. You care about fixing issues when you hear about them, but how do you hear about them? What can the Google-animal see, and what is it blind to?
Why not count the amount of emails coming from those domains and give them a "softer" filtering algo if it's below a certain threshold? Or open a whitelisting program where you can go full Google on people who violate the terms? I think there are (automatable) ways to solve this problems, but only if Google understands that emails are for human interaction, not to receive ads.
(Edited for typos)
There needs to be a way to inspect what caused mail delivery so we are able to fix it. There needs to be a way to provide feedback to Google about mail that was wrongly rejected. If you can push this forward internally, please do it.
This is a far too regular occurrence to just ignore: it is a rule that delivering email to gmail from outside is unreliable, not an exception.
It is kind of funny to me that other email vendors could implement SMTP and co properly and Gmail _more recently_ started to have issues. Just because of this I started to look for alternatives because I will not stand and watch how an ad-tech company destroys the open internet, even if this will be very inconvenient personally.
Personal email server will send ~100 emails a month (maybe 1000 if they are extra prolific). Roughly half of those emails will be in reply to emails that originated on your mail server (you can link those with basically unique message-ids that you generated). There will be a bunch of extra safety measures on the server (reverse DNS, SPF records, no open relay...).
Spammers, on the other hand, will send thousands of messages every hour. And have none of that "other" stuff.
But volume is a good argument. I don't think that spamming could be cost-effective with something like 100 e-mails per day from a single IP-address. So why not just let small servers pass every filter in the world (may be except reverse-dns record) as long as they are small? The danger should not be very big.
Of course I might be missing something, I have no idea how spammers really operate. Or may be Google just don't care about personal servers at all.
They have literally shown what they have tried to do and it is not enough.
If this poor guy is a "false positive" then what is his path to resolution? It looks like he doesn't have one and saying "Yeah, that sucks" or "Because you made it on hacker news we will investigate" isn't helping all the other "small fry" out there.
But the problem with all Google services is that there's apparently nobody listening. Unless you're a paying customer, and even then, I gather that it's iffy. It's all automated, in extremely unhelpful ways.
I do appreciate, however, that it's largely a scale issue. Google is just so big, and operates on such small margins, that it's arguably not economically feasible to provide individualized support.
Unless you make it to the HN front page, or whatever.
As a practical matter, if you want to send email to Gmail accounts, you'd better be using Gmail. And apparently, even that doesn't always work.
How extensively does the Gmail team test interoperability with other mail systems? From my personal experience, one sin committed by many teams who in say they play nice with the ecosystem is to neglect to test integration against other players in the ecosystem. It's annoying to test and slows down development pace, so frequently no developer wants to do it. This leads to a situation where it may be unintentional, but there is definitely a "favored client."
This week I encountered the following situation the first time:
1. I received an email from a Google apps (or whatever it is called) user. My response to that email landed in the spam folder.
2. I exchanged one or two mails more with another person in the same organisation without problems.
3. Suddenly my response got rejected. I tried sending via various means such as via Thunderbird, via mobile on 4G, via mobile on Wi-Fi, changing the content slightly... No chance. My email got rejected. In the I ended up calling the person from number 1 on his mobile to get the second person's mobile number...
Needless to say, I have been less than pleased.
Person 1 was very surprised about their spam filter misbehaving, but at least I can now offer an excuse by pointing to this blog post...
What then should we make of the fact that Google practically ensures that they won't hear about such issues by making it next to impossible to report them?
However, as one of the PMs in charge of one of the most important messaging applications on the planet, you cannot claim the following without stating your reasons:
> Gmail does not, inherently, have any network effects
Yes, compared to facebook, and within the context of messaging apps, gmail's network effect is weaker. But the fact that the very existence of the gmail monolith can kill off small mail servers (not implicitly, but by failure to deliver messages), is a real problem.
I understand it's a difficult problem. It's not a gmail scale problem, but the effects, times the number of small mail servers is important, because those small mail servers will be what is left if any of the big players in this ecosystem collapse.
Don't you see? This post on HN means you have created the CLOSED unhealthy ecosystem. You built what you just spoke against when you shut out email from small providers. Shame on you for speaking idealically and not confronting the reality of the situation.
I am not taking a stab at you personally. I truly believe that you care when you read the story here. But if you really cared enough there would be no story to be reported here.
It's a purely economic and automated solution that should work well enough for small operators (collateral for a small email volume should be small since spammers really need high volume)
Content filters should never be applied to abuse addresses, yet Google happily ignores this and never even replies to abuse complaints sent to @gmail.com or @google.com.
If you reject most non-Gmail messages it does!
In an ideal world, yes. Unless Gmail makes it harder for anyone using a different provider to communicate with Gmail users. In which case you've effectively "encouraged"/forced everyone else to use Gmail to take advantage of the Gmail network. This seems to be exactly what's happened to the author. This is very similar to the complaint voiced recently by the Mozilla leader as well.
I don't doubt that you as an individual have very good intentions. But individual intentions do not translate well to organizational priorities. The best way to prevent Gmail-lock-in in the long term is to have more diversity and competition in this marketplace.
we will make mistakes
What I don't understand: why aren't these false positives sent to the recipient's Spam folder? Why else does that folder exist?
I have SPF, DKIM, reverse DNS, DMARC .. all the things, and yet I still get e-mail dropped.
Will you guys fix Postmaster tools for small servers? What can I do to send e-mail to my friends?
It is also an excellent goal to use AI to scale as many processes as practical to provide ever-broader services.
Yet, it is obvious that, at the corporate level and despite being one of the most massively well-funded companies worldwide, Google is famous for abysmal customer service and most particularly, providing NO way whatsoever for a person who has been damaged by a "false positive" to every correct a real person at Google to get a resolution.
Yes, providing an avenue to real people who could resolve the issue would be expensive. I seriously doubt that it would be so expensive as to make any serious dent in Aplhabet's profitability or stock price (unless, of course, the problems are far more massive than anyone knows).
Moreover, providing the ability for people who have been "false positive" damaged by some spam, account violation, etc. to reach a person for resolution to a problem would also provide large amounts of the fine-grained detailed data that would allow Google to fix its issues at the algorithm level, thereby reducing the need for the support service as well as the frustration.
Why is this not done? Is it that the problems are actually massive but hidden? Is it that management does not care the way you do? Is it that mgt actually wants to drive off all small services in order to dominate? Obviously all speculation, including bad speculation. But leaving zero ways to contact, as well as zero information about the issues, leave the field ripe for all the speculation. This is the sort of reputational problem that can fester for years, seen only as a minor issue -- until it grows so massive and passes a tipping point, where the reputation and market position is unrecoverable. I hope Google does not go that route.
Guess I’ll find out whether correcting it a second time applies to the actual newsletter messages.
I bet you also take the security of your system seriously. What other Internet platitudes can you spout?
Holy shit, GMail's going away? This is the kind of thing we're told just before a Google product is killed.
Downvotes don’t mean anyone’s upset, there’s a large variety of reasons people downvote. Same is true of upvotes, there are lots of different reasons for them. Try not to take either personally.
Downvotes don’t require explanation, just like upvotes don’t require explanation. It’s a great idea to be curious and seek ways to avoid downvotes, always good to assume you might be wrong. Less so to demand you were right, even if you are. :)
Also be aware that HN guidelines suggest avoiding complaining about votes you get, https://news.ycombinator.com/newsguidelines.html
Hope that helps, cheers!
So thank you for explaining the situation to craftinator :-)
That's the point.
It doesn't have to be conscious dereliction, it can just be 'big, dumb bureaucracy'. Interestingly, operational ineffectiveness can happen within well run organizations, particularly if there's no poignant reasons to change.
These kinds of things tend to happen when one part of the organization is mining Oil or Gold and throwing vast surpluses around the rest of the camp.
You have $100 Billion dollars to fix the problem - a hoard of the best and most highly paid talent on planet earth.
So fix it.
I don't mind spending a penny to send an email, I don't send enough to make any difference. But to spammers, it makes abusing the email system unprofitable. And if I receive half the revenue from people sending me email, it'll likely work out to costing me about nothing.
As far as it relates to email, this is why platforms like Sendgrid got so popular, they manage the relationship with Google and others for you.
I do agree that Google's outsize influence in so many markets is concerning.
I guess the only real difference is that with AOL it was less insidious... You knew it was only for aol.com emails and that you weren't dealing with business info. Not that emails to personal addresses aren't important, but at least it wasn't suddenly SomeRandomFortune500Customer.com that sales is trying to email a proposal to right before the deadline that "is make or break for the entire company!"...
Google - well, I haven't had to, but can get in touch with a competent person there, but not officially. Which is not the case with ATT, with whom I have had a mail issue for over a decade. At this point I'm thinking either their postmaster died in his closet office back in about 2006 and nobody noticed, or they simply don't give a damn.
Not sure I agree about being able to reach a tech person at Google, even informally. Bitching on Twitter can get some results, but that’s hardly the way to operate a business...
It absolutely should not be the case that you need a pal in the right department to get something done, but that seems to be where we are - our Politburos are just privatized.
Edit - My mail issue specifically is with WorldNet. I believe that's run by an ATT spinoff call "Maillennium", but maybe that changed. I don't especially care from a practical perspective; there is exactly one person whom this affects and we worked around it forever ago. I just see it as a barometer of ATT's interest in being a competent network operator.
I do too, because there was an actual useful URL that had contact info in the bounces and SMTP errors. IIRC, I actually managed to get a human being on the phone and got this sorted right quick!
Perhaps because email itself wasn't as big of deal back then? Phone calls and snail mail were used with a lot more regularity back then.
Sure there’s an element of truth to it, but email was insanely popular before the iPhone too...
Well, I am almost 40, so not really a millennial at all, but a gen x'er. And I remember being in college and going days, maybe weeks without checking email. I also remember my university giving me an a .edu address, which I was technically supposed to use (supposedly there was important information sent to it from professors), but never logged into once.
I see no problem with this.
If you had to pay anything to distribute your video, the quality would soar relative to what we have now.
Breaking them up would be a remedy to the general attitude that big, concentrated companies often develop toward the average consumer.
Also, Google email deliverability is good, inbound and outbound. I’ve been running an email service and have had no problems with them, unlike Hotmail/Outlook.
IMO what "you can tell" is that they have good support where you're actually a paying customer, and no support for their free products.
I've heard a few examples here of people having trouble getting support with both GSuite and GCloud.
Sometimes I've got support from GCloud. Even once got two blokes onsite because they'd messed up something badly, but at that time I was working on a project that was seemed to be big time bragging rights for anyone involved on both our side and their side.
Other times our requests would be handled by someone who was more or less clueless && who clearly didn't care at all IMO (this was also the same project.)
For smaller customers it seems to be hopeless from what I read here: - Someone getting their account (GSuite or GCloud) closed for no good reason and with no way to get it back seems to be a thing that happens from time to time.
Source: Led a hiring committee this year. Interview invitations and job offers went straight to the junk mail folder. Also serve as Director of Graduate Studies. Admissions and funding offers, as well as general inquiries about availability, all went straight to the junk mail folder. We're not going to change our email service.
I mean, assumming that the sender doesn't have broken SMTP client or doesn't follow standards.
It's gmail's user's problem if they don't accept mail or check their Junk folder or use an e-mail provider that they can't contact to solve issues on their side.
In the old days, if I didn't receive mail because my mail provider was rejecting it, it was me who had to talk to them. Free has a cost. And it's gmail users who should be paying it in their time and fruitless attempts at communicating with google. At least they'd realize what company they're enabling and how much it doesn't care about their e-mail.
> In the old days, if I didn't receive mail because my mail provider was rejecting it, it was me who had to talk to them. Free has a cost.
In the old days average email user was being inundated with low-quality spam and getting mired in fraud. Google fixed that problem for their end users. In doing so, they made it hard for a small % of people who want to run their own email servers, but they've given all of these people an out -- go sign up with someone who knows what they're doing, and google is going to effectively outsource the fraud/spam management to those companies.
You can tell me I have it wrong but I'm literally just describing the landscape that actively exists and how to navigate around it for anyone who wants to, but you're not taking out google.
That suggests a simple solution to the problem that can be done on the gmail side: for any small mail server (to pull a number out of my behind, say less than <200 emails a month to any @gmail address for the last 12 months), white list them if they satisfy the rest of the usual requirements.
Even if someone attempts to game the system and creates a number of servers, for an effective spam campaign it means a large number of servers, costs go up.
It might be worth experimenting with "abusing" this behaviour to put your small server on the Gmail whitelist. Start sending a large number of generated emails to a @gmail.com mailbox, log in and ensure none of it ends up in spam (ideally automate that too :), and there we are. Anyone have any idea how many emails that is? :)
That said, like the OP, despite having DKIM, SPF, proper DNS, and every other measure applied, it's conceivable that gmail might reject some emails from us. I don't have any memory of that actually happening, but I have heard stories like the OP's several times. It's a fairly common topic in conversation forums about self-hosting email servers. I don't expect the situation to change any time soon—I don't expect Google to change their ways and it would require significant pain for me to surrender to the gmail hegemony.
Best to assume gmail is flawed, which it is, as the parent poster did. At some point Google will realize the image problem and act.
Now that I live in a lot of Gmail accounts, the Spam folder is just another folder I have to check on a weekly basis.
I still have quite a few a accounts under Gmail, but I'll probably do a full change this year.
Having control of my email is very important for me. Google could close my account for any number of reasons (billing dispute, hacked account, etc), and I would lose my actual address since the domain isn't mine
I do have a very old address that is on my own domain, and though that is currently routed to Google, at least that simplifies things a lot, but unfortunately for many years I was not consistently using that address for signing up to things etc.
If anything, it means that teams are probably too independent and that there is no central approval process for things like sending emails.
For example, a lot of companies use products like Greenhouse. I am sure this will send email from your domain on your behalf. If you don't know what you have to talk to the DNS administrators to add a bunch of TXT records to enable that, you will just notice that some percentage of emails never get delivered (or if the product doesn't tell you that they're bouncing, you may never know. how could you?)
If you use something like Zendesk, you'll note how many people have been burned by this. By default, they end up using something like support.yourdomain.com because despite detailed instructions on how to set up the necessary DNS records to send email from youdomain.com, people still fail to do it right and then complain "nobody ever sees my support tickets".
My point is, email has been abused so heavily that it is somewhat difficult to set up a working system. That was my experience when I ran my own email server. Although I did OK with delivery, I also aggressively filtered messages and used greylisting. This broke a lot of broken email systems, whose administrators immediately blamed me. (I had a long back-and-forth with some company that wanted to hire me. Their email system was super broken. They blamed me and said that they weren't interested in a programmer that couldn't set up a mail server. LOL.)
Tangent, but this kind of stuff is super annoying and probably happens far more often than anyone wants to admit. A few friends and I have been commiserating re the incompetence of potential employers who've rejected us because of their own misconfigured environments or fundamental misunderstandings of the systems they run. We had one interviewer close out a candidacy because the code sample "didn't run" on their interview's system -- the traceback showed he had a broken half-Py2/half-Py3 install.
Hey, at least they're _trying_ to migrate to 3 :P
users don't even know that it's there.
While’s gmail’s filtering is obviously neither your responsibility nor under your control, you also can’t really fault people for using one of the most popular email services on the planet.
I would hope that your department follows up some other way (you surely have a phone number!). Searches are crazy time consuming and expensive, and grad student recruitment isn’t exactly free either.
Why not? Popularity does not imply quality. If it is a poor service, you can (and should) fault people for using it.
Agreed. There should be some give and take, some minor reputation at stake. It's healthy to have some people applying back-pressure against popularity. I like the idea that laypeople might hear whispers among their tech-savvy friends that it's not considered "cool" to have a gmail address. Much like having an aol.com email address was seen as uncool in years past. It's perfectly reasonable for some people to be working to give gmail.com a bit of that same reputation.
I haven't laughed this hard in a while. How'd that go with convincing people not to join walled gardens like Facebook and Insta?
And actually, more seriously, the reputation push-back on Facebook is finally reaching levels where laypeople are picking up on it. There's still a huge amount of ground to cover, but laypeople are now hearing the criticism of Facebook. You can see many switching from a state of ignorance to "I don't care; I have nothing to hide." Maybe in a few years we'll see more people saying "you know what, I do find these privacy losses bothersome." As with many reputation matters, it's a gradual process with give and take.
More broadly, grad school admissions--and, even moreso, faculty hiring--is already full of unwritten rules, backdoors, and tacit understands of "how things are done." It emphatically does not need more of those, especially petty ones related to email.
People's choice of poor quality service provider X impacts their ability to do their tasks when the service is poor. X can be email, insurance, banking, anything you externalize in your life.
The fact that a service is popular is a poor indication of service quality, if there is a correlation at all. Examples abound for popular services and products that are poor quality. Usually, the best services are a bit up, from the popular choice point, in the price/quality ratio. More so if there are free (as in beer) offerings in the market.
What's the alternative? Making sure that the recipient gets the message by sending a courier to his home address?
I remember when a buddy of mine, back in the pre-email days, applied for a job at a company. About a week later, he received a letter, asking him to call them. Turns out, he forgot to put his contact info into the application, but they were interested enough to look up his name in the phone book (they also have/had street addresses were I live) and write him a letter, hoping it would reach the right person. "Always put your contact into into an application" isn't a petty rule, and neither is "use an email-provider that doesn't randomly reject messages" in my book.
I would expect both sides to make a bit of effort. Applicants should check their spam folders and follow up, especially if there's a known timeline (American grad school offers need to be extended and accepted by the 15th of April, for example).
Institutions should maybe not rely on only an unreliable, unacknowledged form of communication. Since the applications usually collect phone numbers and addresses, a phone call or letter would be a very reasonable follow up; neither costs more than 50¢ to use. I think all of the PhD programs that accepted me did call and it seems like a good way not only to notify, but to actively recruit.
At the very least, I knew that if mail ended up in that inbox, it was usually important.
More to the point, Fastmail allowed me to use Mail.app on macOS and get away from Google's web UI which, despite best efforts on their part - read: none at all - never worked correctly on the desktop with regard to message deletion in IMAP clients.
There's a high 9's probability that it will arrive safely, but there's several ways that things can go wrong, simplest of which is that you are deleted by a misconfigured filter.
There are US ISPs (cough sbcglobal) that routinely throw away mail from some arbitrary IPs despite there being absolutely no SPAM blacklist entries for the IP, even if you are replying to a mail sent from one of their clients.
outlook.com once, (and maybe still does) blocked email from an entire C-block for some reason.
(edit - grammar)
And there's the core of all of this. SMTP is a best-effort protocol that does not guarantee delivery. Until someone comes up with something better, we're condemned to seeing "Help! My very important email disappeared in transit" threads on HN every several months for the rest of eternity.
If I send you an email and gmail doesn't get it to you for either of the above reasons, you won't get a second chance.
(For me gmail is inferior regardless as my privacy is violated)
Maybe this is just my personal organizational problem, but it was like a waking nightmare when I really started to look with fresh eyes at how unstructured my email management strategy had become with the tools gmail gives you. It took me weeks of nights and weekends to sort this all out (finding messages I actually want to save, identifying senders I actually want manually sorted to other folders, unsubscribing from lists I've been ignoring for years because they were conveniently out of site in "promotions") to a point where I felt ready to migrate to a more traditional mailbox setup. I can easily imagine many people seeing how daunting this task is and just deciding to stick with gmail.
I use fastmail because the payment is direct: I give them a few bucks each month.
With free email (gmail) the payment is not direct. They are doing something to get money, but what is unknown.
I guess you mean something uknown and nefarious.
And you are more comfortable with Qihoo 360, the Chinese company that owns Opera/fastmail?
* spam detector is a bit aggressive, sometimes legitimate emails make it in there.
* features are limited unless you pay.
Full disclosure: I pay for their Plus product. My rationale is that I pay twice as much per month for Netflix yet email is hugely more valuable to me than Netflix is.
More full disclosure: I used to work for Google. That's part of what made me move from Gmail.
A good email service is one worth paying for.
May be if I'm paying some money to some local company in my city, I can have some confidence, because I can come to their office and yell at them until they return my data to me. But they can just be incompetent and lose it, so even in this case I better do backups.
[EDIT: Sorry, missed the 'free' in there. Guess you get what you pay for]
If they could solve for my situation and not charge me that kind of money, I would switch from Gmail immediately.
This is why Google keeps winning and everyone else is losing, particularly the people who get the "free" services but don't have any say about those services.
The real numbers might be different, but I heard that Google gets about $100/yr in ad revenue per user. That is, they are getting $800 a year for all of those people. Everything you buy is $800 more expensive and there is no way you can say no to that!
If I spend $300 dollars on a phone every 2-3 years and $40 for service a month, am I then permitted to raise concerns about spending hundreds of dollars a year for email accounts?
By the way, Fastmail prices are rising. One used to be able to get a free account, and the standard plan used to be $40 a year, so might be good to get them sooner rather than later to get grandfathered-in in case of upcoming price risings.
$210 annually seems like a deal if it keeps your family’s most sensitive private messages away from ad targeting.
A lot depends upon how frugal you already are, but I found it trivial to find some extra savings to pay for it. There's a few things I like to treat myself to, and doing that one less time per month pays for it. Eating out one less time per month pays for it too. Ordering slightly cheaper meals for myself when we eat out would also pay for it.
Hell, when I turned off my home server, which I rarely used, I saved $9/month in electricity. If you live in a high power cost state, do a personal power audit. Our family was pretty lazy with turning things off until I figured out how much it costs to run any given thing.
With a family of 6, if you haven't crunched your finances, there are lots of very minor behavior changes that can save you enough money to pay for something like fastmail.
They removed the free GSuite tier a long time ago, and they've obscured some the options you could use to configure a similar setup in gmail. I'm not sure if it's still possible.
There is the huge possibility that this will cause issues with some mail servers because I'm assuming it's doing a send on behalf of (I could be totally wrong). But so far (1yr+) I don't recall any issues.
I switched to Proton for a bit but I just didn't enjoy using it at all. The final nail in that coffin; I was in the middle of selling a house and receiving documents and tons of back and forth and it was an absolutely terrible experience. Every realtor/contractor/etcs signature pictures, etc showed up as attachments and the search was awful so having to dig up contracts and everything else that goes along with a house sale was just an absolutely miserable experience. Every single email looked like it had tons of attachments. I had to go through every single email in a thread to find the actual documents.
So I went back to gmail.
Clearly they keep the hit in the search results for plausible deniability, but they're trying to influence people by suppressing speech. Given that Google is becoming synonymous with the internet, like AOL, this is a very concerning trend.
This has been a well known occurrence. I’m think about all those pictures of “Hillary Clinton is”  typed into each search engine and Google was polar opposite of all others.
I think one needs to be pretty obtuse to not see Google is “curating” results to fit an ideology.
Everyone should be mad at that even if they believe in the same things.
EDIT: Well... apparently some people don't believe me, so I just re-tested. "Hillary Clinton i" in four engines.
I couldn't use "Hillary Clinton is" because "for some reason" Google guessing completely stops working when you type "is" after her name - can someone find me another phrase besides "Hillary Clinton" that "breaks" the search fill when "is" gets added? No? Well.. That sure seems like Google is "curating" then.
> can someone find me another phrase besides "Hillary Clinton" that "breaks" the search fill when "is" gets added?
They did all this while their fathers were undertaking sensitive high-level negotiations with the Chinese government.
If you try Googling for keywords related to this story, Google will claim to have indexed thousands of related pages, but will only serve 33 search results:
If you enter the same keywords into Bing you will be served thousands upon thousands of search results without any of Google's limiting or censorship:
One interesting thing to note is that most of the websites which picked up this story are right-wing, and that Google appears to simply be blacklisting non-mainstream right-wing websites from its search results. Whether or not this is due to any implicit ideological bias on Google's part is uncertain.
Whatever lead to Microsoft being sued for pushing Internet Explorer in Europe and whatnot? Shouldn’t that kick in already for at least some of the recent Google bullshit?
There's definitely for GMail to become the new "Windows", with everyone else fighting to stay connected.
The administrators of the account decided to outsource the smtp relay to google. It is ridiculously unreliable, and regularly sends “no such address” responses to my synology NAS (which emails me periodically).
Similarly, when I used gmail’s IMAP gateway for work, it was regularly down.
I’m pretty sure Google will succeed at killing email. I hope gmail also goes down in flames if they succeed.
I hate to say it, I really do, because I used to respect Google and liked their products, but if you imagine combining the perception of "AOL is the internet" with the way Microsoft acted up to the mid-2000's and I think you have Google.
It terrifies me just how deeply Google has entrenched itself in the K-12 education market. At least when Apple was everywhere, we still had Microsoft Office vs Appleworks/etc.
Google is now more "the internet" than AOL could have dreamed of.
On Google, I'm not so sure. The tech is nice, and everything works beautifully for 99% of the users
I have an email with some account information that I access a couple times a year at most but will consistently need at least once a year. Just infrequently enough that I can't remember the exact contents.
I know exactly what the subject line of the email is and it's a suggested search query in Gmail but about 2 years ago Gmail just stopped being able to find it. I can navigate to the email via label or star just fine but I can no longer search for it.
Is it because it's an 8 year old email? idk what the reasoning is but Gmail as deemed it irrelevant and stopped returning it as a search result. This isn't a platform issue either, it happens in App, or browser on multiple devices.
With all the hand-wringing about what happens when the "computers" take all of our jobs, I think you crystallized how it all ends for us. It won't be a cinematic extinction of humanity. We will just be ignored--each of us trying to talk to a machine with a human voice, and each of us perpetually on-hold.
Seems like it ends up working somewhat like China's social credit policy, but without specific intent. As long as you always conform enough to fall into the normal area of the algorithm you're good, but get off towards one edge for any reason and you quietly get [effectively] booted off the 'net.
It’s not just Google.
Eventually, even when you get fired because they tired of you being a nag, you write down all the times your nagging was right, and someone else hires you to be the math nag, because corporations have accepted they need them — even if they’ll also eventually fire you for being right.
There’s easier ways to make money with the same skill set.
Yes but some people really love to nag.
I work in enterprise for similar reason, I really like sighing at things.
Google sells mail+spam filtering+more as a product, with a seven-digit number of paying customers, each of which pay per month and user. What more incentive could they have?
How did you even get your report to be read by an engineer at Google? This seems impossible unless you have some kind of business connection.
That bit isn't the MSB, though. The MSB is that they do care about fixing interop problems with everyone, they just don't read mail from randoms. I think they assume is that if a problem is their fault it will show up on their monitoring graphs, and watch their error graphs instead of their postmaster inbox.
And it’s a small thing, but I’ve really been trying to correct myself when I use google as a verb. I’ve been trying to say “search it” instead.
You mean not to try to actively sabotage the internet that we once knew?
I worked on a team with a large (real double opt in, not spam) email list. We had to pay for a bunch of AOL accounts to figure out the trigger points, so that we could get our email updates out overnight without tripping AOL's spammer logic.
And there was a time when they had dominant email share like Google has now.
It has become very difficult / unwise to host your own SMTP server, though, but that's been true for ages now. Gmail's marketshare is going to make that veeeery obvious, but it really has been a bad idea for the better part of a decade, at least.
I use fastmail. They have humans that respond quickly when I have a problem (twice in 10 years)
Can you mention (or link to) situations where gmail refuses to send mails to another domain?
This gets couched in it being "hard" like somehow its an insurmountable force of nature.
That's not been my experience at all.
Why is this no longer true?
IMO, other competitors came along offering a better service/experience and AOL just couldn't compete. At the time, you could not know what this better product looked like. I believe the same will happen with the internet giants of today.
Even worse, they have incentive to create these problems.
Gsuite is extremely profitable for them, I am sure. Charging $5/user/month for hosted email must make them a pretty penny.
I would guess that no gsuite sent emails are ever blocked with this suspicious message 550 error.