

More On Gmail’s Delivery Delays - daigoba66
http://googleenterprise.blogspot.com/2013/09/more-on-gmails-delivery-delays.html

======
jrochkind1
A ~2.5 hour delay that effected ~1% of messages -- I've never had a mail
provider so perfect that a delay of a couple hours _didn 't_ very occasionally
happen.

As far as their telling us, the failure didn't cascade causing their queues to
fill up and other horrible things to happen. As far as we know, no mail got
bounced, no mail got dropped on the floor and lost permanently, etc.

A 2.6 second delay isn't even worth mentioning, how could one even tell the
difference between a 2.6 second delay due to a problem on google's side, or
routine problems in the email network outside of google that could easily lead
to a second delay or whatever.

Did people really get upset about this? Was it worse than it seems because of
details they are ommitting? Are they preemptively advertising how well they
dealt with this barely-a-problem-noticeable-to-users problem, because the
message is really "see, look how damn stable gmail is, our worst problem ever
is barely noticeable"?

Or, what?

Maybe those 1.5% of messages were all to the same users, so some people had no
delay, and some people had ALL their mail delayed a couple hours? That would
make it more noticeable.

But in general, occasional (like once a year or less) couple-hour mail queue
delays should probably be expected from any mail provider, no?

~~~
susi22
We use email communication for our help ticket systems and all our users were
affected (~20k). At

9am it was ~10min delay.

10am about 21min

noon about 1h

3pm emails with ~3h delay arrived.

Then it got better. We werent' able to help our users and they of course
blamed that we responded slowly. Actually a bounce message would've been
preferable since then they could've used a phone or stopped by.

You don't really expect your company @example.org to take 3h. Either bounce
withing ~3-4min or you expect it to be delivered by that time.

~~~
jessaustin
_We use email communication for our help ticket systems and all our users were
affected (~20k)._

After this event are you considering a different architecture?

------
alanbyrne
In a 300,000 user mail system I've seen mail queues grow by many thousands of
messages a minute causing hundreds of gigabytes of email to be queued on the
disks.

Once the issue is resolved and the queue starts flowing again it can take a
very long time for these messages to be sent successfully, especially when you
need to scan each one for viruses and account for disk latency that doesn't
usually occur on these servers.

------
ChikkaChiChi
I have a hard time believing these statistics. Unless of course the 1.5%
happened to be every single person who happened to have a Twitter, HN,
Facebook, or Reddit account and was willing to post about it.

~~~
tcoppi
Note that they didn't say 1.5% of users, they said 1.5% of _messages_ were
affected.

~~~
ChikkaChiChi
I'm totally aware of how they said what they said, I just don't necessarily
believe it.

------
arbuge
The date on this appears wrong - I think they mean yesterday, Sep 23. I must
be one of the 1.5% affected - my gmail messages arrived 5 hours late.

~~~
thrownaway2424
That's not what it says. It says that 1.5% of messages arrived at least 2
hours late. If you got 40 messages yesterday then you probably got at least
one that was 2 hours late.

~~~
arbuge
Ah. Better drink more coffee I guess...

------
westmere
The problem for us wasn't so much the email delay, it was that whenever a user
would send a message from their iPhone, they'd get an error message "Server
rejected mail." If gmail had accepted the message but delayed it on the server
side, we probably wouldn't have noticed an issue at all.

