
Mail Loop From Hell (2012) - dbrgn
https://blog.dbrgn.ch/2012/7/29/mail-loop-from-hell/
======
suzzer99
We had a Social email loop when I worked for a company with 33k employees.
Someone sent out a newsletter to the company ALL mailbox, which had some tips
for winterizing, and asked mostly rhetorically "What are your tips for
preparing for winter?"

A handful people replied to the entire company w/o realizing it - stuff like
"Well I'm in California, I don't have to worry about it. Ha!".

I think it would have ended there. But some grumpy programmer in NYC of course
decided to reply "People. Stop replying-all!" One would think being a
programmer he'd be smart enough to reply to just the people who had
unintentionally spammed the whole company. But no. He spammed the whole
company.

And after that it was on. Dozens of emails a minute. Tons of replies like
"Please take me off this list." Tons more grumpy people complaining. Big
bosses pulling rank "Ok I am Vice President of muckety muck and we need to
stop this replying all right now!" And the best: "Stop replying all to
everyone to tell them to stop replying all!" \- replied to all of course. O_o

It didn't help that a lot of people worked in labs and shared computers. So
they'd log on and see a ton of emails w/o knowing what happened. By the end of
the week it finally died down. Then the next Monday people came back from
vacation, saw 500+ emails, and of course asked to be taken off the list. This
went on for weeks before it finally trickled to a halt.

Why anyone was allowed to spam the entire company, or the address couldn't be
blocked, I have no idea. I was tempted to use it to fish for a tennis partner.
And then apologize and claim ignorance of course.

~~~
theoh
> I think it would have ended there. But some grumpy programmer in NYC of
> course decided to reply "People. Stop replying-all!" One would think being a
> programmer he'd be smart enough to reply to just the people who had
> unintentionally spammed the whole company. But no. He spammed the whole
> company.

It's probably _because_ he felt that he was in a position of expertise and
authority that he tried to pre-empt the problem by telling everyone not to
reply to the list. One last message to all, then the problem would be
definitively solved, or so the logic would go.

~~~
bonaldi
If he had such expertise he should have sent his reply with all the recipients
on bcc. Then even if a few people do reply they only go to him and things
fizzle quite quickly.

I don’t quite why so few people break reply storms in this way. Bcc isn’t
_that_ mysterious.

~~~
theoh
The reason is that a bcc'd message doesn't stop people from replying to
earlier messages in the thread, whether or not they are using a threading
email client. Threaded email reading interfaces probably make your strategy
more effective, but they don't eliminate the possibility that people will
respond to earlier messages, defeating your "firebreak" message (the more
addressees, the more likely it is).

~~~
bonaldi
That's true, but it definitely works a charm in Outlook shops — Outlook tries
to get you to reply to the latest message in a number of ways.

------
ineedasername
Another story about inappropriate emails, luckily I wasn't the perpetrator
here.

It was the late 90's. Email systems were generally wide open for exploitation.
(It was the golden age of Spam)

I myself had figured out that it was trivial to send a message and make it
look like it came from _anyone_. Simple:

    
    
      telnet <server> 25
      HELO
      MAIL FROM: <whatever you want>
      RCPT TO: <recipient>
      DATA <body contents>
      QUIT
    

I used this a few times to mess with friends, send them messages from God,
etc. Good times.

Thing is, I wasn't the only one who'd figured this out, and the university had
an "All Users" list with only 3 or 4 people whitelisted to send it messages.

Someone else knew about the forged FROM scheme, and _speculated_ that the "All
Users" list didn't do any fancy verification of the "From" address.

This speculation was correct. And the University President was one of those 3
or 4 people whose email address was whitelisted.

The problem for this student was that they thought to themselves, "No, it
couldn't be that easy..." And proceeded to send a test message.

It was that easy, and their test message sent successfully. The contents
were... explosive. First, it cancelled all classes for the day (of course).
Had it stopped there, this story might have ended as just an ultimately
harmless prank. But it went further.

The message went on to describe how the president, with this free time from
classes, would make herself available to provide, um, carnal knowledge of a
very uncomfortable place at a particular time & location.

It didn't take long. IP addresses were statically assigned. It was easy to
locate the offending computer, which in this case was in a computer lab.

So, cross reference login times against that IP address, right? Nope, not
necessary: The computer was the one reserved only for the lab assistant on
duty. Expulsion was rapid. In an interview with the student newspaper, the
student decried, "I didn't think it would work!"

~~~
umvi
Reminds me of the time I was working at the Air Force Academy hospital in the
basement warehouse in the mid 2000s. There were a few computers, and I wanted
to mess with one of my coworkers (a senior airman) who was sitting at the desk
in front of me.

There's a Windows command "net send" where you can make a pop up dialogue
appear on a user's computer with a custom message. The problem was, I didn't
know his exact username, so I did "net send * 'What's up dog??'". The dialogue
popped up on his computer, he looked confused, and I was giddy. So I went
further and sent a few more, including "You suck at computers" and "Your
computer has been infected with a virus, sucka". It wasn't until a few minutes
later when some visibly distressed IT personnel rushed into the warehouse and
started interrogating people and checking the computers one by one that I
realized something was terribly, terribly wrong. Apparently the dialogues were
popping up on _every single computer in the entire hospital_ , including the
commander's (a full colonel).

I'm sure there would have been more disciplinary action had I not been a high
school civilian intern. Everyone started calling me "Neo" from then on, but in
a derogatory way.

~~~
irishcoffee
I did the same thing in highschool, but on a loop... On a timer... Boy did I
get in trouble.

~~~
andrelaszlo
I did exactly the same with the loop and net send when I was 13-14 years old.
The school was connected to the city's network somehow so the IT department
was not very happy. They figured out it was me but were nice enough to just
say "if you ever want to test something like that again, just check with us
first" and I got an internship with them the following year. I'm very grateful
for their gentle and pedagogical approach.

~~~
sandos
I was about 17 years old and IRC was very hot at the time in school. We had a,
for the time, very fast 512Kbps internet connection, and I had gotten used to
be able to "ping people out from IRC", no idea why I would do that, but hey I
was 17. I was experimenting with ever bigger packets, until I hit something
very close to 2^16 but not just.

Until the friendly admin just suddenly stood behind me, tapped my shoulder and
said: "If you step away from the computer, let me look at what you did here,
there will be no repercussions for doing shit you were not supposed to on a
school computer".

Turns out I managed to send a "ping of death" and the network gear in our own
school had just died. Internet was so unreliable back then that I just assumed
things were a bit slow....

~~~
ineedasername
I like this approach. Kids are going to test their limits, try things out of a
natural curiosity of "what if..." without always understanding the
consequences. The whole "just because you can doesn't mean you should".
Dealing with it as a teachable moment seems the best way to go for minor
infractions lime this.

------
nisa
Not a loop from hell, but let's not stop the procrastination loop: The 500
miles sendmail story still fascinates me, to this day:
[http://web.mit.edu/jemorris/humor/500-miles](http://web.mit.edu/jemorris/humor/500-miles)

there is even a faq: [http://www.ibiblio.org/harris/500milemail-
faq.html](http://www.ibiblio.org/harris/500milemail-faq.html)

~~~
justwalt
I thought of this story a few weeks ago when I was using VS Code on an old
machine and couldn't get the linter to work on longer files. Very similar
problem and solution.

------
krylon
At work, we had a mail loop once; a girl from accounting had ordered toner for
our printers last thing before she went on vacation and enabled an auto reply
on her mailbox.

Five minutes after she left the office, the order confirmation arrived, sent
from noreply@random.office.supplies.whatever. Her mailbox sent an auto-reply.
Our mail server sent her an error message, telling her that her reply had not
been delivered. To which her mailbox happily replied with the same auto-reply
mail...

When I walked into the office on Monday, her mailbox had accumulated around
50,000 of those mails. Fun times.

~~~
duxup
In the early days of "exchange" ... the email client. I discovered auto
replies.

Very first thing I thought of was "What if two clients have auto replies set?"

So naturally I set one up, my cube neighbor set one up. I sent an email to my
cube neighbor ... and fairly quickly email was down company wide for a good
afternoon.

~~~
peteradio
Why didn't anyone say anything when they were implementing this feature. It
seems kind of obvious, not even with hindsight.

~~~
duxup
IIRC rules such as out of office (I'm not sure "rules" really existed). Were
client side only... the server didn't have the option (or at least it wasn't
automatic) to evaluate bad rules / out of office.

------
jtokoph
Back when I was working at an SMS related company I managed to accidentally
infinite loop sending myself text messages. The aggregator and AT&T had no
issues, but my iPhone became completely unresponsive to touch as I saw the
notification badge quickly ramp up from 1 to tens of thousands. If I remember
correctly I had to wait a few hours for the messages to be delivered and
processed by the phone and then wiped my device.

------
minikites
A similar story from Microsoft:
[https://blogs.technet.microsoft.com/exchange/2004/04/08/me-t...](https://blogs.technet.microsoft.com/exchange/2004/04/08/me-
too/)

>So those 15,000,000 email messages collectively consumed 195,000,000,000
bytes of bandwidth. Yes, 195 gigabytes of bandwidth bouncing around between
the email servers.

~~~
bradknowles
That’s pretty good!

Almost as good as breaking all internet e-mail for the whole world in August
of 1996:
[https://www.theregister.co.uk/2018/04/16/who_me/](https://www.theregister.co.uk/2018/04/16/who_me/)

------
ineedasername
I once made a mistake, but it wasn't a just a loop, it was recursive. I was
working in an old version of FOCUS, a reporting language, mixed with DCL.
(This was a VMS environment)

The details are fuzzy now, but IIRC I misplaced a single "GOTO" line of code.
The result was doubly recursive: Each person on the list got the message once
for every person on the list, with the content of previous recipients
concatenated to the message.

So: person 1 got the message once. Then person 1 & 2 got the message again,
only person 2's content was appended to person 1. The person 1, 2 & 3 got the
message, with 3 appended to 2, and 2 & 3 appended to 1, and so on.

I was very lucky that it was an old VMS environment with low disk quotas, so
my "sent" folder ballooned before the full list could run. I think I got
through about 130 recipients, which came out to about 8000 messages.

------
maaaats
Not the same kind of loop, but a few years ago I had to quickly fix email
sending in some PHP service. Being rusty in PHP, I at least managed to write a
for loop iterating over the list and send mails. Or so I thought. I had forgot
"$" in front of the "i" in the for-header, making it never terminate, since of
course that's still valid php.

Luckily it was only my private email in the test-list. But I still got so many
thousands of mails my Gmail account was blocked for days. Also lucky this
didn't affect the delivery of our future mail (not blacklisted).

------
ob2
There was a time when overnight I received 200 copies of an email in my inbox.
The emails were sent manually, typos, delays and all. Very industrious
individual.

I was discussing this with a colleague of mine the next day and it transpired
that they'd recently taken delivery of two DEC Alpha workstations. They were
both sitting on his desk, being configured for actual work.

We set up a system using .forward files where DEC Alpha 1 would forward an
email to DEC Alpha 2. On DEC Alpha 2 we set up the same in reverse.

We sent an email to DEC Alpha 1 and CC'd in our individual.

We let it run for 5 seconds.

In that time the two workstations, doing nothing other than sending email,
managed a combined 25k of messages.

We never did hear from that person again.

These days when I think back about that, it was probably one of the early DOS
attacks. We were a little shocked how well it worked.

------
greendesk
Some years ago I run into a e-mail loop.

An e-mail request is sent to a marketing e-mail service. For a specific e-mail
notification, we do not receive a confirmation from the marketing e-mail
service that the e-mail reached the recipient. So our service re-sends the
e-mail.

This goes on for several days. It turned out a filter did not let the
notification reach our end. By the time we figured it out, someone received
thousands of the same marketing e-mail.

------
tuzakey
This reminds be of the classic Microsoft Bedlam DL3 story:
[https://blogs.technet.microsoft.com/exchange/2004/04/08/me-t...](https://blogs.technet.microsoft.com/exchange/2004/04/08/me-
too/)

------
YeGoblynQueenne
You can still do stuff like that today. My friend broke Yahoo and gmail.
Unfortunately, we didn't document the procedure very well, so the following,
from my personal notes on the incident, (with handles changed to protect my
friend) is all I got:

    
    
      How name_surname broke yahoo mail (and also gmail)
      ==================================================
      - 21/03/17
      > She doesn't quite remember what she did exactly but:
        a) She went into nickname@yahoo.com and added another yahoo mail box-
           name_surname@yahoo.com.
        b) This didn't seem to work (?) so she went into name_surname@yahoo.com
           and added nickname@yahoo.com as a mailbox.
        c) Then she went into gmail and added nickname@yahoo.com to it
        d) And finally she imported mail and contacts from both yahoo accounts
           into gmail.
    
        > The result: in nickname@yahoo.com you can see the name_surname@yahoo.com
          mailbox which contains a nickname@yahoo.com mailbox, which contains an
          name_surname@yahoo.com mailbox, but also nickname@yahoo.com contains a
          mailbox called nickname@yahoo.com and name_surname@yahoo.com contains a
          mailbox called name_surname@yahoo.com. And in gmail, all the email
          imported from nickname@yahoo.com and name_surname@yahoo.com have a very
          long listing of all the mailboxes in which they exist, which is the two
          yahoo mailboxes repeated several times. That's _many_ times.
    
        > Of course, when she tried to delete the mailboxes, whereas she only had
          5 messages it looked like there were 35. They were all duplicates,
          obviously.

~~~
jsjohnst
It’s hard to read on mobile due to your formatting, but based on knowledge I
have of both (and lack of details in your story), I think the most your friend
can claim is “broke my ymail/gmail”, not broke either service.

~~~
YeGoblynQueenne
Sorry about the formatting. Yes, she broke it for her, not everyone.

------
mixedmath
I love these sorts of stories. This is also one of the things that HN seems to
aggregate and bubble up better than any other forum/aggregator for me.

Thank you for the link, and for the post on your site.

------
zentiggr
Somebody managed to find a low level PDL that referenced nearly everyone in
the company globally. Just the reply-all "I'm not sure this as anything to do
with me, please take me off your DL" mails forced the servers to be taken down
and nearly five hours of global mail handling was wiped clean... it was about
seven hours before accounts started acting normally again. Sorta amusing from
the random employee view.

~~~
saagarjha
This sort of thing was surprisingly common at my old job. Someone would find a
list that seemed relevant, ask a question on it, and not realize the list had
as a sublist something like basically all of software engineering. Then people
would reply-all to that asking to be taken off the list that they didn’t need
to be on…

------
brazzledazzle
A few years back I had mail to an address forwarding to a team at a different
company. It was used to cc them on tickets, basically so they could make
changes on their side when certain requests came in. At some point they
decided to forward these directly into their own ticket system. Each email
from our ticket system would forward and land in their own ticket system that
would strip off the identifiers from our ticket system and create a new ticket
which would reply to our ticket system, creating a new ticket on our side and
so on. Unfortunately that queue on our side wasn’t in use by humans so no one
saw the exponential mail storm building. Many mail servers have relay loop
detection based on IDs in the headers but reply loops are a different beast.
Once we realized what was going on it was simple enough to black hole the
various addresses involved but it was pretty cool to watch our ticket and mail
systems DOSing each other in real time.

------
brian_herman__
I love these kind of stories this should be on bash.org

~~~
dbrgn
Then you should check out [https://dbrgn.ch/stories-from-the-
internet.html](https://dbrgn.ch/stories-from-the-internet.html). If you know
more stories that should go on the list, let me know :)

~~~
dTal
Holy crap, the series surrounding the 'recorder' one is the most gripping and
well written tales of combined technical and political achievement I've ever
read on the internet. True hacker porn. Start here:

[https://www.reddit.com/r/talesfromtechsupport/comments/qrqsa...](https://www.reddit.com/r/talesfromtechsupport/comments/qrqsa/ten_minutes_a_day/)

------
stcredzero
I knew immediately .forward files were involved, before I even read it. I had
done something similar and brought down my college's email system back in the
80's.

------
pbiggar
I remember a story from my tech support days: someone forged an email to
support@eircom.net from support@indigo.ie, both of which had auto-responders.

------
mike-cardwell
When I set up an autoresponder in my .sieve file, I specify ":days n", where
"n" is the minimum number of days between allowing an autoresponse to be sent
to the same address.

This isn't fool proof though. It would fail if the other side of the loop
didn't have a similar defense, and also sent each email response from a
different address.

------
jpmoyn
This is a great, hilarious story. Something about IRC makes for a really nice
story telling platform. Thanks for sharing.

~~~
blakesmith
If you like the IRC format, check out [http://bash.org](http://bash.org) \-
some pretty hilarious IRC logs from a glorious time in Internet history

~~~
snazz
There are some gems on there, like this one:

#127039 +(13215)- [X] <wolf> 1\. Save every Free Credit Card Offer you get,
Put it in pile A

<wolf> 2\. Save every Free Coupon You get, put that in pile B

<wolf> 3\. Now open the credit card mail from pile A and find the Business
Reply Mail Envelope.

<wolf> 4\. Take the coupons from pile B and stuff them in the envelope you
hold in your hand.

<wolf> 5\. Drop the stuffed to the brim envelopes in your mail and walk away
whistling.

<wolf> I have now received two phone calls from the credit card companies
telling me that they received a stuffed envelope with coupons rather then my
application. They informed me that it they are not pleased that they footed
the bill for the crap I sent them. I reply with "It says Business Reply Mail"
I'm suggesting coupons to you to ensure that your business is more successful.
They promptly hang up on me.

<wolf> Now, I did this for about a month before it got boring, so I got an
added idea! I added exactly 33 cents worth of pennies to the envelope so they
paid EXTRA due to the weight. I got a call informing me about the money, I
said it was a mistake and I demanded my change back. After yelling at the
clerk and then to the supervisor they agreed to my demands and cut me a check
for the money. I hold in my hand at this very moment a check from GTE Visa for
exactly 33 cents.

[edit]: formatting

~~~
yerasame
Oh, wow. If they have no recourse, I think I'll start doing this, too.

~~~
atburrow
Years ago you could simply tape the prepaid business envelope to a 2”x4” block
of wood, a brick, or anything else heavy and the mailman would take it. Too
much fun...

~~~
mirimir
Yeah, postage on reply envelopes used to be unlimited in the US. In grad
school, there were lots of lead bricks around. So we did that once or twice,
for laughs. But you know, the lead was actually worth something. But not as
much as the postage.

That stopped working in maybe the 80s.

------
creeble
Still easy to do, even between a couple of hosts:

[https://imgur.com/DwbRfDs](https://imgur.com/DwbRfDs)

Embarassing, yes. Especially since it was ongoing for, um, months.

nullmailer really likes to send mail. But some MTA's don't like to accept mail
larger than a certain size. Ugh.

------
michaelcampbell
> I managed to bring down 13 CIA offices, all FBI offices east of the
> Mississippi, and the entire Southeastern university Research Network.

I'm skeptical. Even assuming this is true, would the CIA divulge this info?

------
jancsika
Would be fun to build a GUI app atop an event loop like that.

Subject: Fwd: mousedown 52 48

Then you could trigger interesting dialogs to the user:

Error: failed to receive a mouseup for 72 hours. Do you want to cancel
clicking? Click one (Ok) (Cancel)

------
cafard
In January 2017, NetApp had a reply-all storm. One would think that sysadms
would know better than to participate, but one would not be wholly correct.

------
empath75
I crashed lotus notes at a federal agency after sending a pirated dvd as an
attachment. I somehow did not get fired (late ‘90s)

~~~
lostlogin
A government agency with a file size limit above a couple of mb in the 90s?

~~~
empath75
It didn’t have a file size limit at all until I did that.

------
harikb
The infamous Rudy Giuliani tweet needs to be added here

------
lightlyused
log4net + java dev app server + debugging the db layer + database outage on a
Sunday = annoyed exchange admins on a Monday.

