
Most of the Amazon SES IP blacklisted by SpamCannibal - abhisekumar
https://forums.aws.amazon.com/thread.jspa?messageID=538207
======
_asciiker_
I have been managing email servers for over 10 years, and it has gotten to the
point that I feel like blocking some of the most common ISPs. Seriously.

I am following all the best practises, hell, I even advocate them. It is just
that these days it seems not to matter if you have SPF, Sender ID, DomainKey
and DKIM, PTR, proper MX and even a normal to good IP reputation. There is
still no guarantee what you will be able to reach the inbox of the likes of
Gmail, Yahoo, Hotmail, etc.

I have been filling out huge forms for each and every major ISP for the past
year because one or two users mark a newsletter as SPAM.

Conclusion: There is no common standard because every major ISP can set their
own standards. This will eventually force everyone to use the same services
worldwide.

Where's the freedom of choice here?

~~~
kaoD
> one or two users mark a newsletter as SPAM.

I'm one of them. Those newsletters _are_ spam. I would _never_ sign up for a
newsletter and somehow I'm getting those too. If my intent was not to get the
newsletter, it's unsolicited mail by definition, i.e. spam.

Stop spamming me and I'll stop flagging you. Period.

How not to be flagged as spam:

\- There should be a checkbox clearly visible and it shouldn't be pre-checked.

\- Your "kind" product reminders are obnoxious too and I'll flag them as spam
as well. Did I ask you to remind me of your product? Nope. Unsolicited then.

\- If you ToS say I agree to receive mail, guess what? I don't agree, I just
want to try your product. I'll flag you in a breeze.

\- Social reminders like Twitter's trending around me or people I might know?
SPAM! I don't care if I can disable these, I didn't enable them.

\- You want to offer me discounts but I didn't ask for them? Flagged!

\- I submitted a paper to a conference and it got published? Dozens of "calls
for papers" in my inbox. Flagged, flagged, flagged, flagged!

\- Calling it a newsletter or adding a tiny "unsubscribe" link won't hide the
fact that it's still spam. I didn't click subscribe, I shouldn't have to
unsubscribe.

\--

EDIT: Woah, this seems controversial. Lots of up- and down-votes.

Dear product owners, downvoting me here won't change the fact that me (and
your fellow users) will still flag the shit out of your unsolicited mail. I
guess it pays if you keep doing it, but you should direct your energy far from
that downvote button and closer to "ways not to annoy my users".

~~~
saurik
It is users such as yourself that make spam filters have the kinds of silly
false positives being discussed in this thread: as a user (not as a product
owner; I _am_ a product owner, but I don't send e-mail, so I have no "skin in
the game": so, you can claim all you want that the people downvoting you are
all product owners, but some of us really do just feel you are abusing a
shared system to carry on some kind of personal vendetta) it really sucks that
I can't trust spam filters to not filter valid e-mail out of my inbox because
people like you are so trigger-happy (and in some cases, simply vindictive)
with the spam button... it is my sincere hope that services like Gmail are at
least sometimes smart enough to realize "this person is just being annoying"
and makes your spam votes meaningless :/.

~~~
kaoD
None of those are false positives. Spam is unsolicited mail. Did I solicit the
mail?

\- Yes: not spam.

\- No: spam.

And, since I never solicit promotional email to my personal address, my method
is 100% accurate.

In my opinion the companies sending unsolicited promotional email are the ones
gaming the system taking advantage of the _" well this might be unsolicited
but it's not v14gr/\"_ grey area.

~~~
just_bytecode
By this definition John sending an unexpected party invitation to his friend
Dave is spam.

edit: I guess spam is by definition commercial email. Still, I think it's
possible to have a business relationship with a company where it's acceptable
for them to send an occasional email that you didn't specifically request.

~~~
awda
Business relationships are different from personal relationships, and
"solicit," in this case, does not mean literally request the email. It is
perfectly reasonably to describe unexpected email from a friend as solicited
and unexpected product email from a business as unsolicited (even if you've
had contact with them previously).

It is unlikely that the business is your friend ;-).

------
kordless
> The SES team knows about the spam cannibal listings and is in contact with
> them, they say it's unlikely your open rate drop from 25% to 0.15% is caused
> by the SC listing.

SpamCannibal can cause the originating mail server to get caught in a 'tarpit'
by slowing it down. Given the AWS _CUSTOMER_ was sending a significant amount
of measurable email to a given destination server (which was running
SpamCannibal) it's possible the sending servers are being slowed down. In that
particular scenario, that would affect open rate over a short period of time.

------
sjwright
Let people run rampant on your IP range, and this is what happens.

I run a fairly large website, and I block all traffic from the likes of Amazon
AWS because it's full of dodgy bastards who think they're entitled to run
however many HTTP requests they like. Webmasters, look at your web logs. Don't
be surprised if the majority of hits are coming from bots pretending to be web
browsers.

~~~
guac
SES uses different IP ranges than those used by EC2.

~~~
sp332
It's the same idea though. Web sites might block crawlers in EC2, and mail
servers might block emails from SES.

~~~
billyhoffman
Our company offers a frontend web performance scanning SaaS product. We use
EC2 for our scanning boxes. I've found many of our customers's website filter
EC2 IPs. Its mainly from websites that offer a high demand product with a
large secondary market. (think ticket websites for concerts/musicals/plays,
airlines, hotels, etc).

------
mrsaint
bl.spamcannibal.org is notorious for a higher error rate (false positives).
It'd be crazy if a popular mail service provider like Yahoo categorized
incoming mail based on results from Spam Cannibal. If I added them to my MTA,
I'd rank them fairly slow to diminish the effect of their false positives. See
here:
[http://dnsbl.inps.de/analyse.cgi?type=monthly&lang=en](http://dnsbl.inps.de/analyse.cgi?type=monthly&lang=en)

And see here how to add them to your checks (and rank them accordingly) if
you're using Postfix:
[http://www.postfix.org/POSTSCREEN_README.html](http://www.postfix.org/POSTSCREEN_README.html)

------
ANTSANTS
If you were to remake email from the ground up, how would you solve the spam
problem while keeping it as decentralized of a system as it is now?

~~~
drpancake
Require a small fee to send an email, e.g. $0.01. Small enough not to matter
for legitimate users, large enough to make spam unprofitable.

Bitcoin or some other cryptocurrency would be ideal for facilitating micro-
transactions like this. Interestingly, the Hashcash concept was originally
designed to fight spam, and later became one of the important ideas that made
the invention of Bitcoin possible:
[http://en.wikipedia.org/wiki/Hashcash](http://en.wikipedia.org/wiki/Hashcash)

~~~
okasaki
That wouldn't work for mailing lists.

~~~
wpietri
One could add a proof of permission model to that to cover things like mailing
lists and other regular sources of email. Something like:

I sign up for a mailing list.

The mailing list mail server contacts my mail server.

My mail server presents to me a request for free delivery.

I approve it.

My mail server gives the mailing list server a secret token.

Each mailing list message includes proof that the message has been blessed by
the sender and that the sending server knows the token.

~~~
jrockway
What stops this server from then using that token to spam you?

~~~
wpietri
I'd do it with revocation.

Right now I give out tagged addresses to most vendors, and I know others do it
for lists. When a tag goes bad, I just route any further mail to my spam
trainer.

~~~
jrockway
But that means for every legitimate mailing list you're on, you get one spam.
And you lose the legitimate mailing list just because their token got
compromised.

Dunno, I feel like we should just charge for mailing lists too. Or use usenet!

~~~
wpietri
If I can get it down to one spam each time a mailing list server is
compromised, I'm ok with that.

Even with a charging scheme, some spam will still happen. Give the amount of
BTC stolen so far, and given the number of compromised computers that could be
used to generate cash, looks like there will be plenty of money to spend on
sufficiently profitable spam.

------
belorn
Blacklists are general a very effective method in handling spam, but its kind
of dangerous to use in a commercial setting. A few weeks ago SpamCop blocked
gmail for a few days, causing some "mild" issues for companies that depend on
email.

There is sadly not much options. Either I can accept more spam, or use
blacklist and put the control of the filtering in the hands of a third-party
with none of the responsibility attached.

------
mgkimsal
It might help the situation more if webmail providers provided actual
'unsubscribe' or 'hide' links in their UI instead of 'spam' being the only
feedback mechanism users are offered.

"unsubscribe" links vary in position, language and visibility in various
clients. Making something beyond "this is spam" part of most mail clients,
perhaps with reporting back to the originating sender, would help.

~~~
PythonicAlpha
Problem here is that particularly such "unsubscribe" links where used in the
past (and still are, I guess) to reassure spammers that somebody is there.
Because one problem spammers have is the quality of the addresses they have.
Many spammers use lists from dubious sources and a big number of addresses are
invalid. So, if they get an "unsubscribe", they know which addresses are
better and can deliver more spam to it ...

So many don't dare to use such links and rather click on spam.

The only solution could be some "trusted" functionality that goes via the own
mail provider of the receiver. But of course the mail provider can not simply
send information to the sender of the eMail .... So the thing gets
complicated. As much I learned, for spam clicks there is something like that
available ... some kind of trusted feedback chain that gives information to
trusted senders, that some mails where labeled as spam. Thus those senders can
(indirectly) adopt their eMail campaigns.

~~~
__david__
> Many spammers use lists from dubious sources and a big number of addresses
> are invalid. So, if they get an "unsubscribe", they know which addresses are
> better and can deliver more spam to it ...

That's only true of the dubious "viagra" style spam, where they got your name
from a list. I don't think those even bother with "unsubscribe" links any
more. I only see unsubscribe links from places where I've had to give my email
up to buy something or sign up to a site. Those are generally legit and most
of the techy/startup web sites will unsubscribe you immediately.

The next tier are the sites that unsubscribe you but take more than a week and
will keep spamming their dumb newsletter in the meantime.

The final ones are either broken by stupidity (it's amazing how many web
developers cannot grasp that "+" is a legit email character), or willfulness
and will keep spamming no matter what. I block these at the SMTP level with
503 messages (usually containing some personal insults and swearing) as soon
as they "RCPT TO" the unique email address I gave them.

~~~
PythonicAlpha
Luckily I managed to block much of the spam I got before, so I don't see the
current spam behavior. But some years ago, there well existed some non-viagra
style spammers that just put dubious unsubscribe links inside. I never tried
myself, but was warned that they use the information against you. I would
guess, that verified eMails have a greater value ... but it also might be
negligible now, since with bot-networks spammers don't need to care if they
send 1 billion or 10 billion eMails ...

------
notacoward
I know there are legitimate uses for something like SES, but it has always
seemed a bit like "Spam as a Service" to me. Ditto for every other service
that's designed around mass email, no matter how much YC startups might depend
on them to "improve their conversion rate" or whatever the buzzword is this
week.

~~~
cperciva
_Spam as a Service_

I'd call it "Sender Reputation Checking as a Service". Where said service is
paid for by the sender, but provided to the email recipient. Anyone can send
email directly; but knowing that email has been sent through SES and Amazon
hasn't killed the account yet provides a greater degree of trustworthiness.

In a sense, it's like a bond rating service.

~~~
notacoward
...and we all know how well the bond rating services performed their function.
Sorry, couldn't resist.

On a more serious note, it seems like the "greater degree of trustworthiness"
is only _very slightly_ greater. SES might be better than some server in a
domain nobody ever heard of, but it's still not as good as a provider with a
long history of responsible email use. Many people can and do block SES and
its ilk, as is the subject of this story, because the aggregate amount of spam
is so great even if the individual spammers are transient (like they care).

Amazon could raise the bar, thus raising their own reputation and thus making
the service more valuable to those who can still afford/qualify to use it.
It's probably just not worth their while to do so. I'm not even criticizing
them for that. I'm just observing that online business has a shady side, and
Amazon isn't afraid to partake.

------
driverdan
Considering Amazon themselves send spam this isn't surprising. They like to
send out product spam under the guise of account notifications with no opt-
out. The only way to remove yourself is to close your account. At the very
least they do this to affiliates and to Student Prime members.

------
adarsh_thampy
I had had the same issue with amazon SES. While Spamhaus PBL is not
necessarily a blocklist, due to the default configuration error, all the
emails we send out (from other providers like mailchimp) ended up in spam.
Finally, the issue got fixed by configuring reverse PTR.

------
lazylizard
are people not relaying their newsletters/unsolicited mails/spam thru some
antispam outgoing smtp gateway if they're concerned about being blocked?
filter it yourself before people filter u?

------
leccine
I believe in IP blacklisting. Why should my business be open to countries like
North-Korea or Somali if I don't have anything to do with those? Spam comes
from everywhere, but you can get rid of at least 50% by simply dropping
traffic from certain countries, so the expensive score based spam filtering
get cheaper. Unfortunately Amazon SES is the victim of cyber warfare and
spammers.

~~~
jzwinck
This sort of logic will hurt you with customers who are US persons living
overseas. I cannot tell you how annoying it is when as a US citizen and
taxpayer I am blocked from using US services because I am not physically in
the US. In 2014, where we plug our computers in does not define who we are.
Geolocation is not authentication.

~~~
MichaelGG
Not even just that, but where they ignore your Accept-Language and just make
up a decision on what language you should be using. Google has no way to fully
switch languages - even after going to Google.com and setting English, you'll
see images have tooltips in your "local" language.

Google's Play app does the same thing for a bit of their content. Certain
headers get localized, despite everything else in the app being English.
Netflix has the same problem, and then to further add insult, they send you to
non-English phone numbers for support.

The most annoying thing is that someone probably got a raise for these
"features".

~~~
frik
You can use [http://www.google.com/ncr](http://www.google.com/ncr)

[https://support.google.com/websearch/answer/873?hl=en](https://support.google.com/websearch/answer/873?hl=en)

~~~
MichaelGG
That doesn't actually fix it. When you start searching, the alt text for the
logo is still in the "local" language.

------
austerity
IP blacklists are a waste of everyone's time.

~~~
lazyant
Not my time and my clients'.

By appending in the Posfix configuration file line
smtpd_recipient_restrictions = ... spamcop and spamhaus , spam decreases in
like 95% without even touching your server further (spamassassin I'm looking
at you).

I you add greylisting you get rid virtually of all of spam.

~~~
eli
How much good mail gets mistakenly canned? I'd rather sift through a little
spam than lose something I wanted.

~~~
ScottWhigham
I think we'd all rather "sift through a little spam than lose something I
wanted" but my guess is that you've never run an email server. "a little spam"
is not what you will get - it will be orders of magnitude more spam than legit
emails. This is hard stuff - Gmail, supposedly one of the best, catches
between 2 and 10 emails a day in my "Spam" folder that aren't actually spam.
If I were to turn off the spam filter (if you could) in Gmail, I'd get 2,000
emails a day - of which 50 would be legit.

~~~
eli
I actually have run a mail server before, and now I gladly pay someone else to
do it for me :)

Point taken, but I've been on the receiving end of what I would consider
false-positive blocks by Spamhaus & co. They sometimes have policies about
what's considered spam that I don't think all their end users would agree
with. I've been blocked for having an IP address on the same provider as
someone else who allegedly advertised their website via spam. If you're
running a mailing list, dealing with the anti-spam stuff is at least as big a
problem as the spam it was supposed to solve.

