
A painful tale of SEO, spam and Google's role in all - slaven
http://scriptogr.am/slaven/post/a-painful-tale-of-seo-spam-and-googles-role-in-all
======
venus
My god, the quote from that "white hat" SEO guy just defies belief:

> There will certainly be webmasters out there who will strip you down to the
> bone asking for money in exchange of link removals. These are the most
> soulless snake oil salesmen on earth

To say that about webmasters, already victims of years of abusive SEO
spamming, when they then refuse to help an abusive site clean up its own mess
for free .. I have no words. Could anyone possibly be more of a self-
interested, myopic, egocentric prick?

My rock-bottom opinion of pretty much anyone who has anything to do with
active SEO is re-confirmed for the thousandth time.

~~~
SquareWheel

        "To say that about webmasters, already victims of years of abusive SEO spamming"
    

I suspect this comment was aimed at webmasters not who were the victims of
spam, but those that profited from it. It's those who run directory sites and
article farms that would accept cash to add your link, and now that people
want the links removed, they're double dipping. And these people are indeed
scummy.

~~~
Isofarro
If they paid to get listed there in the first place (paid link for the purpose
of better rankings), then it's fair game to pay to remove the ranking
disadvantage now. I call it an ethics tax.

Paid links were frowned on but not generally penalised. So sites took the risk
they wouldn't get caught or penalised. Now they got penalised. So if it was
okay to pay for a linkback, equally it should be okay to pay for the removal
of a linkback.

~~~
SquareWheel
Perhaps. I don't have much of an opinion one way or the other on if it's moral
to charge for something like that. But the Disavow Link Tool exists for a
reason, and webmasters can use it to manually remove those bad links. Though
they are encouraged to make an honest effort to contact webmasters first.

~~~
bradleyjg
That google 'suggestion' is directly leading to the deluge of crap in the OP's
inbox.

------
kkowalczyk
Blaming google for seo spam is not productive - spammers are legion and in
constant battle to game google's ranking system, whatever it currently is.
This won't change as long as google is used to search for things.

There is a solution to his particular problem: better forum software.

I don't want to trivialize the problem of writing spam-resistant forum
software but it's not such an insurmontable problem (this forum being a proof
positive for that).

For the reference: I've been running a fairly popular forum
(<http://forums.fofou.org/sumatrapdf>) for several years, using forum software
that I wrote.

I don't even require the user to log in, I don't require moderation for posts
and yet I had zero automatic spam (I attribute this to my unusual captcha
[http://blog.kowalczyk.info/article/16fw/Best-captcha-is-
exot...](http://blog.kowalczyk.info/article/16fw/Best-captcha-is-exotic-
captcha.html)).

I occasionally get human spam i.e. someone writes a post with the only purpose
of linking to some other website. I just hellban them after I see the post in
my rss reader.

And I didn't even write any special anti-spam code (other than hell-banning),
because I'm lazy. I can easily come up with simple ideas e.g. putting all
posts that contain links in moderation queue.

To reiterate: his problems were caused by a crappy forum software that didn't
do much to protect from seo spam.

I don't really know how current best off-the-shelf forums fare in this
respect.

I would rather not spend my time maintaining my own forum software so I have
high hopes for <http://www.discourse.org>. I'm sure StackOverflow had plenty
of spam problems so discourse people should understand the problem.

~~~
greggman
I'd attribute it to your custom software. Break phpBB and you can infect
thousands of sites. Break your custom software you get to infect 1 site.

I ran a forum using SimpleMachines. Turned on all the spam fighting stuff.
Didn't matter. Bots or Mechanical Turks made 10-30 new accounts a day. I tried
moderating the accounts but who wants to spend time every day to delete the
fefRFGR34@foobar.com account.

I gave up and shut it down.

~~~
JoshTriplett
If you see Mechanical Turk used for spam like that, you could report it to
Amazon as abuse and get the account(s) paying for it shut down.

I also wonder if Amazon has a content filter, to block URL patterns in Turk
tasks.

As for bots, IP banning goes a long way.

~~~
disgruntledphd2
Not if they really want your site. A place I used to work for was massively
targeted by them, and literally the head IT guy gave up banning because they
just rented new servers and continued the same stuff.

Essentially, like Jesus and the poor, spam we will always have with us (as
long as there's an incentive).

------
coderdude
One way to combat this problem is to deny new users the ability to post links
by replacing the link text with [removed]. HN has a karma system with
thresholds that must be passed in order to get additional privileges (like
down-voting). If you have a site that assigns karma to users (even if it's a
secret number) then you can set a threshold for allowing links to pass
through. It's not perfect but it's better than letting brand new accounts post
links, IMO.

Another solution is to hide all links when a page is viewed by a user who
isn't logged in.

~~~
a5seo
nofollow all external links

point external links through a redirect like /out?to=<url> and put /out in you
robots.txt

change your url structure and simply 404 the pages with spammy links

~~~
jtheory
These solutions assume that the spammers have human beings who are actively
checking that their spamming effort worked on your forum.

I have a forum where I've been battling spam for years, and since the
beginning all external links have been nofollowed... AFAIK this made no
difference whatsoever.

~~~
a5seo
That's partly true. But the reason a forum gets on their radar in the first
place is by failing to follow those safeguards.

The way these guys operate is they write scripts to crawl the SERPs and find
sites that are ranking, then they look for weaknesses (i.e. outbound followed
links that don't go through a redirect).

Or they look at their spammy competitor's backlinks with AHrefs, Majestic,
etc. and discover the forum that way. Then they help themselves to a link.

------
mutagen
I saw the SEO post he's referring to (it was one of the articles in SEOMoz's
top 10 monthly email). Site owners don't realize the amount of work and
headache they cause forum communities when they contract out SEO work without
an understanding of what that work entails. Or they do and just don't care.

Google's Penguin update didn't deter the spammers, either. Here we are nearly
a year later and I'm still cleaning out accounts created en masse by XRumer or
other bots.

------
will_critchlow
I can't put words in the mouth of the SEO quoted, but (unless you know he was
one of the people making removal requests against your site) I suspect he is
not talking about you or sites like yours.

I should add that I don't agree with the rhetoric btw, but I think he is
targeting a different kind of webmaster.

I think he is referring to webmasters who sold links (knowingly outside the
guidelines) for years. They would previously have instantly removed the links
if someone stopped paying.

As soon as Google stepped up their game and removing those links was
important, those same webmasters started charging to take down the same links.

I would personally point to the irony of this (google creating a market that
enriches people who have been abusing their system for years) rather than
calling it immoral. Ymmv.

Hope that helps clarify some things and I hope I'm not distorting the guy's
real meaning.

(written on my phone. Please excuse typos).

~~~
dhimes
I'm wondering about the graph on Gary's site: what type of business' fortune
is linked this tightly to Google's rankings? Is it a particular category, or
are there several categories of businesses that suffer from (or leverage)
this? That's a pretty frightening situation to be in, IMO.

~~~
will_critchlow
Oh, there are many businesses _at least_ that exposed. I personally wouldn't
particularly relish running one, but that's another story...

------
8ig8
How about taking inspiration from D&B[1]?

Don't charge for link removal; charge for _priority_ link removal. Same day
service: $1,000 per link. One week: $500. Etc.

Free link removal: First-come, first-serve at your own leisurely pace.

Sell the old forum to someone else and let them handle the requests.

BTW, PocoMail was a godsend back in the day. Thanks.

[1]:<http://news.ycombinator.com/item?id=4994246>

~~~
slaven
Props on PocoMail comment :)

------
nkuttler
So this "SEO" who helping his clients to "clean up their profiles" doesn't
even know how to disavow links [1]. Well, it's not surprising that somebody
who hired spammers once would hire another idiot later.

[1] <http://www.youtube.com/watch?v=393nmCYFRtA> and
<https://www.google.com/webmasters/tools/disavow-links-main> (login required)

~~~
JimWestergren
Well Google requires you to contact the website owners and do everything you
can to get the links removed BEFORE you submit a list of disavow links. Google
are supposed to check that you really did remove a large part of the links
before disavowing.

------
phasevar
This is a great post that I'm sure rings true with many 'a webmaster. Time is
money. There's no two ways about it. If you want a webmaster to modify their
site in your favor, you should be willing to compensate them for their time in
doing so.

------
jedireza
You could shoot the hostage and add rel='nofollow' to all outbound links. You
should be able to programmatically do this.

Then with an auto-reply (or 'link policy' page) inform 3rd party sites that
the link-juice (good or bad) is no longer flowing.

~~~
Scramblejams
Does this constitute a long-term solution to the problem for active
communities? That is, default to rel='nofollow' for an untrusted class of
users, which presumably includes all the spammers?

~~~
33degrees
that only works if the spammers realize their links are no longer effective;
I'm not certain that that will actually happen...

~~~
tracker1
If you have a relatively small site.. curating content isn't too bad...
limiting the tail of articles so that comments can only be added for X days
helps with that. Only allowing logged in users to comment (and making login
easy via oauth sites like facebook/twitter/g+ etc) I kind of like Disqus, but
would rather have something more home-grown.

------
codezero
I'd be tempted to ask for money as well, but I feel like at the very least,
you should ask for an apology. They may not have known that their SEO agency
was using shady tactics, but since times changed, and it's obvious that they
put bad links out there, it seems like an apology would be more than a token
gesture.

------
javajosh
The problem here is the work. The solution is to write a program. Have the
white hat SEO people write a program that spiders the site with admin
privileges and removes offending posts. It should come with a "dry run" mode
that lets you spot check it. When they get it right, you can run it for them.
It's a win-win: your forums get cleaned up and they did the work.

~~~
bradleyjg
It's not really a win-win. Any time he spends on this is a loss from his point
of view since the SEO's client already ruined his forum. It's some chutzpah to
get annoyed when your victim refuses to help you hide your misdeeds.

~~~
javajosh
If they do the work to undo the sins of the past, it helps the OP recover the
value of his forums for far less effort than if he'd undertaken the job
himself. I'd call that a win.

~~~
argonaut
No, removing all the spam is not going to recover _the community that fled
long ago_.

------
hakaaaaak
If your business relies on SEO or is affected by Google search rank in a
critical way, it may not be a viable business long-term.

Yes, that means I'm talking about a lot of online businesses.

The intelligent thing to do is sell a product or service that has value on its
own and neither relies on SEO nor is it likely to be blames by other sites or
companies for lowering their SEO.

I feel bad for these folks, but if you are planning on starting a business
that doesn't really provide much value on its own that is identifyable outside
of the roach motel of SEO, then you are headed into the ocean in a dingy with
a small outboard motor, imo.

------
arn
I've received similar emails. I've also been tempted to charge a fee for
removal. I just ignore the emails.

------
Father
Link removal requests can also be malicious. A blackhat seo will check his
competitors domain to see if there's a spf record, if they're signing their
mail, and if there's a catch-all (simply by checking if random mail is
accepted). If there's neither they get a list of backlinks, from public web-
crawls or sites like ahref, and request these links be taken down by sending
mail with spoofed email addresses.

------
speeder
This is sad, sad, sad. I wonder how much awesome communities died that way. I
stopped visiting Orkut when communities there got overran with spam. Also the
same apply to some Usenet groups and Google Groups I used to like.

------
lutusp
The solution is obvious -- write a routine that automatically goes through the
entire forum database and disables all the links -- leave the names, but
rewrite the links so they're just text, not hyperlinks. Sort of like:

s!<http://!!g>

The above deletes the "<http://> prefix, but leaves the original destination
name, in case anyone wants to object that their post has been edited after the
fact. So technically, it's no longer a link back to the originating site, but
it's otherwise unchanged.

No human intervention required. Problem solved.

------
bambax
There must be a programmatic solution to this problem.

Do some outbound links have value on this forum? If not, then you could remove
all links, or remove the "link" part of the link (change @href to text).

If some outbound links have value you need to identify those, and it's more
complex, but a Bayesian analysis of posts should be able to score posts on
their "spaminess" and remove the links on only the most spam-like comments.

There may be some false-positive doing this, but since no information is
actually removed (only the links, not the content) it should be quite ok.

~~~
robryan
Random links on forums already have pretty low value. I would assume that
forum spam has come down now that this is the case.

As the article states, websites are looking the clean up the spammy inbound
links as google has threatened to deindex sites that don't make an effort to
do this.

------
rizz0
There's a difference between buying links, which Google actively started
prohibiting at a certain point, and the automated forum spam you are
mentioning. The latter is done using software like XRumer, and has always been
penalized by Google. The former wasn't prohibited in the beginning, when link
deals were often mutual agreements, most of the time involving a traffic
component as well.

The snake oil salesmen that are mentioned, are the ones who actively
participated in the scheme by selling links and making a buck, and are trying
to make another quick buck now that the rules have changed.

Moreover, some links aren't even paid at all, but just look manipulative. For
example, if you developed a wordpress theme, and your link is in the footer of
tons of blogs, you might get penalized for manipulating the anchor text of
your links in a non-natural way. In those legitimate situations, webmasters do
have a moral obligation to cooperate.

I don't think anyone would think that of an honest entrepreneur being spammed
to death by link spamming software.

On a side note, there are plenty of forums on the web that have survived the
spam wave, if it were core to your business, you could have protected
yourself.

------
ajenner
Wow, so Google penalizes a site if a link to that site shows up in spammy
pages? That seems like a new business model for black-hat SEOers: "hey, nice
site you have there - it'd be a shame if links to it started appearing all
over my spammy network - $$$ will make sure that doesn't happen." Really
search engines should just give zero weight rather than negative weight to
links from spammy sites.

~~~
Isofarro
"hey, nice site you have there - it'd be a shame if links to it started
appearing all over my spammy network"

Good way to clue Google in to this network, and then drop it from the index.

"Really search engines should just give zero weight rather than negative
weight to links from spammy sites"

That's what was happening before Penguin/Panda - and it wasn't working.
There's no risk to link spamming. Now there is.

------
BashiBazouk
Here is a question for those who know: is there anything forum regulars can do
when we see spam posts beyond letting mods know about it?

I was active in a now dead forum that would get hit once or twice a week. The
mods would clean it up with in a half a day but until then those posts would
just sit. Does the Google web crawler consider words in replies to the spam
post? I used to reply occasionally with words like: scam, fraud, got ripped
off. I have always wondered if I was wasting my time. A few times I checked
the link to see if it went some where legitimate (Google the base url), if so,
I then searched the site for an informational web form. If there was one, left
a message that their SEO company was using sleazy methods with a link to the
forum post.

Google ought to make a code phrase that forum users can use to red flag spam
posts. Though some of the posts were for Japanese and Chinese sites. The spam
might not have been meant for Google but other search engines...

------
Nikolas0
I had the same problem myself. In fact I still run a couple of forums. And I
get at least 3 "remove my links" mails per day. I guess you can't ask them
money, but they already cost you money (to maintain their spam and since the
panda update read their emails asking you to remove it) so I guess the best
solution is just ignore those mails.

Regarding anti spam I am afraid there is nothing you can do. Real humans will
create an account for 0.01$ and they will post anything (I tried adding custom
code in the post code as well, but those signups are not necessarily bots) In
fact they even post if you don't allow them to post links (they post the urls
with no http)

Next solution I'll try is social integration. Maybe that would work for some
time but even this way spammers will find their way to create thousands of
crap accounts in FB, twitter, etc.

------
JimWestergren
Place a meta noindex, nofollow on the whole forum or at least the infected
areas such as memberlist and user profiles. That could be done within 15
minutes and then you can answer with a standard reply that you did that to all
emails you receive about removing links.

------
iuguy
Does anyone else think that the author should set up something like mailchimp
with an autoresponder to deal with the spam? Something that explains his
policy, why he doesn't respond to emails, why he won't remove the link without
charging them?

Many of the people who will have these spammy links on his forum will have
paid one of the more scummy SEOs out there to raise their SERPs. The people
emailing may not have originally been aware that the links have been put up.
By having an autoresponder address for people to mail to this should alleviate
the spam for all but the dumbest of people (that can't read what to sign up to
to get a response, or those that fail to read a clearly defined policy via
email).

------
rurounijones
Would love to hear feedback from the white-hat to this article.

Maybe get a decent discussion going from both sides. It does seem that the
white^hat's remarks are very harsh, I wonder if he has honestly never thought
about it from the other side's point of view.

------
halcyondaze
I use XenForo and it's relatively simple to stop all forum spam for the most
part. I just set up a custom captcha that only people who are interested in
the topic would know the answer to, and then use XenForo's built in spam
catching system for the rest...though I haven't had anything get through yet
once I made the changes.

Also deleted forum footprint to dodge people scraping my forum off of Google,
so that takes care of about 99% of all spam attempts. Someone would have to
custom register accounts on my forum to get in, after which they would be
destroyed by the spam catcher :)

------
edwinyzh
Maybe we should not blame search engines, but SEO really has bad effects on
the Internet...

PS, due to spams I've closed this forum (<http://writingoutliner.com/forum>),
a online forum for my 'Addon documents organizer for MS Word'.

And I regret I use BBPress as the forum software, because new 2.x versions are
not compatible with the old 1.x versions...

------
Tichy
Maybe it could be scripted? Site owners would have to place some proof of
ownership on their site (to be generated on the site of the forum owner - see
Google web admins site verification), then a script could detect the faulty
links on the forum and remove them automatically.

------
mesozoic
Oh man that is some sweet irony.

Just wait till the next waive of spam from SEOs trying to get competitors
penalized then the competitors firing back and contacting you. Google stirred
up a hornets nest because they couldn't figure out how to effectively devalue
these links completely.

------
Aardwolf
There exist very popular forums that receive no such spam.

Hacker News comments also seem free from it.

Any idea why it is that some forums get targeted with 10000's of spam
accounts, while others don't? What is the trick to protecting if even capchas
don't work?

~~~
mesozoic
They mainly use a program called xrumer to spam and it mostly works with your
typically install of vbulletin or other popular forum software so custom
software won't get hit. Kinda security by obscurity.

------
mangostache2
Why are people getting bent out of shape over some bad links? Can't you just
disavow them in Google Webmaster Tools?

