I'm surprised no one has pointed this out; you can report them for paid/spam links in your Google Search Console. It doesn't solve the problem but it will speed it up, especially if it's a manual PBN (private blog network) and not 'web 2.0s' as they say in the SEO world (links from Wordpress, Blogspot, and other already-hosted varieties). Google is pretty good at finding how the PBN is tied together - usually by looking at the hosting, and it will kill their rankings for a long time.

Also make sure to check your inbound links for negative SEO attacks. Since the most recent algorithm updates will penalize page by page for negative/spammy links it's possible that's happening to you as well. Google likes to pretend like that negative SEO doesn't exist, but it's probably the easiest way to climb to the top of rankings if the others are unaware of the intricacies of SEO. Search console is the only authoritative way to find those, but ahrefs will generally update faster and give you a head start. (I spend about 5 hours/week disavowing negative SEO links, and that's after I mostly-automated it).

In the meantime, you might want to put a little more effort into getting links pointing to you, if for no other reason than to not leave yourself in such a precarious position.

Feel free to email me and I'll look into it if you need more help. (I am over SEO at a YC startup trying to fix payday loans https://www.semrush.com/info/lendup.com+(by+organic), so as you can imagine we've seen it all... I hope).

https://techcrunch.com/2013/12/25/google-rap-genius/

That is a really interesting example. They had the best product by far. Google penalizing them is almost like penalizing its own users because they provide a much better product for lyric searches, in my own opinion. Every other lyric site before them was generally full of spammy ads with no added value, but Genius actually tries to add to the experience. Some songs more than others, but still a world away from the old lyric sites.

https://technical.ly/brooklyn/2017/03/16/genius-layoffs-verg...

I do agree that their product is the best by far; they're my go to site for lyrics.

https://www.semrush.com/info/genius.com+(by+organic)

http://www.theverge.com/2017/3/15/14924238/rap-genius-web-an...

I have had to help clean up one or two sites.

You don't accidentally create a PBN. It's more than accidental crappy pages. It's automatically-generated link wheels with spun content and capitalizing on expired domains, etc.

Unless your quoting okay meant that the content is barely okay aka not good at all. Then the links the sites get could be crap too and some other factor could be the reason for the penalty.

By "okay", I mean content written by someone in their first language, with intent to be useful. Not necessarily great, but not intentional filler.

The problem is that many of the larger "link networks" are controlled by big brands.

But if you know what you're doing, disavow is awesome. It works.

Problem for me was I didn't know if I was getting hurt by the spammy links or not. You don't know if you have a problem or not.

The tricky part is links that look spammy to you can be viewed by Google as legitimate. So as a practice I only disavow the spammiest of spammy links.

I run a few sites and have seen competition pop up very fast, outrank me for a few months (one was even for almost a year), and then disappear completely.

You then have a choice, ride these waves out, or play in the mud. At some point, Bad SEO will get caught and penalized/de-indexed, which for anyone that's not a household name, can entirely ruin your business if this is a main lead channel.

I hate to sound generic but I think you just have to stay on your path, and bolster your own link profile with good links. This might be a good opportunity to hunt mid/longtail queries to boost traffic in less competitive SERPs.

As a secondary, bigger picture thought, I think this is also an opportunity to hunt for more inbound channels. Relying too much on SEO is playing with fire.

I was chatting with a marketing exec in the real estate business and he said something I found chilling: "Google is transitioning to a pay for placement model."

You can see this in things like:

The move to 4 ads on top of SERPs - https://moz.com/blog/four-ads-on-top-the-wait-is-over

Making the Ad designation more subtle - http://blog.eyequant.com/blog/2014/03/20/why-the-new-google-...

Deep ad integration on mobile - http://www.wordstream.com/blog/ws/2016/05/24/google-mobile-f...

This is what makes competing against blackhat SEO so difficult. People who are doing blackhat SEO know this so they don't create huge amounts of links for a site that they want to stick around for a long time. They'll either rank a revolving door of sites that are easy to turn up, or they'll rank sites that feed to their main business (such as a "review" site).

There is really no defending against this - right now Google can't algorithmically detect these links with any sort of speed or reliability and SEOers can create new sites faster than Google can accurately detect them.

Google's search product is ripe for disruption, if anyone can get the resources together to take them down.

One of those resources would be facilitating a modification to the laws that make Google's line of business (internet search) legally shaky.

Can you elaborate a bit on this?

The copies of content that are stored in RAM are eligible for copyright protection, which means that just downloading a page without authorization, even before you've done anything with it, is an act of infringement. If you download 1000 pages from a site and it's determined that the license doesn't apply to you, you've got 1000 acts of copyright infringement just by storing them in RAM, let alone any "derivative works".

Then there is the issue of storing the page's contents for analysis and re-hosting portions of it for display. If you do image search, Perfect 10 v. Amazon said that although the storage and rehosting of images were unauthorized copies, they were a "fair use" under copyright law and therefore not infringing. Because fair use is an affirmative defense, this is not a very portable ruling, and it should be expected to be applied unevenly (and already has been wrt to RAM copies, at least in Ticketmaster v. RMG). The judge may find your non-Google image search non-transformative and therefore unfair and infringing. If Google had been sued on this point when they were a smaller company, they almost surely would have lost.

Then there is the CFAA, which makes it unlawful to access any server "without authorization" or in a way that "exceeds authorized access". The CFAA prescribes both criminal and civil penalties. Aaron Swartz was being prosecuted under this law for scraping publicly-funded research out of a paywalled database.

It can be, and frequently is, argued that accessing the site with an "automated method" like a scraper automatically "exceeds authorized access" due to clauses in the Terms of Service, which require users not to do this. This is often a successful argument, though there are notable exceptions.

On the flip side, robots.txt has been considered legitimate authorization for CFAA purposes in some cases, so if you're always obeying robots.txt, you will at least have that to fall back on. Search engines are well-understood and most people are probably not going to be haughty about your trying to index their site unless your scraper is super-aggressive and causes performance problems. In most cases, search engines usually don't need to crawl any single site in particular to be successful, so if someone really doesn't want you in there, you can just move on to the next one.

Google was designed for a self-published, late-90s web where the whole internet was accessible via URL. A modern search engine would recognize that the common man's voice is no longer accessible through the links they've placed on common-mans-site.com, but rather through their social media footprint on sites like Facebook and Twitter, and that accessing these profiles is critical to actually assessing what normal people find valuable online. The old method leaves you stuck in corpo-vision, since no one has a personal web site anymore.

Now, if Facebook or Twitter learned that accessing their data streams was critical to a major competitor's algorithm, they could sue under this cluster of anti-scraping laws and get access permanently shut off (unless, of course, you're big enough that the judges decide everything you do is fair and authorized, as they did in Perfect 10 v. Amazon, but you can't expect that to apply). If Facebook and/or Twitter wanted to launch a web search product, I have every confidence that they would have access to a much better dataset than Google.

This has a lot of negative implications for the future of the internet as an open platform, and drives the momentum back toward the bad old days of a keyword-based web controlled by a few central services.

I am not a lawyer and this is based on my layman's understanding. Consult an experienced lawyer specialized in this area before you believe anything I say.

Sadly the seo is very heated in that niche, so we went for long tail queries roughly 12 months ago.

Yes we know, we would love to get of the seo addiction, but for us others channels are not working as good (we do newsletters, FB, Twitter, Google, print etc. also).

For Google: https://www.google.com/webmasters/tools/spamreport

And you can submit feedback to Bing on a search results page -- be detailed!

It may not be fast, but it helps them.

It goes like this. You want to hurt a competitor so you hire an SEO agency to do a bunch of black hat stuff to the competitor's site. Then you report the competitor's site to google in hopes that they penalize them. Now google is put in an awkward position where the site that looks guilty is actually innocent.

For Google, the best place to focus is moving away from penalties for the most part and just trying to devalue spammy tactics. It'll never be perfect, but at least this way you avoid companies attacking each other.

[1] Sometimes though you placed a big and a small ad since some people want to deal with a small company and not a big company so you needed both.

It's best to avoid fighting this head on and stick to doing things properly - content, presentation and links. You may be under pressure now, but it's worth it in the long run. They will get their comeuppance.

If you're lucky you might even get an cryptic answer from the source during one of the Google Webmasters Hangouts - https://plus.google.com/collection/8926cB

(If you're interested I'm happy to have a five minute look in case I spot an opportunity you might have missed.)

Also, sometimes sites have a short-lived improvement on Google by using black hat tactics. However, sooner or later Google catches up and penalizes them. So be careful about copying them!

Organic marketing means you have 10 million dollars from other people and 500 instagram followers and begging your own employees for likes. It means you waste $10,000 per day celebrating over a brief 200 person bump in your daily active users. It means one retweet has you saying "we're blowing up!"

If they are keyword stuffed junk then your SEO people are right penguin slammed keyword stuffing hard. I know because I was using these tactics back then. On the other hand a keyword specific micro site with useful information can rank and likely eventually pass on useful pagerank to the main site.

The question I don't have the answer to is the period of time the site becomes beneficial. If you buy a new domain and setup a micro site I personally think it will take a year or more before the linked domain can benefit based on what I've seen. I base this on the fact when the micro site starts to rank itself and receives traffic. At that point I think it is safe to assume some rank is being passed onto the main site.

So if I were in your shoes I'd analyze the age of the bridge/micro site, quality of content and the ranking for the targeted keywords for that domain to decide if your competition is benefitting from it.

I also highly suggest investing in other inbound traffic channels. Having all of your eggs in the Google basket is risky. You will sleep better at night knowing your customers are coming from multiple channels.

I suspect the OP is in a non English speaking country where anecdotally there is a lot more spammy crap going down.

https://en.wikipedia.org/wiki/Doorway_page

Historically complaining on HN has been an excellent way to get the attention of people who care and can do something about things at these big faceless companies that skint on customer support.

I for one remember when Matt or Android devs etc would wade in if one of the community got their point across on HN.

Who is Matt Cutts' replacement?

had been on leave of absence

Why was he on a leave of absence?

Who is Matt Cutts' replacement?

Their SEO strategy included buying the Drupal plugin that adds Facebook/Twitter links to your page. Then they pushed an update that includes invisible text "BUY YOUR ACLS WITH US" and a link to their homepage. Google immediately ranked them first and they stayed for 3 weeks. Then they got blacklisted.

I purchased email-outreach linkbuilding from a company whose name rhymes with Winter-mitt harkening binges. Complete scam, and they sold us links and then sold the same links to our competitor. I actually have their client list and know that they are doing this to other companies.

END RESULT. Internet marketing is very important to our company so I decided to bring this function in-house. Last year we created 500+ high-quality inbound links. So hang in there kid, keep doing it the right way and things will work out.

But that's all that stops me. It's just safer to build slow and natural.

Even when these grey networks go down, that often just means those links no longer provide value rather than an actual penalty.That is an important distinction.

Since that site enjoyed higher visibility if there was any value to the site while it was in that improved position it likely secures more references,links,sales, income to support more legitimate content creation and traditional marketing.

If it was my personal business or side project and I was losing in the rather unreliable and unfair game that is the Google dance because the competitor was "cheating" I would absolutely return that tactic against them.

It is surprising that still seems to work but playing fair won't help you catch up unless they are so blatant that they get a manual penalty.

But, if you came to me as consultant I couldn't suggest that because it's unreliable, unethical and dirty. I would admit that it still works though.

Is there any reason you can't win via "ethical" tactics, are you truly creating content and resources that people might naturally link to and promote?

One of the sad oft-repeated stories is that if you create great content and resources ..the links will come ... but that leaves something very important out, only us webmasters,seo's,bloggers can generate links so unless your content appeals to that audience in addition to your consumers.. how are you going to get natural links and what is the carrot to the piblisher audience to do that for you?

If you can answer that and do influencer targeting / "blogger outreach" then you can play fair.

How many people have seen great backlink generation from getting their article or business here on HN? Can you get on here naturally? Then maybe you have the type of site,business,content to survive in natural SEO, otherwise maybe that shouldn't be your primary channel.

I certainly prefer PPC for "boring" topics, as you can eventually learn your all your costs, work off real conversion data and scale and test off of a relative constant.

I'm generally calloused to believing the first theory about why your website performs below a competitor is the correct one.

Have you made sure your site loads faster, do you know how engaged your users are and what have you done to increase that engagement.Is your mobile experience better than average, do your meta titles encourage people to click or has your seo battle descended into keyword stuffed titles? Is your on page stronger than them? Who says so? Have you audited your own content and architecture inside and out to make sure that rather than they have a leg up, you actually just have some deficit. Did both your sites start at the same time, have you been preserving your architecture as you rebuilt the site, do you check server logs for malformed Url requests, 404'so?

Might want to be sure some of these relatively easy comparisons and factors are in your favor before deciding that the link network you judge as valueless are providing that value that beats you.

My apologies if you have truly exhausted all those concerns, but I so rarely see that to be true.

In regards to "keyword heavy domains and keyword heavy content with all links pointing to their site", the search engines tend to favor on-topic content links, especially if those domains have trust and authority. I wouldn't necessarily call that a "black hat" strategy.

For lots of companies and businesses SEO at the level your talking about just isn't worth it. There are simply more effective ways to spend the capital and get the results you want.

Organic is great but it isn't the only game in town.

What does your company do (the vaguest of terms will do... Sells clothing, software as a service)

What does the rest of your marketing look like? Ad buys, email, product distribution for review or placement, and PR...

Can anybody recommend an up-today resource for PROVED SEO techniques (white, grey or black hat), like a book, vlog, or similar? There is a ton of information about SEO out there but it is quite difficult to filter the garbage from what it actually works.

At one point years ago I got half way through finishing an eBook/info product that was an SEO checklist. But half way through I felt that some people would find link building "spammy" no matter what. At that time, I was manually building large Web 2.0 networks with good quality content, and having it all funnel up to my main site. Even this can be seen as "grey hat" at the very least to some.

In the big picture, these strategies are very expensive and not effective at all.

We had about a one year head start and were the only place that provided the service we offered. Eventually, a (bad) competitor showed up. They engaged heavily in spammy SEO tactics and social media astroturfing and surpassed us within a couple of months, even though their product worked only 10-15% of the time.

We refused to engage in those spammy tactics. Partially out of principle and partially out of just not having interest in that muck, as it's not remotely attractive to people who are used to being honest and doing meaningful non-spam work. But that decision absolutely did cost us.

My advice is to think carefully about what may actually be over the line and what you've been socialized to believe is "uncool" or "distasteful". Then do those "uncool" things without making much noise about it (potentially even making some noise against it). Like everything else, there is the written rulebook and then there is the way things are actually done.

Business is competitive and if you're not taking every advantage you can, someone else is going to come in and eat your lunch. You have to be aggressive. That's hard to internalize (for me, it's taken multiple significant business failures), but it's crucially important to be successful. It's a dog-eat-dog world out there.

As for Google and SEO in general, every successful firm is at least "grey hat"; they're absolutely leveraging their portfolio as a "private blog network" (that is, a large, high-quality link ring) and otherwise regularly doing things that are borderline or off-book. If you grill them long enough, you can get close to having them admit this.

Google has been totally and completely gamed for years. You have to know that going in. If you're dependent on SEO, you're going to have to play the SEO games with all of their uncomely accouterments, or your business will lose out to someone else.

It's really that simple, and no one wants to admit it. All these people say these tactics only work for "a little while", but they're mostly wrong; unless you're being blatant, it seems to be quite easy to hide from Google's spam team.

The real thing here is that Google is ripe for disruption. Virtually every query you input is going to be based on someone's application of SEO strategies meant to trick Google instead of based on the ultimate relevance/quality of the content.

After that, I didn't spend much more time manually requesting links back. Whenever I did, I was always met with either no response at all or requests for money. Note that I was not really "link building" in the conventional SEO sense of emailing 1000 webmasters each week and getting 5 of them to add me back. I was just propositioning sites that I thought would have interested readers, under the naive assumption that they mostly wanted to be a useful resource to their readers.

I could've bought links in one of the various ways that's done: an SEO firm, a blackhat forum, etc., but I was naive and wanted to believe that word of mouth + social media + high-quality offerings and content would take us the rest of the way without making us a dirty "link buyer" who would surely get caught and destroyed by Google.

The competitor showed up and quickly demonstrated that none of that mattered (they actually didn't have any content at all, and ranked with about 4 pages total on their site). With a professional spam operation, you can rocket to the top pretty easily.

Even though SEO does not work well, my advice is don't buy links from link brokers...they won't work, and such services are almost always a scam. The websites selling the links have inflated/useless metrics... Hiring an SEO agency also not worth it, unless you have millions of dollars. Any mid tier SEO service is almost is guaranteed to be a scam. They will bill you $2,000/month for links they buy for $200 that you can buy yourself.

Buying links off forums worked well in 2005-2007 before Google cracked down and removed the pagerank metrics.

Zillow can spend millions buying links to rank for all the real estate and mortgage keywords, because they have a blank check. Few have that privilege.

In 2005-2007 SEO was much easier..now all those techniques no longer work well.

The problem is google changed their algos to discount links from many sites (such as Reddit and elsewhere); second, they have put much more emphasis on domain-niche authority. That means, if you write a viral article about daytrading and it gets shared and voted thousands of time on a social media sites, it may temporarily get a good SEO boost for those competitive keywords, but then slide back to page 8 or so. A the old, established daytrading sites established news websites will outrank, due to authority.

In which case, you'd probably be better off using your budget to purchase placement from Google[1], which causes the ratchet to tighten.

The future of Google Search looks grim and portal-like.

1. https://news.ycombinator.com/item?id=13915016

A more accurate statement would be:

2005-2007 SEO does not work that well anymore.

Outside of SEO, now popular is also fake social media profiles. What's the guidebook there to "Social Media Optimization" (I'm sure there's a real acronym for that by now), and how are Facebook/Twitter policing their datasets to prevent spam and fraud? afaict they're not, beyond basic anti-scraping methods like blocking anonymous proxies, etc.

Not trying to be aggressive, just really don't think that there is any reasonable way to do organic internet marketing anymore. It all boils down to either spam or advertisements. Spam (used broadly here to include astroturfing, PBNs, etc.) is the more effective solution by far, which means if you don't do it, you'll lose to the competitor who will.

I've come across SEO guys who insist they have the secrets to modern SEO, but on further inspection, they're just referring to using all of their clients' sites to link back to each other and insisting that they're the only ones who know how to do this.

It is true that an SEO firm's rate may be worthwhile if it buys you access to a well-governed link ring, but personally I'm skeptical of the abilities of anyone who has chosen to be involved in that type of career path. It's usually sales types who think they're good at computers, i.e., people who don't really have any idea what they're doing beyond convincing someone to pay them money.

If you meet the Buddha in the road, kill him.

My experience is that there's 3 things that work:

1. Write a lot. No matter how much research you do, you can't predict what will rank. You can increase the odds, but there are no guarantees. In the context of a software company (one of my previous jobs) do customer support and post in-depth answers to support questions. Write about users' success stories. Write about product features that might be hard to understand. Write about industry issues. Eventually you'll start to get some results. It's not an overnight thing.

2. Keep writing. If you think you're writing enough, you probably aren't.

3. Build relationships in you domain. Contact other companies serving the same audience who aren't your competitors. Post on message boards and Facebook groups without ever selling anything. Give away the most valuable thing you have: expert knowledge. People will read what you write and share/link to it.

I know it's much easier to just write the whole thing off as scams and hucksters, but good writing is what it's all about today.

I'll add a big caveat that there are MANY industries I don't know the landscape of. In the past, I've written in-house for largeish ecom and saas companies. Currently I work for an agency on a variety of clients, mostly smaller local businesses.

My personal experience with this, after paying for months of professional content writing, is that you get a few people posting links on their zero-PageRank Wordpress blogs and you get a few people attempting to share links or spread the word, but over-aggressive forum admins have filters that automatically remove all off-domain hyperlinks, and, in some cases, openly-hostile admins will remove any link to or mention of anything that isn't their own site. One of the largest forums in our niche had this policy; if you didn't pay them $5k for an ad package, any mention of your site whatsoever would be removed. You also get competitors, often large ones, blatantly ripping/reskinning your content without giving any credit, and then everyone shares the link from the better-known source instead.

Google claims that good content is key, but what they mean is that "good content will get links, so links mean content is good, so we just count links".

The premise that links correlate with quality is no longer valid in 2017. People no longer puts links on the internet to content they like, at least not in the format that Google expects and/or is able to access. The simple fact is that Google is made for the web past, and that it doesn't work on the web as it stands.

Our competitor had only 2-3 publicly visible pages that informed on the mechanics of their service and provided 0 related content. We had a blog that featured new posts with updates that were relevant to the niche almost-daily, including fully original high-quality photo essays with abundant explanatory text. Despite this, with a little bit of spam voodoo, they blew us away on SEO in every non-long-tail keyword.

No idea if it has any impact.