
RIP Google PageRank score: A retrospective on how it ruined the web - adamcarson
http://searchengineland.com/rip-google-pagerank-retrospective-244286
======
jandrese
I feel like no matter how Google or any other search engine ranked pages, SEO
firms would be there to game the system and make a mess of the web. Making
Pagerank visible to people with one particular toolbar does seem like a fairly
major misstep on Google's part. The majority of the people who would really
care about that are the kind of people you shouldn't be encouraging.

One of the obnoxious things about SEO is that if one person is doing it
everybody has to do it. It's not necessarily enough to simply offer a better
product at a better price. Luckily Google does try to reduce the effect of
SEO. I notice for instance that StackExchange almost always beats out Expert
Sex Change links these days.

~~~
hire_charts
I'm not sure how much of it is that google tries to reduce the effects of SEO,
or that they just keep moving the target. Every time they change their
algorithm it seems like a bunch of sites who were practicing one strategy
disappear, and before long a new strategy is discovered causing new sites to
rise to the top.

Ultimately Google's ability to quantify both quality and relevance will
converge on _actual_ quality and relevance. In other words, the way to "game"
the system and practice SEO will be to create a site that is actually relevant
with high-quality content. But we're not there yet, and who knows how close
we'll actually get.

~~~
brosirmandude
This is definitely what is happening. I work in SEO and follow the industry
closely, but the SEO industry itself is a giant echo-chamber. But essentially
what it boils down to is:

\- make something your audience wants

\- make content your audience wants to share

\- "earn" links by creating useful resources & content

Essentially, the way to get great rankings in Google is by actually having a
website that people want to go to and is relevant to their interests. Trying
to game the system is much harder these days, and honestly for the most part
isn't worth the effort.

~~~
vehementi
Well a bad part of it is that "something your audience wants" has proven to be
"awful buzzfeed clickbait titles", so human shittiness trumps value in this
way

~~~
Trundle
You may just be working with a poor definition of value if you consider giving
people what they want to be at odds with it.

~~~
vehementi
Is heroin a very high value product?

~~~
unchocked
Absolutely, judging from what people are willing to pay.

Arguments against heroin are invariably rooted in negative externalities. I
have never heard anyone argue that there is no demand for heroin.

~~~
vehementi
The "value" of heroin comes from it tricking your brain with chemicals that
you need it and will die otherwise. I WANT IT GIVE IT TO ME == value, without
follow up questions? Similarly buzz feed tricks you with catchy titles. Oh
look the end result was I clicked it, therefore it has tremendous value
without further analysis.

~~~
Trundle
"Tricking your brain with chemicals" is a ridiculous way to dismiss something
that people like. Every desire or want we have is the result of chemicals.

~~~
nitrogen
Our desires and neurochemistry evolved in a very different environment from
today. I think it's fair to say that a chemical that mimics an evolved
desire/reward system without actually fulfilling the need that system serves
is "tricking" the brain.

~~~
Trundle
Because of our different environment, none of those "needs" really exist
though. Being part of modern society basically guarantees we stay fed and
sheltered. Very few people are trying to procreate as much as they can. Some
of us are choosing not to at all.

We eat high calorie foods because our brain tells us its good even though
we're a little fat. We fuck each other because our brain tells us its good,
while we use contraceptives. We go out and try to be successful because our
brain tells us it's good even though our base needs are already provided for.

Sure you can say that we're tricking our brain, but only so far as you can say
everything we've done since we discovered conscious thought and developed
agency is tricking the brain. Hence why I find pointing it out in a specific
case a little silly. "Hey you're only doing that because it feels good!" Well
yeah duh? Same reason any of us are doing anything.

------
rco8786
Really crummy title. PageRank is the reason we HAVE a search as powerful as
Google, and largely the reason the web is as good as it is today.

Raise your hand if you want to go back to AltaVista/AskJeeves.

~~~
takeda
I actually miss old AltaVista I always could find what I was looking for with
their boolean expressions. Yes, perhaps it was harder to some, but it worked
well.

With Google, I feel like I have to fight with it. If I'm searching for
something obscure or perhaps a word that is misspelled on purpose it thinks it
knows better what I'm looking for. It also often returns searches without the
word that I searched for and often ignores when I prefix it with + or put in
quotes.

~~~
lobster_johnson
If you quote a word or phrase, it will be treated as required, just like "+"
used to work with AltaVista.

You can also click on the "Search tools" button and select "Verbatim" from the
"All results" dropdown. This causes the search to only perform exact matches:
[https://support.google.com/websearch/answer/142143?hl=en](https://support.google.com/websearch/answer/142143?hl=en).

And yes, I also spend a lot of time fighting Google's inferral rules. I
remember doing a search for Biber at one point, and it asked if I meant Bieber
instead.

~~~
flyinghamster
Not quite: I've sometimes put "CentOS 7" in quotes when looking for something
CentOS 7-specific, only to have Google return pages and pages of results for
older versions of CentOS. These days it's almost a crapshoot whether or not
I'll get relevant results from Google.

~~~
Gracana
As lobster_johnson says, verbatim mode is key. I wish it was the default - I'm
always using specific phrases the google re-interprets into something more
popular. A big pet peeve given my line of work: I know autocad is more
popular, but _please_ google, I really do want results about draftsight!

------
justinlardinois
I never knew PageRank scores were visible, and I never used the web before
Google.

But this article is so far up its own ass.

> Ever gotten a crappy email asking for links? Blame PageRank.

Never mind that web rings were around long before Google and used the same
tactics.

> Ever had garbage comments with link drops? Blame PageRank.

There are way more reasons spammers exist than just boosting PageRank.

The author is acting like a) Google had less of an influence on the web before
PageRank was public information and b) the web was somehow better both back
then and before Google existed. There will always be people who want to game
search engine results, regardless of how much information they know about
their own standing, and the web was pretty much un-navigable pre-Google.

~~~
hughw
> Never mind that web rings were around long before Google and used the same
> tactics.

If you "never used the web before Google," how would you know this? I suppose
you might have read about it. I did use the web before Google, and I don't
remember web rings back then. There simply was no reason to do so, before
Google started ranking pages based on the links.

I do remember lots of BS meta keywords.

~~~
haberman
I used the web before Google too, and I remember them.

Also, I do consider reading about something to be a valid way to learn about
things you didn't directly witness, so it's strange that you would discount
this.

> There simply was no reason to do so, before Google started ranking pages
> based on the links.

Webrings were useful for people who found a website interesting and wanted to
visit other similar websites. The incentive to be in a webring was that you
could get more exposure. Not to mention that the 90s were full of trendy
things like this: "under construction" gifs, "valid HTML" buttons, etc. Web
rings were one of those "clever" things you could add to a website.

~~~
james-skemp
Having had pages in Webrings way back when, it was definitely about exposure,
but I think it was also an extension of groups (Yahoo! Groups being the
biggest I can recall, but I think there was also OneGroup (?) - it maybe had
purple in it's logo?).

With Tripod, Geocities, Angelfire, and the like it was fairly easy to get a
really basic page up, typically with a bunch of links to pages that you
checked on regularly, and might be of interest to others.

At least that's how it was for me. I think the rings I was part of included
Terragen, POV-Ray, and Star Wars, and I can recall getting a couple people
started with basic (and of course very ugly, with that same star field
background for the SW-related pages) for people that I met in the various
groups.

Good times.

EDIT: And don't forget the 'made with notepad' icons. Or recommending 800x600
or 1024x768 as the best resolution to view a site.

------
kyledrake
I first learned about this after starting
[https://neocities.org](https://neocities.org) and seeing a bunch of really
garbage pages that were full of random text that linked to a derpie site
somewhere.

We get pagerank SEO spam from time to time, and it's pretty annoying. I have
the tools to take care of it within 5 minutes every day, but I do worry that
if we grow to a certain point it may no longer be possible for me to handle
the problem alone.

I'm sure many other sites have similar problems with comment spam, and I'd
love to hear some advice on how to deal with this from sites that have the
same problem.

Right now our main lines of defense are a recaptcha (our last remaining third
party embed, ironically sending user data to Google I'd rather not send to
deal with a problem Google largely created), and a daily update of an IP
blacklist we get from Stop Forum Spam.

I tried to do some Bayesian classification, but didn't make much progress
unfortunately. And nofollow really isn't an option for me, as it would involve
me manipulating other people's web sites and I don't want to do that.

~~~
mmanfrin
This might be very heavy handed, but could you build something that added
rel="nofollow" to all links on user pages until trust is verified?

~~~
kyledrake
I've pondered doing something like this, but it kindof crosses a line I'm not
yet comfortable crossing (changing users' HTML without them noticing), and
would require a way for me to convert from/back when the user edits their
site. Technically it's pretty difficult: HTML/XML parsers like Nokogiri like
to "fix" html when they parse it, so they can do a lot of changes to your
document you didn't want to make.

I've said I would decide to cross it if I ever ran into a major security
issue, but so far that hasn't been the case. I've even decided against web
page auto optimization ala things like ngx_pagespeed for this reason.

One place I do cross this line is for our
[https://neocities.org/browse](https://neocities.org/browse) gallery in order
to deal with iframe issues (I make the A links on a site open a new tab with a
javascript inject), but that passive model won't work for pagerank issues
because it uses the browser to make the changes.

And honestly, let's be real, even if I put on nofollow and Google stops using
pagerank, they're probably still going to do it because they're basically
shady pagerank scam artists in the end, fooling gullible people into sending
them money. They don't understand they're feeding into a botnet meets army of
underpaid pagerank spammers, and it's basically impossible to fix this with
education.

It would be excellent if Google gave me an API to report pagerank spam. For
all the money they've made on pagerank, it would be nice if they could defer
some of that money into helping us deal with this, and would definitely help
to improve their search results.

------
jzawodn
Hah.

Back in 2003 I wrote:

"PageRank stopped working really well when people began to understand how
PageRank worked. The act of Google trying to "understand" the web caused the
web itself to change."

[http://jeremy.zawodny.com/blog/archives/000751.html](http://jeremy.zawodny.com/blog/archives/000751.html)

It's amazing that it took this long.

~~~
wpietri
You might be interested in Goodhart's law: "When a measure becomes a target,
it ceases to be a good measure."

[https://en.wikipedia.org/wiki/Goodhart's_law](https://en.wikipedia.org/wiki/Goodhart's_law)

A friend mention that recently to me and it was really eye-opening. Once you
start looking, the pattern is everywhere.

------
tyingq
The real problem is that Google was losing the link spam war until very, very
recently. It was trivial to game them up until 2010, and only really became
relatively difficult somewhere around 2012.

And, the solution looks roughly like "weigh established authority to the point
where it trumps relevance".

------
Animats
Google has dealt with web spam by replacing it with their own ads. Search for
"credit card" or "divorce lawyer". Everything above the fold is a Google ad.
Air travel searches bring up Google's own travel info. No amount of SEO can
compete with that.

(I still offer Ad Limiter if you'd like to trim Google's in-house search
result content down to a manageable level.)

~~~
alphadevx
> Google has dealt with web spam by replacing it with their own ads.

This.

Google are not interesting in protecting us web users from SEO, but themselves
(or more specifically their core ad business). After all why would you pay for
Google ads if you could just get free traffic from Google via SEO techniques?
So SEO is the logical competitor to Google.

~~~
johnward
I also believe this. Almost all updates to the search engine seem to be to
drive SEOs towards adwords. It's not about protecting users, it's about
profit.

------
al2o3cr
Better title: "How SEO Asshattery Turned The Web To Shit"

------
rgovind
Does anyone here think we need a search engine which lets us maintain large
blacklists of websites. For example, if I am searching for information about
airbnb, I do not want news websites like NY Times, WSJ, Forbes, Business
standard etc to show up in the results at all. Any business related question
on India is invariable dominated by Times of India and other newspapers. With
google, its becoming increasingly difficult to filter out websites.

Edit: Changed "on airbnb" to "about airbnb"

~~~
legitster
Check out chrome extension "Personal Blocklist"

~~~
alxndr
I had used that extension for ages to prevent w3schools from appearing in my
search results, but removed it when I eventually got tired of seeing search
results jump around.

(Because the extension works on a JS level, it needs to wait for the results
page to load before it can strip out what you want to hide. Too often I'd see
the results page begin drawing quickly, with a couple w3schools results at the
top of the results, and my eyes would scan down to find the first
non-w3schools result; then a second later, in the middle of finding the first
such result, the w3schools results would be removed, and the non-w3schools
result which I'd finally found and was about to click on has moved.)

~~~
yxlx
I used that extension for the same reason you did and like you I removed it
because it was annoying. What I started doing instead when on a new computer
is I'll include _mdn_ in the query when it's web related so i'd search _js
history mdn_ and such and after a while Google learns that I value results
from mdn highly so when I search even without saying _mdn_ , those results
come up high in the results. For example when I now search for just _js
history_ , the first result for me is "Manipulating the browser history - Web
APIs | MDN".

The Google search bubble is powerful and can be harmful in some ways but once
you learn of its existence and are careful about what results you click, it
will work for you in a great way.

I am still uncomfortable with the amount of stuff Google knows about me. I
sometimes try ddg or even yahoo or bing but they're not as good.

~~~
alxndr
To achieve the same result, I've now got a browser shortcut (aka omnibox
search engine) for an "I Feel Lucky" Google search restricted to the MDN site
for whatever keywords I'm looking for. This means I type "mdn js history" and
Chrome picks up the "mdn" prefix to use my shortcut, expands it into
[https://www.google.com/search?q=site:developer.mozilla.org+j...](https://www.google.com/search?q=site:developer.mozilla.org+js+history&btnI)
and I end up on the same page you mention. (The "feeling lucky" search isn't
perfect, but it's usually good enough for me...)

------
_yosefk
"How gravity ruined flying"? PageRank looking at links isn't some arbitrary
thing, it's a source of information every good search will take into account.

------
hartator
I think it's odd to perceive the end of a relative transparent metric -
whatever relevant or not it has been - as a good thing.

------
gchokov
Indeed, one of the things I don't like (hate?) Google about is the SEO and
PageRanking BS. All pages in the last 10 years are starting to look the same.
All pages are becoming what Google wants them to be.

------
kin
Just because I can't see the score doesn't mean I'm not going to what I can to
increase it.

------
kazinator
What difference does it make if the semantics of PageRank are still in place
for determining position in the search index, but it is just hidden?

You can still infer the approximate rank of a page by where it places relative
to other pages, when searching for relevant keywords. Someone wanting to place
ahead of the competition still has a function for measuring how well they are
doing in SEO.

------
hackuser
Another way to look at this is a blow to openness and a concentration of
Google's power. The PageRank scores still exist, but they now will be known
only by (some? all?) Google employees.

Therefore, the data is no longer open and power is now more concentrated:
Those who know someone at Google can find out their page rank score; the
99.999...% of the rest of the world cannot.

------
VikingCoder
I just lost a ton of respect for Danny Sullivan.

Every system can be gamed. Every system where money can be made WILL be gamed.
It's a predator-prey relationship.

The way this article was written made it sound like Google Search was a bane
when it arrived. And sure, it was the worst Search Engine at the time, except
for all the others that had been invented up until then.

~~~
sullivandanny
You're reading things into this that I didn't write, I'd say.

When Google arrived, it was a huge advance in search. It offered an obvious
improvement in relevancy, which is why so many serious searchers switched to
it from AltaVista and then users of other search engines moved over.

Nothing I wrote suggested that Google was bad, didn't offer great relevancy or
anything like that.

My story is about what happened when Google revealed PageRank scores for pages
across the web. That fueled an explosion in link buying and selling. It
allowed people to attach Google's own score to a page, a value if you will
that Google itself placed on those pages, which made it easier to then assign
a monetary value.

In turn, that lead to many of the woes that the web as a whole has to deal
with today: understanding how to use nofollow to block links, to stay in
Google's good graces. Spam mail pitching links, trying to buy links. Link spam

I'm sure we'd have had some of this even without PageRank scores ever having
been revealed. Perhaps it would have been as much, even. After all, it was
well-known that Google was leveraging links as part of its ranking algorithm.
The market would have been there.

But I do think that releasing the PageRank scores accelerated market faster
than it would have done otherwise.

Back to the gaming -- again, it feels like you're reading stuff I didn't
actually write. I'm certainly not saying that Google itself introduced the
ability for people to try and game search engines. That was happening even
before Google existed. Of course, Google initially thought it was immune. In
1998, Sergey Brin even said this on a panel that I moderated:

"Google’s slightly different in that we never ban anybody, and we don’t really
believe in spam in the sense that there’s no mechanism for removing people
from our index. The fundamental concept we use is, you know, is this page
relevant to the search? And, you know, some pages which, you know, they may
almost never appear on the search results page because they’re just not that
relevant."

Google soon changed its view and introduced extensive spam fighting efforts.
Those were inevitable. As you say, it was prey that would attract predators.
And even with the link selling, it has done an admirable job fighting off the
spam. It's not always perfect, but it's a very robust system.

Nevertheless, the spam attempts will continue regardless if Google actually
blunts them because, as the article explained, there's simply so many people
with misconceptions that they'll chase anything anyway. PageRank scores fed
into this, that's all.

~~~
VikingCoder
Thanks for the response.

> My story is about what happened when Google revealed PageRank scores for
> pages across the web.

And I'd assert that people already knew if they were number one in the search
results, or not. And that metric continues to be the main thing they pay
attention to. Well, that and their traffic numbers from Google. My point
being, we all knew Google was using links to rank, and the search result rank
was visible just by doing a search on a few of your synonyms and adjacent
terms, market, brands, trademarks, etc. The battle over spamming links was
inevitible, whether they revealed PageRank numbers or not.

> I'm sure we'd have had some of this even without PageRank scores ever having
> been revealed. Perhaps it would have been as much, even. After all, it was
> well-known that Google was leveraging links as part of its ranking
> algorithm. The market would have been there. But I do think that releasing
> the PageRank scores accelerated market faster than it would have done
> otherwise.

I can agree with that, but that's not the tone that I get from your article,
at all.

The tone I get is that Google created this monster, visible PageRank score,
and those crappy emails, link drops, and need to use nofollow, are uniquely
Google's fault, and it all could have been prevented if they hadn't ruined the
web in 2000 by making it visible.

> Google initially thought it was immune.

Your quote from Sergey doesn't imply to me that he thought they were immune.
It tells me that they mechanism they intended to use to fight spam would be to
reduce its rank so low that "...they may almost never appear on the search
results page..."

You may think that's a pedantic difference, but I see it as a meaningful
difference. You can't claim that we're all immune to measles, mumps, polio...
But we've reduced the incidence to an incredibly low level, here in the
Western world.

------
vorg
Reading this makes one realize how easy it was for Groovy's Tiobe ranking to
jump from #82 to #17 in just 12 months, as shown at
[http://www.tiobe.com/tiobe_index?page=Groovy](http://www.tiobe.com/tiobe_index?page=Groovy)
, and the other spikes in its history.

~~~
aikah
Are you saying that someone working on Groovy is doing "link optimisation" in
order to get higher on Google thus Tiobe?

~~~
vorg
According to
[http://www.tiobe.com/tiobe_index?page=programminglanguages_d...](http://www.tiobe.com/tiobe_index?page=programminglanguages_definition)
, Google is by far the largest component of Tiobe's calculation: Google.com:
7.69%, Google.co.in: 5.54%, Google.co.jp: 4.92%, Google.de: 3.69%,
Google.co.uk: 3.08%, Google.com.br: 2.77%, Google.fr: 2.46%, Google.it: 2.15%,
Google.es: 1.85%, Google.com.mx: 1.54%, Google.ca: 0.92%, Google.co.id: 0.62%.

It seems the most likely reason for Groovy's strange behavior on Tiobe.

------
runn1ng
PageRank is still visible today?!? Where? (I am just curious, I thought it's
not visible anywhere for years)

~~~
tyingq
[http://toolbarqueries.google.com/tbr?client=navclient-
auto&c...](http://toolbarqueries.google.com/tbr?client=navclient-
auto&ch=751310424703&features=Rank&q=info:http://news.ycombinator.com&num=100&filter=0)

The value of "ch" is a checksum you have to precalculate.

