
New Chrome extension: block sites from Google’s web search results - dannyr
http://googleblog.blogspot.com/2011/02/new-chrome-extension-block-sites-from.html
======
Matt_Cutts
I just wanted to say thanks to all the people on Hacker News who asked for
this option. We'll look at offering a "block site" option directly in the
search results over time, but it takes longer to write, test, and launch that
code.

In the mean time, use this extension to clean up your own search results and
tell us which sites you don't want to see in Google.

~~~
ErrantX
So, does this mean expertsexchange will eventually get shoved down the search
results?

(AKA great stuff!)

~~~
simpleenigma
I installed the extension and blocked experts exchange first thing ... not
sure if I'll even use it for any other sites ...

~~~
SwellJoe
ehow.com, ezinearticles.com. I'm sure there are many others, but those piss me
off daily.

~~~
lambda
Yeah, those and WiseGeek for me. I keep on getting WiseGeek in searches
(especially if I search for a question, which I occasionally do) and the
content is complete and utter crap.

------
dsl
1\. <http://bit.ly/gTADhE>

2\. Click Install, close page

3\. Open each of the links below in a new tab, click block on the first result

4\. Win.

<http://www.google.com/search?q=Mahalo>

<http://www.google.com/search?q=ehow>

<http://www.google.com/search?q=experts-exchange>

<http://www.google.com/search?q=livestrong.com>

<http://www.google.com/search?q=answerbag>

<http://www.google.com/search?q=bills.com>

<http://www.google.com/search?q=chacha.com>

<http://www.google.com/search?q=associated+content>

<http://www.google.com/search?q=efreedom>

<http://www.google.com/search?q=questionhub>

<http://www.google.com/search?q=squidoo.com>

<http://www.google.com/search?q=about.com>

<http://www.google.com/search?q=yellowpages.com>

\----

Edits: fixed formatting, added suggestions

This method is fine. The actual data sent to Google when you block a domain
does not contain the search query (or the referrer).

This is what gets sent when you block a domain:

    
    
      http://www.google.com/gen_204?atyp=i&oi=site_blocker&ct=addToBlocklist&ei=[CSRF-cookie]&cad=mattcutts.com
    

and unblock:

    
    
      http://www.google.com/gen_204?atyp=i&oi=site_blocker&ct=deleteFromBlocklist&ei=undefined&cad=mattcutts.com
    

(Interestingly the CSRF token is broken when unblocking.)

~~~
billybob
Doesn't this tell Google "Mahalo is a spam result if I search for 'Mahalo'"?

Seems like you should do this when you get one of these and you wanted
Stackoverflow.

~~~
jmillikin
I don't think it records which search first resulted in the block, only that a
block exists

------
jmillikin
1\. Does this remove results from each page, or from the resultset? In other
words, if 7 of the first-page results are blocked, will I see only 3 results
on that page?

2\. Any plans for a Firefox extension? I'm willing to install Chrome just for
running Google searches, but would rather add it to my main browser.

e: After a month or so, I would absolutely _love_ to see the top 10 or so
blocked domains. It's OK if you can't do this, but it would be
interesting/amusing.

~~~
Matt_Cutts
Right now, you'll see only three results. We'll look at refreshing, but in
answer to #2, we're also looking at putting this more directly into Google's
search results.

I think having block options directly on Google's search results is the right
long-term answer. But this extension lets people clean up their personal
results while sending block data to Google that we might be able to use as a
signal to improve overall search quality.

Thanks to everyone at HN who poked us by asking for this, by the way.

~~~
jmillikin
> _We'll look at refreshing, but in answer to #2, we're also looking at
> putting this more directly into Google's search results._

Don't worry about refreshing -- I'd much rather see three good results than 7
bad ones. The idea of being able to block content farms entirely is
_literally_ making me giggle at my desk. Focus on that!

Thank you, thank you, thank you x1000 for this.

~~~
Matt_Cutts
I've been using it for a few weeks while we tested it, and it really does feel
nice to block a site that you never want to see again. :)

------
arnemart
The website used as an example in the first screenshot
(<http://thecontentfarm.tumblr.com/>) just made my day.

~~~
Matt_Cutts
I made the screenshots this weekend, so I threw that in there.
<http://thecontentfarm.tumblr.com/> is pretty funny.

------
ck2
Why do I need to use Chrome and then an extension if this is being offered by
Google?

Make this a google labs feature directly for Google itself in the
personalization options.

(also please make it available via a URL option, not just cookies or
javascript)

~~~
JoelSutherland
I'll take the version that walks today over the version that runs in 6 months.

------
georgemcbay
Well played, Google.

Not only did you just make your search engine 50 times more valuable to me,
but you've just ensured I'll be spending almost all of my browsing time inside
Chrome.

------
macrael
Why make this a Chrome extension rather than a google labs type feature?

~~~
Matt_Cutts
The Chrome extension was faster to roll out and lets us iterate faster as well
because it's outside of front-end pushes. We're looking at offering block
links in the search results too, but that code takes longer to write, test,
and launch.

~~~
rbranson
It seems we've come full circle -- features easier to push out in client
software than through modification of server code.

------
jjcm
Any chance we can get the blocklist transfered via sync in the next iteration
of this plugin?

~~~
WesleyJohnson
I second that suggestion. It would make the extension that much more awesome
to not have to block sites on each device. Might also help to lower the noise
on Google's end, so they're not seeing multiple blocks on the same domain from
the same users, though I'm sure they account for this already.

~~~
aboodman
It's a bit difficult to implement sync of arbitrary data in a Chrome extension
now. The extension developer has to setup their own storage and implement
their own sync code. Either that, or some developers just store their data in
the bookmark system, which has its own problems (please don't do this Matt!).

We (the extension posse) are working on making this really easy to do [1], but
nothing exists yet.

[1] <http://crbug.com/47327>

~~~
Matt_Cutts
Aaron, thanks for stopping by with an update on Chrome sync--much appreciated.
And thanks for making Chrome extensions fast/easy/powerful to write.

------
brianwillis
So HN, what sites are we all blocking?

~~~
jonknee
My first round includes eHow, Mahalo, Associated Content, Experts Exchange,
Squidoo and Examiner.

Update: I added answers.yahoo.com as well, which has consistently terrible
content and yet is often highly ranked.

~~~
klbarry
I really often get exactly the answer I need from Yahoo Answers, just to throw
that in there...

~~~
pamelafox
I've found that there are some more taboo (but valid) questions that are only
ever answered on Yahoo! Answers, presumably because people aren't worried
about their identity there.

For an extreme example:
[http://answers.yahoo.com/question/index?qid=20071113122412AA...](http://answers.yahoo.com/question/index?qid=20071113122412AAHwZQD)
(Warning: Taboo. And the answer probably isn't relevant to 95% of the posters
here. :)

~~~
jonknee
I found Wikipedia's answer to be of much higher quality (and properly cited).

------
WesleyJohnson
Assuming it's a normal extension and has to abide by the same rules that Non-
Google Authored extensions do, the extension manifest indicates the extension
doesn't have access to do any cross domain posts so all the filtering is done
client side. Digging a little deeper, it looks like the blocked sites are
stored in Chrome's LocalStorage, which if memory serves me correctly is
somehow isolated per extension.

It should be relatively easy to listen in on the background page while the
extension is running and write a script to extract the list of blocked sites
or update it with a master list so you don't have to block dozens or hundreds
of sites manually.

Not that I think everyone should blindly block everything everyone else does
on HN; I personally loathe Experts Exchange, but I do find an answer I needed
from them now and then.

I was more curious than anything.

Update: As "dsl" posted above, it does look like the extension makes a call
out to a Google Endpoint to record the block as well, but I don't believe that
call actually filters the data for you. That's still done client side. So it's
probably best not to call the end point directly or update the blocked sites
list directly, but actual use the extension as intended?

~~~
nolok
Yes they block client side (several posts from matt confirm it in this
thread).

But you mention something important; for me I don't want to 100% block most of
those websites, but to give them lower priority (never in my top 10 for
exemple). Like you, as much as I hate Experts exchange sometimes they do have
the answer when no one else does. This extension sems to solve that by giving
a "show" link to display hidden results (when there are any).

------
runevault
Normally I don't post this sort of thing (try to focus on valuable content)
but dear god THANK YOU!

I'm cackling maniacally while I block expertsexchange, Mahalo, and several
other sites. I'm so happy right now.

------
jellicle
Free karma points to whoever creates a corresponding Firefox extension...

~~~
toni
There is a Greasemonkey script[1] which does blacklisting. It works by sending
"-site:example.com" with every search query. The script does a lot more
cleanups like removing link tracking and ads, so if you don't feel like using
all the functions, look in the code and check out "RemoveBadLinks()" function.
Maybe you want to comment all other public functions in init().

[1] <http://userscripts.org/scripts/show/79742>

~~~
shadowpwner
Wouldn't that reach a limit pretty soon? I've commented on this before, but
there's a max character length for the search box.

~~~
toni
MaxLenght of the input field is 2048, so you can put lots of domains in it. Of
course, it's not a complete solution but it works to some extents, specially
because it filters out stuff on server side and not with a css' display:none.

------
vaksel
would be cool if there was a subscription option where you can subscribe to
some master list that gets updated by people you trust.

or just simply a bulk insert

~~~
evilduck
My concern with an "AdBlock Easylist" style centralized block list is that it
removes, or at least blurs the consensus feedback to Google of what is a
content farm, what is a low-quality non-farm site, and what is useful.

A user above listed about.com as something they wanted to block, while I find
them to be not very in depth, I wouldn't classify it as useless, and certainly
not a content farm like livestrong.com.

------
raintrees
I was just griping about this to my wife yesterday. The noise is drowning out
the signal in my recent searches...

~~~
Matt_Cutts
Please install the extension and let us know what sites are annoying you.
You're exactly the sort of person we want that feedback from. In the process,
you'll keep them from showing up for you again.

~~~
raintrees
Oops? Maybe I am not referring to the same problem - Rather than identifying
"content farms" I am hoping to block sites that are returned that do not have
the search terms on the page returned, and therefore should not be in the
results, in my opinion.

Is this Google's intention for usage?

------
lawfulfalafel
Isn't anyone else kind of bewildered by this?

I mean this is kind of like if a kid pissed all over the floor in wal-mart,
and when you notified an employee about it they gave you a mop to clean it up
yourself.

~~~
georgemcbay
It is more like being in a Walmart full of zombies and having all the Walmart
employees killing zombies as fast as they can but then when you ask what is up
with all the zombies they hand you a shotgun so you can help them by killing
the zombies in your immediate vicinity.

On the one hand, in an ideal world there wouldn't be a large and ever growing
number of zombies at Walmart, but on the other hand, thanks a million for the
shotgun because I'd rather get to killing some zombies myself than whine about
the fact that the Walmart employees aren't killing them fast enough while they
are eating my brains.

~~~
lawfulfalafel
Really? This is kind of like some sort of weird time warp, and the Yahoo
mentality of "curating" the internet is back in fashion. Which is weird,
because Google came to power because that strategy doesn't work.

------
rsoto
Having a big company behaving like a local one-- listening to the customer's
opinions is really nice.

However, this is a feature that Google actually had. Why did you remove it? I
accept the Search Wiki was not particuarly a success[1], but the remove option
was very nice.

Alas, thanks for listening. I'll be waiting for the server-side option.

1: [http://googleblog.blogspot.com/2008/11/searchwiki-make-
searc...](http://googleblog.blogspot.com/2008/11/searchwiki-make-search-your-
own.html)

~~~
Matt_Cutts
I think users are more savvy about recognizing spam/low-quality/content farm
sites than a few years ago. At least some of the previous features also had
unfortunate UI issues, e.g. the default was to block a single url for that
single query.

------
jackcat
Positive move by Google, which should help the best content to rank in search.

However when Google start using this feedback data directly in their algorithm
there are two dangers, which both can be overcome:

1\. Abuse by SEOs trying to get their competitors blocked

2\. Bad use of block data on a site wide basis in the search algorithm. Eg
just because certain pages on a site cause users to give block data doesn’t
mean other pages on that site contain unique useful content. High volume
publishing or community sites will have a large degree of noise, but due to
nature the cross section of society as members there will be also valid useful
content in the seams.

One approach is to let individual users block sites forever (as the extension
now) but use the block data on a page, not domain basis in the search
algorithm. Individual pages can also increase in quality over time (the qoura
business model) so changes in page content needs to be considered too.

The Internet is a better place with non editorial user generated content
(hacker news is user generated) after all the human race mind based knowledge
is a not all published online. Google just needs to figure out which pages
really add value in that specific search no matter which domain they sit on.

------
enmanuelr
You mean I never ever again need to see a search result pointing to experts-
exchange?! This is the best gift ever! And it's not even my birthday.

------
topcat31
A few thoughts and requests:

1) I'd love to use this tool but because of my job still need to look at the
"official" search results from time to time. Is it possible to allow &pws=0
override the blocked sites so &pws=0 both session-based and browser based
personalisation?

2) There's a lot of talk among startups about ignoring what users want because
they don't know best. I worry that this move from Google might actually be
detrimental to user experience. If you end up blocking a bunch of thin sites
then the chances are high that you're not going to get better results for your
search query, you'll just get fewer results (since usually Google only returns
these thin sites when there's little else to offer). Also - as users get
trigger happy and block a bunch of sites they may well be harming their own
search experience. I'd love to hear your thoughts on how Google can mitigate
against this? Especially when this rolls out to your average user rather than
just the tech-savvy HN crowd.

Tom

~~~
ydant
If you don't allow the extension to work in incognito mode (it's opt-in by
default), then you can just open up an incognito / private browsing window
instead (which Chrome makes pretty low-cost). This might have the benefit of
removing any influence on your search results due to Google cookies that would
otherwise be sent.

~~~
topcat31
I don't think the plugin works via cookies though does it? Might be wrong....

~~~
ydant
By default Chrome does not allow plug-ins to run at all in incognito mode. You
have to explicitly allow the setting highlighted in this screenshot (available
from the URL "chrome://extensions")

[http://img6.imagebanana.com/img/0om09caa/20110215_135721_Sel...](http://img6.imagebanana.com/img/0om09caa/20110215_135721_Selection_001.png)

------
Kylekramer
Good work, squeaky wheels.

So: will this eventually be a search settings option once it is less beta or
permanently an extension thing?

~~~
Matt_Cutts
We'll look at offering this as a more direct option over time, depending on
how popular the extension is and whether the data from it looks good.

------
nooneelse
Hey, I think I "called" this a few days back... yep:
<http://news.ycombinator.com/item?id=2199498>

This little bit of successful nerd prognostication cheers me up more than
perhaps it should, but oh well.

------
zitterbewegung
Although this is Chrome only this is a great extension that I believe a few
people on this site wanted. I remember other people made a mashup but this
looks like a slightly better solution. I wonder why they don't want to do this
server side though?

~~~
stanleydrew
I imagine releasing server-side changes to Google's core search functionality
is quite a bit harder than releasing an extension. If nothing else, there are
tons of tests that need to pass and there's probably constraints on speed
regressions and such. This is a much quicker way to get the same data so that
they can determine whether the data is actually useful as a ranking signal.

------
SwellJoe
I feel like I just got a new upgraded Internet.

~~~
brianpan
And we make fun of our parents for thinking that Google = internet. ;)

~~~
panacea
We do? I thought it was the blue IE6 'internet' icon?

~~~
SwellJoe
E is for enternet.

------
atomi
>...explore using it as a potential ranking signal for our search results

Democratic censorship.

~~~
drivebyacct2
What's the difference in Google censoring spam and users censoring spam?

~~~
Matt_Cutts
I'll pass along an anecdote from Gmail, which also uses explicit user feedback
to label spam emails. Some people come to Gmail and say "Why did you block the
[mass low-quality] email that I sent?" It's a pretty good answer if the reply
is "Enough people marked your emails as spam that the emails were considered
spam."

The hope is that users are savvy enough that we can get a good signal out of
the block signal.

~~~
atomi
This would only be a fair comparison if and only if you allow users the option
to view the blocked search results.

------
measure2xcut1x
If this ends up being used as a "voting" system that factors into determining
what sites show in public SERPs, folks could crowdsource competitor blocking
using say Amazon's MTurk to knock competition off of Google, no?

------
tuhin
Finally Google accepts the elephant in the room. Just a few days ago a friend
while being interviewed had mentioned about search being ineffective for
content farm sites and the Google employee was like "No Search is fine.
Nothing wrong there". Finally e-how, about.com, expertSEXchange.com can RIP,
atleast for me. I also hope that over time, Google will use this data from
users via Chrome to somehow incorporate crowdsourced search results in it's
results not just for one user but all of them.

------
ramki
I thought i was the only one doesn't want to see "experts exchange" in google
search results. No, i'm not alone... :) :) Looks "experts exchange" is
annoyingly very famous...

------
saturdaysaint
Nice. My first target - every local restaurant result I get that appears
before either (1) the official restaurant website or (2) Yelp. Local search
results are always gunked up with yellowpages.com and local newspaper spam.
Also, I find it highly suspicious that urbanspoon has consistently better
placement than Yelp, despite having consistently weaker content.

------
mythobit
I created a site on app engine that essentially does the same thing via Google
Custom search engine. Granted mine isn't as integrated or user friendly but
still. The site is: <http://blacklist-search.appspot.com/>

Why can't Google offer something like this rather than only allowing it via a
Chrome extension?

~~~
Matt_Cutts
I think we can, but that sort of code takes longer to write, test, and launch.
You'd want it to be really robust before you showed those sorts of options to
1B+ people a day. So the Chrome extension is a good interim step to see how
much traction it gets and what sort of data emerges as a result

------
hackerku
I know this is a legit extension from Google.

By why is this extension not marked as Verified author?
[https://chrome.google.com/webstore/detail/nolijncfnkgaikbjbd...](https://chrome.google.com/webstore/detail/nolijncfnkgaikbjbdaogikpmpbdcdef)

The "nolijncfnkgaikbjbdaogikpmpbdcdef" makes it look suspicious as well.

~~~
Matt_Cutts
Hmm. I'll ask if we can fix that.

------
Jem
"If installed, the extension also sends blocked site information to Google,
and we will study the resulting feedback and explore using it as a potential
ranking signal for our search results."

If that happens, what's to stop this being used by companies to influence the
results to get rid of competitors?

~~~
stanleydrew
Did you even read the snippet that you quoted? They will "study the resulting
feedback and explore using it as a potential ranking signal for our search
results."

They are well aware of the ways in which such a signal could be gamed. That's
why they will "study" and "explore" ways to use the new data as a "potential"
signal.

~~~
Jem
Was the twatty attitude really necessary?

I asked a simple question expecting someone with industry-appropriate know-how
to expand further on Google's theoretical/possible intentions.

We already have an entire industry dedicated to gaming the search engines
ranking signals/etc; we have some very talented black hats who undermine
existing attempts to keep SERPs clean. I don't think it was that stupid a
question.

------
RazorSky
I was just bitching about experts-exchange last night and wanted this feature.
Thanks for sharing.

------
kaffeinecoma
Thank god. Goodbye to: devcomments.com. osdir.com, and mail-archive.com!

~~~
catch23
osdir isn't actually too bad -- mailing lists are very poorly indexed by
google and osdir exposes some of the data to be indexed.

------
algorias
Looks promising, but I'm unsure about the security angle. Google has just
added a way for anyone to "DDos" competing websites into oblivion. I hope
there are measures in place to prevent that.

~~~
stanleydrew
They aren't yet using the data for ranking.

~~~
paraschopra
They are or probably would be doing that. See this comment by Matt Cutts:
<http://news.ycombinator.com/item?id=2218531>

~~~
nolok
And you really think no one in the google search team has thought of how it
could be abused ? Come on let's be serious here, they are not going to use
these results verbatim or even block websites for everyone just because some
users wanted to.

~~~
AlexBerger
Even at that, just look at all the problems Reddit etc. have had with bury
brigades working to game the system. I imagine with the reputation defender
styled companies out there right now, this would be a natural service.

------
donbronson
When I first read the headline, I assumed it meant blocking all of Google's
sites from the SERPs (Youtube, blogger, etc). Perhaps this would be a nice way
to rule out any potential nepotism.

~~~
Matt_Cutts
You can block google.com in the extension if you want (we didn't put in any
code to prevent that), but your search results might behave in
weird/unexpected ways if you do.

------
ChuckMcM
I note with mock irony that this works fine on blekko.com in any browser. You
push the "this is spam" link in the SERP results and poof, its dead to you.

(Disclaimer: I work at Blekko)

------
EastSmith
I know this questions is asked in different forms couple of times in the
comments, but here it is again: 1\. Will there ever be Firefox extension which
do the same?

------
presto8
Awesome extension!

An option to hide the icon from the toolbar would be nice.

~~~
Matt_Cutts
To hide the icon, you can right-click on the icon and select "Hide button." To
bring it back, click Wrench->Tools->Extensions and click the "Show button"
link.

I love Chrome. :)

~~~
presto8
I don't see that option on Linux v.8.0.552.224. But maybe I need a Chrome
upgrade. Thanks for the tip, I'll try and get it working!

~~~
Matt_Cutts
I'm using the dev version of Chrome on Linux, currently "10.0.648.45 dev". The
developer channel has always been surprisingly stable for me.

------
Upset
I think this tool will be "Very Misused" by a lot of people just to SQUASH
their closest competitors. I can see some companies hiring "paid blockers" to
squash competition websites and even in the Search Engine world you watch
Microsoft and Google block and report each others sites in an even continuing
search engine war what a sad day for free speech George Orwell 1984 :(

~~~
Upset
Enlightened people seldom or never possess a sense of responsibility. George
Orwell

------
jonmc12
Google should make one of these for Bing also - with the option to send google
my Bing blocklist.

------
narkee
Why should I have to be signed in to my Google account to be able to use this
functionality?

~~~
aubergene
To attempt to prevent people gaming/spaming the system

------
hedaru
Is it also integrated with WOT (Web of Trust) way to block and report bad
sites? How if so?

------
stcredzero
I'm going to try it first thing. I hope all of my Chrome instances sync the
block entries.

------
username
Finally I can block ycombinator.com Thank God!

------
natmaster
Sounds like an excellent feature. Just seems kinda weird that Google would
start using user-click data when they were complaining so much about Bing
using it.

~~~
ericd
This is not comparable, at all.

------
arkitaip
Is there something similar for Opera 11?

------
billmcneale
answers.yahoo.com, here I come.

------
freddealmeida
just wondering what happens if you block google.com? is that a meta-block

------
barista
Is it possible to block Bing from copying search results using this? ;)

------
Upset
One does not establish a dictatorship in order to safeguard a revolution; one
makes a revolution in order to establish a dictatorship. George Orwell

------
Upset
What can you do against the lunatic who is more intelligent than yourself, who
gives your arguments a fair hearing and then simply persists in his lunacy?
George Orwell

------
Upset
Progress is not an illusion, it happens, but it is slow and invariably
disappointing. George Orwell

------
Upset
If you want a vision of the future, imagine a boot stamping on a human face -
forever. George Orwell

------
Upset
There are some ideas so wrong that only a very intelligent person could
believe in them. George Orwell

------
Upset
Freedom is the right to tell people what they do not want to hear. George
Orwell

