

Belgian newspapers: Google blocking us on searches - robtoo
http://www.google.com/hostednews/ap/article/ALeqM5jj9ad340cxT7i1ADyPk_ksWRtLgQ?docId=79891bacb5a84c068862fba59868b1d6

======
dexen
The newspapers went to court asking for broad relief (unlisting from google
results), instead of putting in place proper META tags and /robots.txt to
direct what may be, and what may not be indexed and cached. The court handed
them just as much of the proverbial rope as they asked for.

Google's web crawlers asked the WWW servers to GET, were served with HTTP 200
OK, found no prohibition in /robots.txt and so followed the standard protocol.

Note Google Web Search and Google News use separate bots (Googlebot and
Googlebot-News) to give website owner full control.

~~~
reso
I can't decide whether I find things like this hilarious or disturbing. They
could have fixed it with engineering, but they were either so uninformed or
incompetent that they tried to fix it with a lawsuit.

~~~
pavel_lishin
Or maybe one incompetent overrode them all.

Manager: Evans, get in here! Why are our news articles all over The Google?

Evans: Um, sir, they index our -

Manager: I don't want to hear your technical mumbo jumbo! Get legal on this to
get it fixed YESTERDAY!

Evans: There's no need, it's literally a 5-minute fix, I could just add an
entry to robots.txt while the coffee is brewing...

Manager: What did I just say about mumbo jumbo? Get out of here, and send in
Stevens from Legal on your way.

~~~
Toady
Again, the lawsuit was over Google News' display of lead paragraphs and other
excerpts from articles. It didn't pertain to the general search engine.

~~~
pavel_lishin
The lawsuit may not have, but the ruling certainly and explicitly does.

~~~
ToadyCensored
That's incorrect. The court's ruling specifically pertains to the use of
article content in the Google News aggregator and its storage in Google's
cache. There is no mention of simply linking to articles in the search engine,
nor would there be reason for one since the lawsuit was over the use of
copyrighted article excerpts, not hyperlinks.

What Google is doing is akin to the punishments Microsoft used to dole out in
the 90s, such as increasing Windows OEM licensing fees or threatening to
revoke licenses entirely. Google is playing a dangerous game in monopoly-
conscious Europe.

------
bugsy
OK, it says they filed a lawsuit against Google because they didn't want
Google to LINK to their articles without paying a fee for linking. They won
the lawsuit in court and so Google removed links to their articles from their
search index, as they requested, and now they are mad about it. Why is the
story "Google blocking" them. Google's only doing what a court ordered after
the papers sued asking for that.

~~~
pavel_lishin
As I understand it, they wanted Google News to stop linking to their articles;
not the regular search results.

~~~
dexen
From court order, via <http://www.chillingeffects.org/notice.cgi?sID=2160>

_> Order the defendant to withdraw the articles (...) from all their sites
(Google News and "cache" Google or any other name (...)_

The other articles linked from
[http://slashdot.org/story/06/09/18/115203/Google-News-
Remove...](http://slashdot.org/story/06/09/18/115203/Google-News-Removes-
Belgian-Newspaper) are down now, but I believe the original lawsuit was in
that very spirit: remove their articles from all Google's websites. Not just
from Google News.

Logical argumentation aside, Google's in no position to second-guess a court
order ;-)

Why did Associated Press put such a negative spin to the story? We can only
guess...

------
itcmcgrath
The only comment that comes to mind: Don't bite the hand that feeds you.

The newspapers really didn't think that one through, did they...

~~~
shrikant
I suppose to extend that analogy: the newspapers didn't believe they were
biting the hand that was feeding. They saw it more as cutting off the _other_
hand that pushed their heads towards the food.

Google is saying both hands are equivalent, but the newspapers are saying they
want one hand to feed them, and the other to keep away.

I can buy that, actually. Sure, I don't agree with the initial ruling, but
IMHO de-indexing the newspapers from the search engine was unnecessarily
retaliatory. I can understand _why_ Google did it, but having made their
point, I think they should gracefully back down claiming ignorance. This is a
clear misuse of their 'monopoly' on web search for benefit in other areas.

 _EDIT_ : Oh good, my Google fanboi-ness is not in vain. The court ruling
actually ordered that links be removed from "Google Web [sic]" as well. Oh
well, the joke's on the newspapers then, as it appears they asked for it! (see
roel_v's comment: <http://news.ycombinator.com/item?id=2776384>)

~~~
turtle4
Google supplies the tools for the newspaper to have marked explicitly, using
technology, the pages they want included/excluded from search, with no
ambiguity. Instead, they chose to use vague, court language, with threat of a
fine per link, to force google's hand. Personally, I don't know why google
-wouldn't- drop them, permanently.

There is no monopoly on web search. Search works for everyone, specifically
because there are established protocols and conventions that allow all these
systems to work together. If your company chooses to ignore these conventions,
I don't think it is unreasonable that you get ignored.

You don't have a -right- to be included in a web search. The reason google
includes you (or any search engine) is because you are making a trade, they
get better results which they can monetize via ads, you get more traffic,
which you monetize through ads. If you opt out, why shouldn't they? What's in
it for them? They aren't running a charity, they are running a business.

~~~
ToadyCensored
Your post demonstrates a misunderstanding over what the lawsuit was about.
Publishers weren't suing to make Google stop linking to them. Google News was
displaying full excerpts from copyright articles, sometimes entire paragraphs,
and on top of that making money off that content through the display of ads.
It was a copyright infringement lawsuit.

 _There is no monopoly on web search._

Google has over 65% market share in search. Second is Yahoo with only 16%.

------
slater
Anyone have a copy of the ruling? Might just be the judge(s) misinterpreting
what the newspapers were asking for?

~~~
roel_v
Ruling (in French): [http://www.ie-
forum.nl/backoffice/uploads/file/IEF%20Hof%20v...](http://www.ie-
forum.nl/backoffice/uploads/file/IEF%20Hof%20van%20Beroep%20Brussel%205%20mei%202011,%20R_nr_%202011_2999,%20817%20%28Google%20INC_%20tegen%20Copiepresse,%20Societe%20de%20Droit%20d%27auteur%20des%20journalistes%20%28SAJ%29%20en%20Assucopie%29.pdf)

Can't copy and paste from there; relevant passage:

"...orders Google to removing the sites google.be and google.com, more
precisely the links (in cache) visible through Google Web and the service
Google News (...) of Belgian Francophone and German-language publishers
represented by Copiepresse."

So yes, Google is interpreting the ruling correctly, there is no way to
interpret it in such a way that the regular google.com shouldn't be blocked. I
don't know if the newspapers asked for too much or if the judge didn't
understand what he was doing; I didn't read the whole ruling.

~~~
Toady
The lawsuit was over the use of lead paragraphs and other article excerpts,
specifically in the Google News aggregator, not plain links in the general
search engine.

~~~
roel_v
Yes of course, but the ruling was broader than that, that is the whole point
of contention.

------
lenni
Maybe I just don't get it, but why do they want to be included in web search
but not in Google News? Their actions seem bizarre.

Both are noindex-able separately, correct?

~~~
dexen
_tl;dr: Yes they are indexed separately, and yes it's up to site owner to
decide which content may be indexed for Google News -- via proper META tags
and robots.txt._

There is an important difference in UI between Google Web Search and Google
News: the Web Search presents a very short snippet of text and link, while
Google News presents a bit more of the news content. Effectively, Web Search
directs viewers straight to the original website, while News lets them just
_browse_ Google News, sending less traffic to the original website. [1]

Which means Google News may provide less ad revenue to original website --
probably the crux of the problem here.

To handle that, Google provides two different bots: one with user agent
`Googlebot', the other with `Googlebot-News'. If a website wants its content
only in Web Search, and not in News, it's supposed to return proper
/robots.txt -- and that does the trick. Way quicker and cheaper than any
lawsuit!

[http://googlewebmastercentral.blogspot.com/2009/12/new-
user-...](http://googlewebmastercentral.blogspot.com/2009/12/new-user-agent-
for-news.html)

\----

[1] but hey, less is more! whoever makes it to the original site via Google
News has clearly show elevated interest in the news piece, and thus should be
worth more to ad providers.

------
Bootvis
The papers are back according to this news article:

[http://babelfish.yahoo.com/translate_url?doit=done&tt=ur...](http://babelfish.yahoo.com/translate_url?doit=done&tt=url&intl=1&fr=bf-
home&trurl=http%3A%2F%2Fwww.nu.nl%2Finternet%2F2568534%2Fbelgische-kranten-
weer-google.html&lp=nl_en&btnTrUrl=Translate)

They also hint that the complaints have been lifted by the papers.

~~~
feb
Yes, they are back in the index:
<http://www.pcmag.com/article2/0,2817,2388635,00.asp>

And in a simple test, it looks to work too:
<http://www.google.com/search?q=copiepress+lalibre+critique> The second link
(lesoir.be) is one of the newspapers.

------
Luc
This is not a case of technical ignorance, though it's hilarious to pretend it
is. The French-language newspaper association wants to receive money from
Google (for the ads displayed next to their content) and figured it would be
easier to pressure them through the courts than through other means.

~~~
ToadyCensored
The lawsuit was over the use of copyrighted excerpts from articles in the
Google News aggregator and cache, such as lead paragraphs.

Google is trying to pretend that the court ruled all hyperlinks to be illegal.
Google was defiant before over this ruling when it ignored an order to publish
the results of the verdict on its website, so it's not far-fetched to believe
Google is once again throwing its weight around and reminding people of the
power it wields over the livelihoods of content creators.

------
franze

       User-agent: Googlebot
       Disallow:
    
       User-agent: Googlebot-News
       Disallow: /

