Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Something really scary is going on in Germany (martinweigert.com)
338 points by gorm on Dec 6, 2012 | hide | past | favorite | 152 comments


Belgian papers sued Google a few years ago over similar uses of their content. It sort of backfired on them when google stopped crawling their pages. The internet and Google are more powerful than the German print media. http://blogs.wsj.com/tech-europe/2011/07/18/belgian-papers-a...


What a great article. So brief, yet so much justice.

Ok, ok, also a little bit of schadenfreude.


>so much justice

I suggest the "careful what you wish for" lesson applies both ways. "Justice" implies that Google can and should get away with a lot of things under threat of removing your content from their index.


I get what you're saying but in this specific instance, Google was right to delete them from news.google and the general index. The newspapers can't have it both ways (pay us to show up on Google News but keep us for free on Google Search).


See my comment below, but to recap the point, Publishers don't necessarily understand the value of being listed by Google. Not listing them is both the fastest, and most difficult to argue with, way to educate them as to that value.


Invalid generalization. If you want money to be part of the index, they should be able to remove you.


But everybody knows this ... Google knows it, we know it, and Google knows we know it.

Google knows that although they could get away with a lot in the short-run, abusing this power would hurt them badly in the long run. They gain huge benefits from being seen as more or less neutral (to the extent possible—some people will never view them that way).

As a result they seem to have been very, very, careful in how they use this power, only wielding it when explicitly ordered to by an authority with power enough to force the issue.

I'm sure the humans at Google are often tempted (who wouldn't be?), but they appear to also be smart enough to resist that temptation...


Since Germany is a populous and wealthy country, Google has a strong commercial interest in not removing German content. It's hard to imagine what nefarious thing Google would force Germany to do this way.


why there are other Publishers that will step in to fill the gap New Scientist has just launched a German language site for example.


It suggests you might just want some competition in search department. Why not defect to Bing?


That's only half of the story:

> AllThingsD reports that Google has now re-indexed newspapers in the Copiepresse group. This is the right move and also a very self-interested one by Google. If it were to “punish” publishers that didn’t want to be included in Google verticals (e.g., News, Places, Shopping) antitrust investigators would use that as evidence against the company.

http://searchengineland.com/beligian-newspapers-claim-retali...


I wasn't aware of this story, but this is exactly what I would do if I was Google. Hey, you gotta protect yourself cause the next thing you know, the papers will sue for the search results.

I'd take evil to the next step, where I would now charge the papers for listing search results AND news articles.


To be fair, there were a few positive/neutral articles, for example by Frank Rieger, a popular member of the Chaos Computer Club, in the FAZ (Frankfurter Allgemeine Zeitung) but they are in the minority.

Of course, there is the obvious ignorance regarding technical facts like: "robots.txt is from the stone age. On or off for everyone is the only possibility" [1] which reminds me a lot of the discussion we had some time back regarding internet filters. But the one thing I find really dangerous is that they (meaning major newspapers, politicians, etc.) managed to spin the story so that the narrative is now "greedy google" vs hard working journalists. I applaud google for their efforts (and I am fully aware of their commercial interests in this matter) but I slowly begin to think they did their cause a disservice. If a discussion takes place its always about google and their lobbying. The extend of this law which could lead to bloggers being sued (btw: a side effect of the very fuzzy written law which leaves a lot open and almost certainly will need a court to decide on the details) when they link to news articles is almost never mentioned.

One last thing: Recently, two big news newspapers had to shut down and that print sales are declining is nothing new. I cant remember the last time I bought a newspaper and I am also pretty sure that although blogs/twitter/whatever are a good addition they cant replace classical media. There is definately a need for the discussion for new sources of incomes for classical paper based medias as ad sales from their online publication wont cut it. Perhaps something like a "culture/media tax/flatrate" as we currently have with the GEZ (for the financing of the public tv stations)? I dont know, but the #lsr is certainly not the way to go.

[1]http://www.golem.de/news/leistungsschutzrecht-springer-vergl...


> Recently, two big news newspapers had to shut down and that print sales are declining is nothing new. I cant remember the last time I bought a newspaper and I am also pretty sure that although blogs/twitter/whatever are a good addition they cant replace classical media. There is definately a need for the discussion for new sources of incomes for classical paper based medias as ad sales from their online publication wont cut it.

... I like Clay Shirky's comment:

> Round and round this goes, with the people committed to saving newspapers demanding to know “If the old model is broken, what will work in its place?” To which the answer is: Nothing. Nothing will work. There is no general model for newspapers to replace the one the internet just broke.

and

The newspaper people often note that newspapers benefit society as a whole. This is true, but irrelevant to the problem at hand; “You’re gonna miss us when we’re gone!” has never been much of a business model.

http://www.shirky.com/weblog/2009/03/newspapers-and-thinking...


Isn't the problem here that the benefit is not directly visible? Same reason there is still no global agreement on a proceeding against global warming. There is no doubt that you need a functioning, diverse and independent press for a functioning democracy. So indirectly everyone (corporations as well) benefits from this institution.

In germany we have a couple public tv stations which are financed through a fee everyone owning a tv must pay, regardless if they watch those channels or not. They arent bound to a viewing quota, are independet from the government and have an educational mandate.

Well, at least in theory: In practice huge amounts of money are just wasted by a gigantic bureaucratic apparatus, politicians and the church have a saying in the content and the stuff shown is becoming more and more like the crap on private tv stations.

I think something like this could be a viable solutions. But implementing it, especially the algorithm which decides who gets how much is very hard if not impossible. To much attack surface for illegitimate interventions.

edit: typed that comment before I had time to view your link. great read. Never really thought about the "save newspapers vs save journalists vs save society" aspect.


You appear to be arguing that we should subsidize businesses for which the market rate for their product is virtually zero. The idea that what's in the interests of big media companies and their owners is somehow in the interests of the creative talent (think - do musicians outside a tiny number of stars have any negotiating position with record giants?) and that that is in turn somehow in the interests of society just doesn't follow for me. Every industry that fails makes the same argument - the public should pay to save us (and you are paying if as a Google customer you are forced to have your services restricted because Google has to pay to subsidize these failed businesses). But just as the piano sheet music industry was successfully replaced by the recording industry, new industries rise and replace old ones and meet the demands of the market. Just because we cannot see what that future is - because if we could we'd be building it and be billionaires doing so - doesn't mean it won't happen.

Twitter's coverage of the Arab Spring was better than any Western media in the early days... food for thought.


>Twitter's coverage of the Arab Spring was better than any Western media in the early days... food for thought.

I would disagree. First of all, my parents dont know how to use twitter. But lets say thats just a generational problem which will solve itself over time. There is still another issue:

The signal to noise ratio on twitter is very low. To get a good information on, lets say the arab spring, I have to spend quite some time digging through many tweets to distill the relevant information. A not dismissible overhead. And even if I managed to find out all relevant tweets and managed to form a coherent image from those splinters of information (which could be hard because I may be missing relevant background information) I still dont know if I based my model on legitimate sources. Regimes knows just as well which power lies in the social media and are eager to spread false information and propaganda.

Of course, false information is a problem also existent with classical journalism, but to a much lesser degree. They build their reputation as a reliable source of information (especially abroad in very intransparent situations like the arab spring) over many years with a network of trusted correspondents.

Thats something which is very difficult to replace with social news mechanisms.


Tell folks who read the NYT in the run up to the invasion of Iraq, or people who watch Fox News - or any of the US mainstream media - that the traditional media has less misinformation.


Good point. I didnt say they were free from misinformation. Just that I know I can trust eyewitness reports from international correspondents more than I can trust a random twitter account.

To tackle the problem you describe its important to encourage critical thinking and to not rely on one newssource. And we need investigative journalists. And whistleblower. And social media. And luck.

Its very hard to counter governmental lies on such a big scale.


I hear what you're saying. I don't necessarily disagree fundamentally, but I do think most of these arguments about the news media revolve around the myth of the noble scrappy reporter fighting for the truth and the gritty editor who backs him against interference from above, and the owner that pulls for them both to succeed. I think that Hollywood version just no linger exists if it ever did. Media companies and their owners care about generating profits and the power and influence they get from controlling the picture of reality try masses get. The people who work in them seldom represent their platonic ideal any better. Most will write whatever pays them the best or assists their move up the media food chain - watch them abandon all pretense of principle when they get the TV gig!


> The newspaper people often note that newspapers benefit society as a whole. This is true, but irrelevant to the problem at hand; “You’re gonna miss us when we’re gone!” has never been much of a business model.

In case you collect logical fallacies, this is a wonderful example of "Argument from adverse consequences":

A

But A leads to B, and B is bad

Therefore, Not A

This is reasonable if you're deciding whether to pursue A as a course of action, but it is idiotic if you're trying to decide whether A is true or not. A will be true or false regardless of whether you like what it leads to.


Well, if we're going to miss them when they're gone, then we should figure out a goddamned way to replace it, rather than saying, "The Internet is just too much revolution for you to handle, man."

Full-time investigative journalism cannot go away, for the sake of society. It's a matter of how we will continue it, or what model and best practices will supplant it. iReport doesn't count.


I'm not sure we are going to miss them when they're gone. The point of Clay's article, as I read it, is that's their argument.

In reality, I agree with Clay (and you, I think), which is to say that it isn't newpapers we might miss, but solid journalism. The interesting question is whether these are indpendent variables. I.e. could we have newspapers without good journalism and good journalism without newspapers.

I think a good case can be made that journalism has been declining for years, irrespective of the effect of the internet.

I also think it's quite possible we may end up with good journalism and no newspapers. I don't think wikileaks is the answer, for example, but such an organisation couldn't exist twenty years ago. I think we'll see a lot more attempts to create a 21st century journalism model before one sticks.


It won't go away it'll be replaced by other things. It already is. People get more news faster from voices on the ground in events like the Arab Spring than they ever did from a newspaper. So for immediacy other avenues are arising.

Newspapers and media companies, have had no problem killing off their news departments in the name of profits - eliminating foreign desks, reducing investigative reporting, focusing on lifestyle reportage and opinion pieces.

High quality opinion writers are finding success as bloggers - some are able to make a perfectly good living as such. They don't need the label of a newspaper to sit beneath any more.

There will always be outlets for investigative journalism. There will always be some form of news show and publication. Advertisers gain a certain value from placing their ads alongside high quality news content. Just not enough to support billions of dollars of newspaper revenue.

If the commercial radio and TV stations disappeared from the Bay Area we'd still have NPR - a high quality news alternative. Why? Because there are a certain number of people always prepared to pay something for high quality content.

Long way of saying I think you're right - we'll end up with good journalism and no newspapers.


"But the one thing I find really dangerous is that they (meaning major newspapers, politicians, etc.) managed to spin the story so that the narrative is now "greedy google" vs hard working journalists."

Isn't there a certain degree of truth in this? Truly, there is a big problem if the rates for borrowing content is too high. But I think what the German public is responding to is how journalism has changed and not in ways that are necessarily positive. There is far less incentive to be a good journalist now since the quality of a given article is likely much lower than in the past -- research is more limited, articles are written in a greater hurry, even some articles these days are largely written by computer or by individuals overseas.

What this law is reasonably trying to address is that content of whatever form -- be it newspaper articles, images, whatever -- has inherent value that should be recognized. Surely enforcing that value with an iron fist like the RIAA is not the right way to go. But allowing free expropriation (even of abstracts) may also be unfair.

I think a good analogy, fifty years ago, would be a newspaper that sends out people to read other newspapers very early in the morning (say at 5 AM) and then produces its own newspaper at 6. Surely such a thing would not have been possible fifty years ago, but something similar is possible today with the advent of the internet. If this behavior had occurred fifty years ago and hadn't been regulated, imagine what would have happened: the overall quality of newspapers would have been diluted and the incentive to produce good articles would have likely declined.

Now, of course, this isn't quite perfect. Again, Google is borrowing very small snippets. And surely -- if anyone remembers this -- the French courts were wrong several years ago when they allowed some newspapers to sue Google for simply posting a few sentences or a link to an article. But what if newspapers could charge a modest fee commensurate to the value an article link is to Google? Over time, the fees could certainly accrue. The question, I think, is how high these fees are and how this sort of regulation is imposed.


I think the problem here is that this goes against the basic premise of the web itself and hypermedia. What about my blog where I post the (in my opinion) most important news of the day and link to the sources. Okay, those links are manually curated. What if I write a script that automatically posts (tweets?) all the articles I visit. Ok I am not a search engine but where do you draw the line? My point is that this is very hard to define and write down as a law.

Is a discussion necessary? Absolutely. Do we need a strong press? You bet your ass we do. Do we perhaps need to restructure the current model and fund the fourth power? Yeah, I dont know. Is this the way? Certainly not.

But I think this is only another symptom of a trend currently manifestating where people with ties and no insight into the matter try to control and profit from a medium they are just beginning to understand. The ITU conference, this guy in the UK who wanted to sue everyoneone retweeting an article,....


the French courts were wrong several years ago when they allowed some newspapers to sue Google for simply posting a few sentences or a link to an article. But what if newspapers could charge a modest fee commensurate to the value an article link is to Google?

I don't understand the distinction you are making between these two situations.

Think about the larger precedent that you're setting here, however. Should I not be able to quote a few lines and link you to a news story without paying money to the source I'm linking to? (whether the license was compulsory or not). What is google news but a factual stating of "here's what a bunch of sources said about the news today"? If a major event happens in your home town, how much money will it cost you to put up a blog post saying, "here's a roundup of coverage on this event"?


Just for the record, French laws about "droits d'auteurs" and quotations allow you to quote any text under "droits d'auteur" as long as it remains a quotation (the definition of which is vague, and varies from one case of "jurisprudence" to another, but 300 words is generally considered reasonable)

This sortof is an essential preamble to free speech. Now, i agree with you, there's "quote" and there's "quote".


[C]ontent of whatever form -- be it newspaper articles, images, whatever -- has inherent value that should be recognized.

I don't buy the assertion of inherent value. There's a ton of crap out there.

If there's any value in content it's subjective, not inherent.

People who are so convinced of the value of their content are free to erect a paywall and rake in the cash.

But what if newspapers could charge a modest fee commensurate to the value an article link is to Google?

It' perhaps something to consider, but it has to be a consensual act, where the value is negotiated. Google may already be providing commensurate value by driving traffic to a site.


If it had zero value why would Google be interested in it?


Google's value is in its comprehensiveness. Both the marginal value and cost to maintain a single piece of content in their index is extremely close to zero.


> If it had zero value why would Google be interested in it?

Again, value is subjective. Even if you think it has zero value, even if every engineer at Google thinks it has zero value, it might have value to someone out there. If nothing else, it could be a part of a corpus linguistics study.


> I think a good analogy, fifty years ago, would be a newspaper that sends out people to read other newspapers very early in the morning (say at 5 AM) and then produces its own newspaper at 6.

Newspapers and newsmagazines have always been very mercenary/cavalier about this sort of thing. Especially pre-web but even today, newspapers routinely rewrite stories from competing papers without even the slightest mention of where they got that information, radio shows discuss today's print headlines and read excerpts, readers' digests (e.g. The Week) pay no license fees for the stories they summarize and it's an old lament in newsrooms that local TV news just copies whatever's in the paper that day.

By comparison, an aggregator that pulls headlines from news sites and links them back to the original content seems rather innocuous.

(Of course, I don't intend to argue that because journalists steal, aggregators shouldn't feel bad about stealing either. But it puts things into perspective. And if headlines and links aren't fair use I don't know what is.)


After last few years here in Poland I've stopped believing in good reporting. No matter what you say, press is going to present it the way it wants, and it's impossible to break the glass and say to public what you really want. If there's order to show you're stupid, you will look stupid, no matter what you say. If there's order to show you're great - you will look and sound great.

I wish traditional massmedia fast demise. There's nothing there worth saving at this point.


I'm very disappointed with Europe. There is a trend of idiotic legislation drafted, & voted on by people who don't really understand the implications. They just don't understand the internet & technology enough and/or aren't smart enough. It' embarrassing.

The other recent example was/is the brain-dead UK/EU cookie law. Is our privacy any more protected? No. All we got was some dumb generic "cookie policy" popups that we have to "agree" to and further balkanization of the web. Additional costs to having a website (are there any scaremongering companies offering auditing to make sure your website does not expose you to legal risk?). Disempowering people by raising the barrier to running a site.

All downside. No upside from any perspective. We can't even blame lobbies or interested parties because literally nobody got anything positive from this. Just pure stupidity.

They should have know better than this.



Thanks. It's relevant though that nothing like this is likely to come from the EU itself.


Re: 'Disappointed with Europe', please see recent FOIAs to the privacy body http://www.asktheeu.org/en/body/edps and subsequent to 'Home Affairs' (new EU body, ~immediately after introduction rubber-stamped US-friendly surveillance laws) at http://www.asktheeu.org/en/body/dg_home

EU had, and perhaps still has, the promise of a strong political body as a counter-balance to US and Chinese interests in the world. IMHO the promise is rapidly fading though, under what appears to be high-level influence and manipulation that demonstrably favours US-friendly outcomes. This is not a new phenomenon.

Tried to integrate results with Wikipedia, contacted the EFF, forwarded to journalists, nobody will write up the results. Trying reputable university-affiliated Wikinews journalists now. :/


Step 1: Make Google pay for including you in search results.

Step 2: Force Google to include you by demanding something like search engine neutrality.


Step 3: Google requires payment from German publishers to be included in the search index, exactly offsetting fees incurred in Step 1. It would be quite reasonable, I think.


Step 4. Only commercial sites from Germany are in Google, cause it's too much paperwork to have all these fees canceled if you're not doing this full time.

Both Google and newspapers in Germany are glad from such resolution, only some naive young people complain about something. They're surely pirates.


As this site is nearly down, I add my thoughts as a comment here as well:

Well, some media (FAZ) did a pretty good job, letting Frank Rieger explain the "Leistungsschutzrecht". OK, even there, it was one article of many. And only one.

And what is new, when it comes to the press not publishing anything, that goes against their own agenda. Even across a lot of publishing-houses. Well nothing new under the sun.

What is really, really bad, is the fact, that the law is so fuzzy, that everyone quoting from another source might be potentially liable. This law is so bad, because it just might kill the independent voices. And I think there might be a reason for this.

Because the press oftentimes has no incentive to dig deeper, to ask critical questions, when it comes to the really important questions, this job is left for the independent voices, that do this out of a feeling of necessity. But if these voices are silenced through fear...

... well, I think you get the drift. And I know, this sounds a lot like conspiracy - and I am not saying, my thoughts come anywhere near the truth, but I just wanted to share the thought.

Just one example: The so called "Netzsperren" (blocking sites, because of content with filter-lists - aka censorship) were reported by the big media as being bad, after a lot of independent bloggers had written about it and the discussion just could not be "ignored" any longer.


Some have argued that even just linking to an article might get you sued. After all, most URLs include the title of the article. Obviously, courts will have to decide, but the possibility alone is frightening.


Some years ago here in Germany there has been an court sentence, that a link is no infringement. >The publishers wanted to kill Google News in this way.

Some years before that a friend of mine had written a news search engine, that did exactly the same as Google News, but he did not have the money to fight before court. So he had to remove a lot of publishers from his index.

So I do not think, that linking will become a problem here - but who knows. But the real problem is, that this law might silence independent bloggers. And it was them, who brought important topics into the public discussions, that were ignored by the big publishers, like a planned web-censorship law, that was discussed some years back...


FWIW, I've written to my member of parliament [0], expressed my dissatisfaction and asked him to oppose the law. He wrote back saying hat his parliamentary party is already opposing this law but for slightly different reasons: It won't increase the quality of journalism and will just create a flood of lawsuits. Lastly it is far too vaguely phrased as to not have grave side effects.

The changes of the opposition are slim though as there is a conservative majority the parliament.

Interestingly, he has a personal axe to grind with Axel Springer AG as he was a big part of the student movement of '68 which was so intensely vilified by said company ("Youth in the street - Germany going down the drain ...").

[0] http://en.wikipedia.org/wiki/Hans-Christian_Str%C3%B6bele


fuck ströbele ! he voted in favor of the war in afghanistan and yet he still participates in all the peace rallies. He's just another lying corrupted politician


This is why Germans must vote for Pirate Party at the next elections.

Oh, and I've noticed Twitter shows snippets from websites, too, now, so this will affect them as well.


I would really like to like the pirate party but in their current state they are absolutely unvotable for me. This is one of their core topics but I havent heard anything from them yet. What I have heard is that the youth organizations of almost every party (yeah I know, including the young pirates) have voiced their concerns regardings the lsr.


That does seem strange. Any idea what is happening to them? Are they going through leadership change inside the party or policy change, since that last scandal with one of their female leaders?


People closer to the pirate party might disagree, but from my outside perspective it has much to do with procedural problems the very nature of the party brings with them. They are primarily concerned with themselves. Lets take flat hierarchy goal for example: A good thing in theory which leads to a very tiring process when it comes to making decisions.

On the last big conference for example there was a last minute petition for the further research of time travel. And it has to be discussed. Sure, it was dismissed relatively quickly and it was meant as a joke but those were 30 minutes lost for good.

Apart from that, and this is just my personal opinion, they are missing likable leading figures (a concept they dont really like). Head of the pirates Bernd Schloemer for example is more of a silent leader. One of the most famous pirates Christoph Lauer comes across as rude, uninterested and smug. A social behaviour 101 wouldnt be that bad for some of the members.


the pirate party certainly has some procedural problems. they would probably be the first to admit just that. But those problems stem from the fact, that they are trying to implement completly new procedures for political assemblies. procedures that take advantage of modern technology instead of just replicating an essentialy unchanged process from the 18th century. And if they should succeed, it could redefine democracy.


As does Facebook and Google+. If you put a URL into your post it'll download the page and put in the title, a thumbnail picture, and the first sentence or two.


Unless you're a fan of individual liberty and don't like state controlled redistribution of wealth.


Redistribution of wealth is necessary to a certain extent, to prevent social inequality from growing. This is a fundamental goal of the German social state as defined by the constitution.

If you don't like state controlled redistribution of wealth you should start a revolution or emigrate.


The "extend" to which the German PP wants to redistribute wealth includes a basic income guarantee (without any requirements). Everyone would get X € per month. Fun fact: They decided that they - as a party - want this, but have no idea how to finance it.


Warum habe ich nicht es gewusst? ;) Hast du einen Link?


I'm very curious to know how they would determine how much Google should pay, and how to distribute that money to the different publishers.

Anyway, I think that Google, if the law is passed, should refuse to pay and stop publishing news from publishers from Germany (there are always still Austrian and Swiss newspapers for German news).


Just because they share the same language doesnt mean the journalistic quality regarding local/national news is equal.

A german newspaper will always be more detailed and in-depth when it comes to national matters.


Of course the quality and quantity of German news on Google would worsen, but the point would be to see if it is Google benefiting more from the newspapers, or the other way around.


It would be interesting to see an official statistic from the big papers where they get their traffic from. I get my news by surfing directly to their homepages. I think I read somewhere that they get most of their hits through the google search although I cant really belive that.


I would imagine the bulk of traffic comes from headline news. Some people check the paper every day, some people catch a bit of news on the bus or TV or whatever, then google it.

I think the rest of the traffic historical articles, looked up by people trying to win arguments on the internet. I don't know how many times i've been pulled to the NYT because of that sort of behavior. I would guess all of that traffic comes from google.


I do the same.


Could newspapers actively allow search engines etc. to quote them? I am thinking of heise when I say this, one of the few utterly "good guys" where "we won't sue, you can trust us" would actually work.

Or would this law mean the state attorney can simply sue Google et al, regardless of the stance of the supposed victim?


IANAL but I think this doesnt fall under the responsibility of the state attorney since it isnt part of the "strafrecht"/criminal law. I guess the process would be comparable to how copyright violations are currently handled.


Oh, so I would assume without anyone to sue, there'd be no case? If that is correct, I would predict if these publishers get their way, those that make use of the law would paint themselves into a corner. So it's not just about protecting our interwebs, but also about protecting them from their own stupidity ^^


Interesting and sad. I don't know if it is possible but I am wondering if such a law could include an 'opt out' policy. Something which said to search engines and the like that they are allowed to snippet your articles.

Then have Google turn off indexing of everything that doesn't have opt-out enabled.

My expectation would be that the opt-out publications would flourish and the ones who had opted "in" would quickly die or decide to join the "opt out" group. I can't imagine anyone looking at their referrer links would think this scheme was a "good" idea.


robots.txt ?


yes. They dont want to block google or other crawlers. They still want their traffic, but they also want to get paid for getting crawled and listed.


Well I can only read English transcribed interpretations so perhaps a native German speaker can correct me.

As I understand it this is their argument:

Google crawls site X, a user visits Google News or Google Search, Google displays a page which has extracted content from site X as a link (usually the headline) and a "snippet" of what that title is referring to. Google also has advertisements on the page.

Now, as I understand their argument, The publishers claim Google makes money off someone who clicks on an advertisement on these 'constructed' pages. Google's news page is full of headlines and snippets that came from other publications. Google doesn't pay those publications, but the only "reason" that someone is reading the news page in the first place is because those headlines and snippets are on that page. Therefore Google should either share any revenue they got from the ad click with the people whose 'content' was on the page, or they should pre-pay for that content in the first place.

Does that seem like it captures it? If you agree that it does, then we can speculate that the publishers lobbying for this 'law' believe that in this economic transaction Google is getting a better deal than they are.

Except that they conveniently ignore the economic benefit they are getting from Google for telling the world that their web site has interesting content (or at least content related to the news interest or search interest of the web searcher).

Presumably these publishers make income either through sales or advertising on their web site. And those sales are proportional to the traffic at their web site. Google could charge to include them in their search/news results (and they in fact do that in search with AdWords) and would it be more or less than the papers would charge to use a snippet?

The easiest way to educate a publisher on the value of having their results appear in Google is to stop having them appear in Google. Ask any web site that was knocked off the first page by the Panda update how that feels. Those guys really "get" the value they receive by being up there. Publishers don't get that yet. (well not all of them). So Google stops indexing them. Their traffic goes back to pre-1995 levels (which means nobody goes there) and their internet costs (hosting, etc) now exceed the revenue from online advertising. Whoops! Education achieved.

Of course I could be totally off base here, there could be some moral argument I'm missing but frankly I think its all about the money here and not all of the 'value' is accounted for.


You get it right, from the parts of the debate I followed this is their main argument - "they display extracts of our texts, and people then don't visit our page but stay on Google News", which is of course unprovable.

(you could get the amount of people clicking through by stopping to list these snippets - the visitors you loose are the visitors that used to click through)

Another thing to add is: Google runs no ads on Google News, so _directly_ they don't make any money using other people's news (indirectly by binding customers etc.)


In addition: Google has no ads on the google news pages.


Well why should it be that way? If I want to get traffic to a new website I have to buy that traffic in - through ads or other mechanisms that cost money. Why should Google not charge them for the traffic they send instead? The value Google brings to the papers is far higher than the value the papers' content brings to Google.


@ChuckMcM That sounds right, but to entertain another perspective for a second, maybe the point being made is that the amount of power Google has over publishers is unhealthy. The Publishers seem to think that search engines should be fighting for their business, not the other way around. Its hard to tell what such a drastic change in the system could do, but maybe it doesn't have to be bad. I for one believe that in order to have the most accurate search results, the rankings should be based on content, not money.


Ok, lets go with that for a minute.

"Maybe the point being made is that the amount of power Google has over publishers is unhealthy."

Framing it in an adversarial way may confuse the issue. The #1 challenge for any business is getting customers. When ever a business emerges that has a large influence on customer acquisition, it annoys businesses.

Lets re-frame the debate into one from the last century. In the US the telephone company knew the address and phone number of every business in a city by virtue of providing the service. One of the ways they leveraged that is they would publish a book, called the "Yellow Pages" which listed every Company and their number. Because the information was collected into one place, the it gained economic value (information economics being an interest of mine). It had so much "value" to customers that it was the first place they looked for the phone number of a business. Businesses realized that for generic things like plumbing, locksmiths, auto repair, being at the front of the list was better than being at the end of the list. Since the list was lexicographically sorted you started seeming names like AAA Locksmith, and A1 Plumbing. Hacking the name to be at the front. Then the Phone company decided to offer up "ads" where you could place an add on the same page as the listing, now even if your name was Zlotnicks Plumbing you could put an advertisement on the first page where plumbing started. That got you business. The complaint then was "Since everyone uses the Yellow Pages, I am forced to pay high prices to get an ad in their pages just so that people will know I exist."

By that same logic businesses may end up paying search engines to appear (and they do for Shopping links according to Bing's Scroogled.com web site).

And this comment: "I for one believe that in order to have the most accurate search results, the rankings should be based on content, not money."

I can totally agree with that, but I may internalize that differently that you do. I think that giving a search engine a choice "pay us or don't crawl us" they will simply opt not to crawl. But to understand why that makes sense economically you have to think about how the information involved gets its value. In this case a collection of 99 versions of a news story for 'free' is more valuable than 100 versions of the same news story. And its worth is exactly zero for uninteresting stories.


If they succeed they will simply remove themselves from the discourse and accelerate their obsolescence.


Sure, and I would welcome that. But I think smaller publishers (including bloggers) might have problems, too. The law would apply to their blogs as well, they would have to figure out how to allow Google and others to still quote them. It might be so complicated that it would cause a lot of blogs to shut down.

Other news processors were already mentioned in the article. I just fear that it would make it too costly to experiment and would kill small publishers.


Would it work to just put your blog under a permissive distribution license (Creative Commons or the like)? I know nothing about the German legal system or the exact phrasing of this law; could someone with more domain knowledge comment on how this would interact with licensing terms?


I don't know the exact phrasing either. Also not sure if everybody wants to put their content under CC.

I think the government could make this messy enough to even foil the CC plan. For example they could set it up like GEMA for music - there would be an entity distributing the money and collecting it on behalf of the content providers. Besides, is there a standard way yet to announce CC content? That would be necessary to make it work for search engines.



And then when your blog quotes or links to some other site (let's face it, a huge amount of blogging is discussion of news stories ...)? - they can't put that under CC and they might even be asked to pay for the privilege of linking to the story - they basically get hit both ways.


Blogs and smaller such institutions could license their headline and snipper 'content' for a nice fee of $0.


The question is, how do you license it? What if licensing it costs a government fee of 10000$ for the bureaucracy?


Why is it that every time that something like this happens, Google is the only tech company that comes forward with a message? Doesn't having a free and open internet benefit any other company?


I feel that its because google is of a size where the idea of getting a letter from its lawyers is a scary prospect.


The bigger trend to this story is that information changed from something that was scarce to something that is now abundant.

What the German publishers don't seem to understand is that their once so valuable and scarce goods (information and news) are becoming less and less valuable. Looks like they are trying to defend a dying business model with legislation.

Herbert Simon said in the 70s: “A wealth of information creates a poverty of attention.” Google is a major dispenser of attention, and I think the German publishers are doing themselves a huge disservice by making it harder for Google to send attention to the publishers.

The big unresolved issue is: how can we finance good journalism in an era where the value of static information approximates zero very quickly? Maybe the answer is in moving away from static information to an information process as the product... Something that can't be copied easily.


Exactly right. I made a similar point here: http://news.ycombinator.com/item?id=4886376

Media companies are in the attention harvesting business NOT in the information delivery business. The marginal value of the information product is virtually zero. The marginal value of attention redirected to ads is some rate greater than zero - some rate greater than the cost of delivering that information product. So profit can be made. But the power is in delivering the attention, not the information.


From the first comment on that:

"What’s wrong with newspapers being paid for the content that they produce? No one has to use their headlines if they don’t want to pay for it."

What about fair use? Does that exist in Germany? I'm a quote geek for example. I love collecting "favourite quotes", giving a source (link if possible). I started out with the general quotes everybody knows, but of course I also copy and paste from the web in general, and sometimes I actually type what I read in a book, and translate it to English. Man, I even love talking about it. I love quotes.

Now, I consider that "fair use", and since I do it mostly in English the noobs left me alone so far. But I don't even know if there is such a thing as fair use in Germany... any ideas?


In the spirit of DRY, here's a link to the a comment of mine, 200 pixels away on this very page, answering that exact question:

http://news.ycombinator.com/item?id=4883289

But I suppose I really should have refactored the comment out into a top-level comment and linked to it from both. :)


Okay, that kinda leaves Israel since I'd never move to the US :P Thanks!


There is the "Zitatrecht" (quotation right), which allows you to quote short excerpts, provided they accurately represent the original text and you acknowledge the source. But unlike in the US this is only valid for texts, not for other copyrighted material like movies. ( This means that youtube remixes are most likely illegal in germany.)


The Economist talked about this a week ago in an article :

"Taxing times As newspapers’ woes grow, some are lobbying politicians to make Google pay for the news it publishes"

Not necessarily the end of the world (see Brasil) but might not have the intended consequences :

http://www.economist.com/news/international/21565928-newspap...


Frankly, I can understand the publisher's point, but I'm surprised that they do not seem to see how this could backfire if Google simply stopped crawling their sites.

But what's more: I think if this law became a reality it wouldn't affect my personal web usage at all. News sites are about the only type of website left where I still type in the URL and go to the page directly instead of doing a search.


I think the intuition behind these proposed laws is that Google has become so dominant in providing the information substrate of society that it should be subject to regulation as a monopoly. The argument would be something along the lines that search works best when provided as one comprehensive resource. Therefore, as a natural monopoly it should be regulated for similar reasons that other utilities are (electricity, water). The publishers are trying to make a case that their product has value as a "public good." Classical economic theory says market forces alone will produce a sub-optimal supply of a public good with out regulation.

Something like that. It's not a totally implausible argument. It does seem notable to me that Google is able to extract value from content that the creators can't.



I didn't listen much in high school English; is linking to a Google cache of this blog post ironic?


Only if it was a blog post lobbying in favor of the law.


No. When a site is down, it is customary on Hacker News for someone to go to Google and get the last cached (saved) version of the web-page from them to post it those that can't access it to read, as a sort of convenience. It's also common for the plaintext of the article to be posted here.


I meant that it seems like precisely the sort of thing that would be outlawed by the proposed legislation.


IANAL but trying to read the law, it seems to be a rather blatant lex google.[1] They specifically state "search engine or similar services" should be prohibited from (in an extreme interpretation) linking to newspapers, if they do not pay. ( The rest of the law is probably just redundant, since it seems to reimplement copyright for a small subset of already copyrighted material.)

[1]http://dip.bundestag.de/btd/17/114/1711470.pdf (German)

[EDIT: spelling]


Google should state a new policy. We're not going to spend our money on servers and bandwidth indexing and linking to German publishers who won't pay - I guarantee you the value of the traffic they get from Google dwarfs what Google gets back from them in terms of marginal revenue directly attributable to their content. That's the thing about Google. They make tons of money - but given that they're a prime mover of traffic around the Internet they make a damn site more money for other folks - including newspapers.

So Google should charge. Then they can do like the phone companies with termination fees and call it a wash.


The proper term for pushing this law is "rent-seeking". You must include our content, and you must pay us for it.

This is a general problem with people and businesses perceiving themselves as working for society. They cannot see the wrongs they cause - because it's all for an even better cause.

German newspapers can use robots.txt and obey the social contract of the web like the rest of us.


If the newspapers could make a profit putting pictures of cats riding unicorns on every page with any text written in swahili, they'd do it every day it worked for them and not give a damn about the societal benefit - they'd cry that they're businesses and they've got to be allowed to make a profit. This is exactly what's happened over the decades as they cut international staff, and investigative staff and focused on lifestyle and entertainment news more and more.


>You must include our content, and you must pay us for it.

Where are you getting the first part from?


This ultimately comes down to the fact that newspapers are going out of business because not enough people want to buy them any more, and they're lashing out in desperation.

I think there's a simple argument with content and markets that all of these media companies disingenuously ignore. If no-one is prepared to pay for your content then the market rate for your content is, by definition, zero. We pay in two ways - direct purchasing, and delivering surplus attention beyond that required to consume the media, which can be redirected towards various forms of advertising. When you cannot run your business on advertising revenue alone it means that not enough people are giving enough surplus attention of enough value to cover.

This argument about the "inherent value" of one form of content or another - of the need to pay artists or creators, or in this case journalists - doesn't extend to other realms. If I decide I want to be a carpenter, I cannot build a table and demand someone buy that table for $1,000. If the market won't bear that price for the table then the government isn't going to force others to pay me $1,000 because I feel that's what it's "worth".

The market rates for all forms of media have plummeted due to there being more supply of attention-draining media than there is demand either in terms of hard-dollars in direct payment, or surplus attention to be redirected towards paid advertising (and its ilk).

When old media companies had a stranglehold on distribution because paper production and distribution was expensive, or video production costs were prohibitive, only a small elite were able to publish their opinions, and the monopolistic distributors were able to charge a premium for the limited access to information or entertainment they provided. They weren't paid well because they provided an outstanding product (though it often was), they were paid well because they limited supply. Those limits are gone. Many many people produce entertainment and informational content. Many do it just for fun and are happy not to get paid. Many more do it with the hope of getting paid anything without the expectation of the lavish salaries and expense accounts of journalists of old.

This undercuts their economics and doesn't even touch on the fact that the newer voices often offer media that is more attractive to younger audiences. Not to mention declining quality of the product in many cases. Many media companies as they've become bloated monstrosities have undermined their own product quality with short-term-profit-focused decision making which had long term harm.

Is it really the case that piracy accounts for all problems in a record industry where the giants spent the pre-Internet-boom '90s crushing independent labels, monopolizing market channels and creating a modern-day payola system on radio where programming was rigidly sliced and diced to the lowest common denominator? Is none of the loss of popularity of the New York Times down to their abandoning their predominantly liberal subscriber audience during the Bush years and being guilty of mis-leading story after mis-leading story in the build up to the Iraq War, destroying people's confidence in their role as a reliable neutral arbiter?

tl;dr The publishers referenced by the OP aren't happy at the market rate for their product and services and want government to rig the market to pay them a cushy wage. Such subsidies rarely save industries in the long term, and the public should be outraged - because legislation of this sort is a public subsidy on a privileged elite in no uncertain terms.


I think the point that is unnoticed is that medium matters. More specifically, medium & distribution matter.

Mediums beget business models, not content. Game Of Thrones has business model closer to X Factor than to Avatar, because similarity of the medium is more important than similarity of the content.

The best way to see this is by looking at the history of porn, the content industry with negative lobbying power. Adult cinemas had the same business model as (and competed with) live shows. Certain types of content got produced. Then home videos changed it entirely. DDifferent types of content got produced for different people. Then DVD + online sales, changed the industry (growing it again). Then live streaming shrunk it.

Each time the medium & distribution changed the whole industry changed. Different producers, different consumers, different content, different size industry, different business model.

Online consumption of news media is not the same as dead tree. That is reality. All of the characteristics of the industry change when that happens. Douglas Adams said it best "It's like trying to explain to the Amazon River, the Mississippi, the Congo and the Nile how the coming of the Atlantic Ocean will affect them. The first thing to understand is that river rules will no longer apply."


Is there no such thing as Fair Use in German copyright law?


"While many other countries recognize similar exceptions to copyright, only the United States and Israel fully recognize the concept of fair use."

http://en.wikipedia.org/wiki/Fair_use#Influence_internationa...


This is entirely misleading, because it makes it seem like the policies in other countries is questionable with regards to fair use, or perhaps with lesser rights.

In France, for example, there is no such thing as fair use, but that's only because 'fair use' in the US is not defined as what you're allowed to do, however, but as a sort of vague statement about perhaps being able to use things.

In France, it is strictly codified: You are able to make private showings of copyrighted material You are allowed to make private copies (with a few exceptions, e.g. software backups allowed, but copying your poster so you can hang it in two different rooms is illegal, as is copying your software to two different computers so two different people can use it) You can make copies of non-educational works for educational uses (ie. no photocopying textbooks, but you can copy yesterday's newspaper article all you want) Parody Citations Etc.


Isn't this a case for the internet bat signal? I must admit I still haven't included it on my sites, mostly because last time I looked they only provided JavaScript hooks making my site vulnerable to their site being hacked. Perhaps they have an API by now?


They should google ''Wyatt's Torch'' before voting.)


Why is this topic so complicated? A Google news snippet very rarely would suffice for a user, if they are interested they would click through, if they are not, they would not visiting the newspaper anyway. Google are not showing ads in Google News, although they might show them in search results. But still, no profit that newspaper could address is being taken by Google. In fact, Google is providing traffic.

So taking all the philosophical statements out of the way, what is the econimical case that the newspapers are making? If Google actually showed a bigger part or the whole articles, they'd obviously be taking revenue from newspapers, but it's not the case.


Right. Back in the pre-web days when I was a mere teen I would spend breakfast, and often late afternoon going through the English broadsheet newspapers. Usually back to front to read the sports first.

But the key is that the first read, the morning one, involved skimming articles for their gist and reading the ones that interested me. The afternoon session possibly involved reading more, due to boredom.

The real problem the newspapers have is that most of the articles they produce fail the skim test. I don't want to read more. I get the gist. I'm not interested. If they're good I click through and read them on the original site... often then clicking to other things I find there. Win for the Google link to them. If it's a hot topic I'll often read many articles from many pubs on the same topic. But in a world with so many news sources the truth is they don't stand out, I am not going to check a thousand sources very day, or even ten. So I'm not going to wade through to find the things I'm interested in. If they don't want their headings and summaries in Google I don't care about their product.

And I'm a Google user. I pay Google's bills. I don't want them raising my rates in other areas to pay publishers for links to content that I may skim over and never want to read in depth. Why should my costs for Google - in terms of what they can deliver me for all that surplus attention of mine they harvest over a year - go up to subsidize failing newspapers producing product for which the market rate doesn't support their costs?


But it's Google who's making a big deal out of this, not the publishers. They could have just delisted the publishers without a peep.


Do they really thing this will stop the continuing decline of their newspapers?


I'm sure some lawmakers have heard of robots.txt, so what is their rationale?


Greed?


Technology generally finds a way around poorly written laws but I think there is a real underlying problem. Google News has become its own thing now. The headlines, snippets and icons I get from a variety of newspapers is often enough to skim over the day's news and I only click on maybe one or two stories for more in-depth reading. I think a way for Google to charge news sources and for news sources to charge Google both make sense but they have to be developed by willing participants, not by legislators.


Nothing scary, they just are aiming carefully to shoot their own buttocks with some good German schrapnel. If there will be no readership for them, they will not be able to sell ads and die in 3-5 years, depending on long term contracts. And this is a good thing. Since you can use free Russian and Urdu headlines for free! Oh, I forgot to mention Chinese.


It would seem that Google could quickly kill this law by de-indexing all the relevant newspapers, right now. Why are they waiting?


No one likes to burn bridges, even if they get sued. They could do a "this is how google would look if you go through with this" day, but for now they look to be taking the more safe route with lawyers.


So, in other words, the posters defending Google are being totally disingenuous when they claim that Google couldn't care less about these petulant little publishers and that the publishers need Google far more than vice versa. Apparently, Google really doesn't want to take these publishers off Google News.


Why would that kill the law?

All it would do is make Google news useless for German news.


My guess is that if newspaper sites lose a big chunk of their traffic they might reconsider who should be paying whom. I'm also guessing that without support from newspapers, the law is less likely to get through.


Presumably the newspapers are aware this might happen happen.

Then (in their minds) people will just go to the actual newspaper to see the headlines.


This is rather interesting that it would be coming from Axel Springer. As someone involved in the Berlin tech scene I've seen them really investing a lot of money in innovative events and companies. My guess is there is some typical corporate disconnect there. I doubt this makes it through the legislation.


which is why the law proposal, called “Leistungsschutzrecht” (hashtag #lsr), is currently being discussed in the German Parliament

Next time a German native speaker tells me his loves his mother tongue because of the possibility of concatenating words together, I will point them to that hashtag.


That statement doesn't make sense. If they didn't contatenate, the string would be even longer and less useful as hashtag, or it would be concatenated again, as most hashtags in English are concatenated words...


Note that the hashtag they are using is #lsr, not #Leistungsschutzrecht .

The irony is that they are not taking advantage of concatenation, in a language register where it is widely used (twitter), and in a tongue where it is idiomatic (German).


what is really scary is not the proposed law but the public relation campaign google is mounting against it. It is a (very likely) unprecedented internet lobbying effort involving not only googles advertising network, but also a link on google.de (no longer visible). To a casual observer, watching the short youtube clip or reading the campaign slogan "Willst Du auch in Zukunft finden, was Du suchst?"("Do you want to continue to find what you're searching for ?") it could seem like the government was trying to shut down google search.

It seems like a scary prospect to think of the ways google, a private company, could start to use its unparralled reach to lobby for legislation in its favor around the entire globe.


I think our decentralized streams protocol should solve this kind of thing.

Basically RSS with push and access controls. You subscribe to a feed and get pushes / pulls as long as you are paying it. Why isn't there a standard protocol for this on the web?


I will put a contrarian point of view (just for the sake of discussion):

The law is good because Google can afford to pay and the hordes of blogspammers can't.

Therefore Germany will be the first country without blogspam.


> blogspam

You have a weird definition of 'blogspam' if using a headline to point to an article you're commenting on qualifies as 'blogspam'. There's nothing in the law (from what I've seen) that says "If you have more than a paragraph of text, this doesn't apply to you" or anything similar to that.


The reason newspapers are sacrificing their credibility for money is quite simply because they're running out of money faster than they are running out of credibility.


This is scary? I expected Nazi zombies!


Here are a few points to consider:

1) The notion that the systematic clipping of one or two sentences from every news story you can find on the Internet for the purpose of selling ads is "fair use" is totally absurd.

2) The "robots.txt" argument misses the point because the publishers never gave permission to use their content, so the presence of an "opt-out" mechanism is irrelevant. The mechanism must be "opt-in", i.e., Google must ask to use their content.

3)If we believe that intellectual property has become an obsolete concept altogether, we should be prepared to accept all of the consequences of this, e.g., a site might pop up using a new TLD that accepts a search query from a client, passes it on to Google using a spoofed IPv6 address, and passes the result back to the client. We also shouldn't care about plagiarism, trademark infringement, etc., because the Internet has clearly made these concepts obsolete and laws related to them unenforceable.


And here's why those points don't matter:

1) The debate is centered on Google News. As far as I know, News is one of the few services where Google haven't, don't and won't show ads. "Systematic clipping of one or two sentences [...] for the purpose of providing an ad-free overview and linking to the respective stories" sounds like "fair use" to me.

2) They are publishing their material on the web, for the world to see. If you'd want an opt-in for every crawler out there, no search engine would ever be able to take off and index a significant portion of the web. By the way, an opting out from all (obedient) crawlers is as simple as "Disallow: /"

3) Passing a query on to google through a spoofed IPv6 address doesn't make any sense. Apart from that, there's several services offering just that functionality. Try DuckDuckGo, for example. (I'm not quite sure which search engine they use nowadays, but afaik, the project started out using Google.)


2) There is no such thing as "fair use" in Germany's copyright law.


>The debate is centered on Google News. As far as I know, News is one of the few services where Google haven't, don't and won't show ads. "Systematic clipping of one or two sentences [...] for the purpose of providing an ad-free overview and linking to the respective stories" sounds like "fair use" to me.

Then Google should have no issue with not linking to publishers who don't want to be linked to. As an aside, I disagree that it's fair use, because it's done systematically as part of a commercial enterprise.

>They are publishing their material on the web, for the world to see. If you'd want an opt-in for every crawler out there, no search engine would ever be able to take off and index a significant portion of the web. By the way, an opting out from all (obedient) crawlers is as simple as "Disallow: /"

If you believe that Google is infringing the publishers' copyrights, then it is not the responsibility of the publishers to opt out. See my response to your first point.


The "robots.txt" argument misses the point because the publishers never gave permission to use their content

What, they put it on a public directory with semi permanent URLs for no particular reason? That's a pretty flawed argument!


There's a difference between linking to something and using an except from a published work without permission. (Yes, it would be fair use in isolation, but not as part of an automated, systematic process that's part of a commercial enterprise.)


> 2) The "robots.txt" argument misses the point because the publishers never gave permission to use their content, so the presence of an "opt-out" mechanism is irrelevant. The mechanism must be "opt-in", i.e., Google must ask to use their content.

The crux of the matter is that you're missing the newspapers motive. They don't want Google to ask their permission, they want google to pay them, which is a huge difference. They DON'T want google to ask for their permission and then delist them from Google News if they had to pay for it.

I am on Google's side on this matter, this sounds like extortion. Google should be free to delist them if they want money from their link, in a capitalist society you should be free to chose who you deal with in your business. I live in France, our government wants to tax google to help the newspapers and they didn't take google's answer (that they'd delist the newspapers) well, because they don't want google to respect their IP and ask for permission, they want Google to pay up no matter what.


>The crux of the matter is that you're missing the newspapers motive. They don't want Google to ask their permission, they want google to pay them, which is a huge difference. They DON'T want google to ask for their permission and then delist them from Google News if they had to pay for it.

I haven't missed their motive. Of course they don't want to be delisted from Google News. In a sense, they're like patent trolls, but with more of a moral high ground. Unlike patent trolls, they have produced something and sent it out into the world. It is not their job to preempt what they rightfully view as copyright infringement, even if that infringement might benefit them.

>Google should be free to delist them if they want money from their link

They are.


1) Normal search at google takes one sentence, title of the webpage and links them together. To support that service, they have ads. If news take two sentences, how is that totally absurd compared to one, and why is search totally acceptable but news search not?

2) as other has said, public sites are indexed in an opt-out system. Opt-in would not work, and would destroy any form of search system, indexing, yellow pages, and so on.

3) This is not an intellectual property issue. Its a term of use issue if one want to be included in google search. One can not say yes to be included in search, but not if there are ads around. Either you want google to give you service, or you don't. I wish I could get a gemail free from spying, a google search thats anonymous, and free telephone calls without any recording/advertizements, but no. I can't get that from google, because their services to have a price tag.


1) If someone wants Google to stop displaying a one-sentence excerpt from their page, Google should respect that.

2) I never said that indexing should be opt-in. I said that the use of excerpts should be opt-in.

3) I should think that someone can decline to allow their content to be used to generate ad revenue for someone else.


> the publishers never gave permission to use their content

Yes they did, yes they did. The publisher explicitly granted access by putting up the news content in an public subdirectory of a public webserver. News items don't just magically `pop up' in the public subdirectory of a webserver. Usually at least two employees (writer, reviewer) have to log into, and explicitly perform several steps in an CMS to publish a news item.

The decision of publisher is encoded as HTTP server configuration (including CMS, if any). The server obviously provides access only to some resources, as the operator wishes -- the ones meant for public consumption, while other resources are password-protected, only accessible from certain network addresses, or not accessible over HTTP at all.

Just as much as an image resource can be, and often is, protected from deep-linking based on HTTP `Referer' header, a news item can be protected from being indexed by Google News bot and displayed as a snipped in Google News based on `User-Agent' header.

In more details:

A typical User Agent that HTTP GETs some resources from the server describes itself with a `User-Agent' header. Google is honest here, and openly indicates its user agent as a spider bot, with explicit purpose of indexing content for web search.

Even better, Google uses a /separate/ spider bot for Google News service, and it is entirely reasonable to serve the general websearch bot with content -- so the website itself remains indexed -- and deny the news bot access, to prevent Google News from displaying news snippets.

Upon HTTP request, the server reads through request, including resource URL, headers (including User-Agent and Cookies, if any), compares that info to configuration. If, and only if, it matches configuration for unrestricted access, the resource is served. Otherwise, access denied condition is indicated and optionally authentication requested.

Let's assume for a moment the law goes through as requested by the press. The most probable implementation would be, Google includes a particular header in HTTP requests -- indicating itself as a `news-bot-that-has-subscription', and the server would, in turn, compare the header with configuration and either reply with news contents, or perhaps indicate access denied, should subscription expire, or for some other reason. The mechanics is pretty much same as it would be today to restrict Google News bot from gathering, and Google News from displaying (some or all) news snippets.

In short, the publishers have already configured some resources as protected from /some/ User Agents (based on request URL, headers, cookies, network address or anything else), and it is entirely their own choice they serve the news to both Google bots (websearch and news).


>> the publishers never gave permission to use their content > Yes they did, yes they did. The publisher explicitly granted access

I'm failing to see how "read access" and "license to redistribute for profit" are the same thing


That's a strawman; the bots never indicated intention to read. They indicated (via `User-Agent' header) intention to index content and got HTTP 200 OK response to that.

Nothing in the HTTP protocol specs (RFC 2616) even suggests it is only meant for direct human consumption. To the contrary, it states,

  user agent
      The client which initiates a request. These are often browsers,
      editors, spiders (web-traversing robots), or other end user tools.
...and the Google bots clearly identify themselves as indexing bots.

Please don't be obtuse. The whole matter is not about Google doing something strange or different than any other search engine. It's just about collective bargaining of one business group against another business. Some german press companies stood up as a (self-appointed and thus somewhat suspect) representation of whole german press industry and are trying to get a bigger slice of the cake.


If I don't lock my house, that doesn't give you the right to break in. You're absolutely right that publishers could make more use of the tools at their disposal to protect their content in whatever way they wish, but they have rights regardless of whether they do.


This analogy only muddles the picture. We are talking about private property, but not a home, more like a shop or pub.

Its only raison d'etre is to serve certain guests. We are talking about a commercial server that is managed by a professional admin, and is explicitly open to only some of the public, and it already screens and filters guests, based on various criteria. For example, the server may disallow hotlinking of images, serve some content only upon receipt of a HTTP cookie confirming payment and serve different content for mobile devices, some movies blocked in certain countries, based on IP geolocation. The decision is based on HTTP request data and will of the owner, encoded as server's configuration. Ask yourself, how much different is Google News bot?

How about this analogy instead: ``if I hire a professional bouncer for my high-street shop, and the bouncer stops some people from entering and lets others in, that doesn't mean those let in can indeed walk in''?

Which would be crazy, and press would have a field day with such a shop.


Some access control is really easy to implement, other types are not, or they're not cost-effective. It's perfectly reasonable for a content provider to combine technological measures (like e.g. IP geolocation) with legal ones (terms of use, copyright).

That said, you're absolutely right: how hard can it be to block the Google News bot for a news organization? Not very. My point is merely that you can't draw any legal or moral conclusions from a 200 OK response, that's absurd.

To riff on your example, if my high-street shop has a sign that says "only people over 18" but doesn't have a bouncer, does that mean people under 18 should feel free to enter because nobody's physically stopping them?


"That's a strawman; the bots never indicated intention to read. They indicated (via `User-Agent' header) intention to index content and got HTTP 200 OK response to that."

The publishers aren't objecting to being indexed. They are objecting because snippets of their articles are being copied, placed on Google News, and used to sell ads without their permission, and they're not being compensated.


How are profits generated on news.google.com?




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: