Hacker News new | past | comments | ask | show | jobs | submit login
Audit suggests Google favors a small number of major news outlets (cjr.org)
455 points by hhs 9 days ago | hide | past | web | favorite | 383 comments

There's a bit of mixing of correlation and causation here.

Most reasonable people would consider Google to be an inferior search engine if it surfaced results from the Daily Caller, Kos or Breitbart above those of NPR, BBC and the WSJ.

Search engines therefore surface the more authoritative / linked to sources first. These smaller sites are trying to achieve through regulation a level of prominence that they can't get in the marketplace.

It seems though that they now surface "authoritative" sites over clear user intent in their search.

For instance, search for "has Infowars ever been correct" on Google and then Duck Duck Go. For Google, you will see results that have nothing to do with the clear intent of the query, such as Wikipedia and mainstream news articles trashing Infowars. For Duck, you will get several results that actually answer the question.

Since Google pioneered natural language processing for search queries, and these sorts of queries worked fine years ago, the only conclusion is that Google is actively burying certain results to manipulate the narrative. Why does their natural language search break all of a sudden when I want to find out something that is unpopular?

Doing that search gave me a Quora link as the first result, and then two links to infowars...which is an awful result.

I'm seeing 2 out of 10 links as Infowars, links #3 and #4. The top two are Quora. mediamatters and rational wiki are examples of other front page results.

They all directly address the query. Why exactly are they "awful"?

You seem in your answers to be prioritizing "direct answer to the question regardless of reliability" rather than "useful information about the question's subject." I suppose there's an argument to be made for that, but it's an argument that leads to prioritizing a search engine that will return the answer "orange pinstripe" to the question "what is the color of the sky" over a search engine that returns a scientific article about how sunlight behaves in atmosphere.

(In any case, this all seems a bit orthogonal to whether Google is right or wrong to prioritize widely-recognized news outlets over smaller ones when it surfaces searches for news articles, doesn't it?)

I don't believe it to be orthogonal. There is a deeper philosophical question that has barely been touched on what search engines should return in response to user queries, which also applies to surfacing news. In the past with Alta Vista, word matches were the heuristic and it was extremely obvious as to how to measure the quality of the result. Now you've got very abstract heuristics, such as intent of the user, trustworthiness of the source, whether the source is healthy and good for society, the correctness or honesty of the content, how much revenue will be made with particular set of results, etc. and the public is being left out of the discussion as to which heuristics are important to apply.

How about as general rule of thumb, I tell search engines what to search for instead of the other way around?

That's wild. My results look like this: https://i.imgur.com/AtrMoLs.png

The person you are responding to is referring to DDG.

Infowars won't actually accuratley answer of it's been correct so Google is following the users clear intent and giving them good answers

You will get results that respond to the question. That's not the same correct answers.

How could a search engine ever know what the "correct" answer is? Relevance is more important.

In any case, Google's results for this particular query are certainly less "correct". They are more of a non sequitur.

> How could a search engine ever know what the "correct" answer is? Relevance is more important.

I strongly disagree, IMO actively surfacing a highly-relevant but incorrect result is worse than not surfacing anything at all.

As for determining what the "correct" answer is, the authority of the source is certainly a good place to start. Obviously nothing is fool-proof, but Yahoo! Answers is certainly less likely to be correct than Wikipedia, for example.

Let's continue with my particular query as an example. I already don't trust Alex Jones and Quora and Yahoo answers. I have my own opinion as to which sources I trust and which ones I do not. So first I would internally give different weights to results based on how trustworthy I find them. Then, I would look at the actual content, and look to cross reference the details to assert their validity.

Now with the Google results, there is nothing even remotely relevant to my search, so I don't even have candidates with which I could do further research into their veracity.

Once again I think Google just thinks they are smarter than the average user, but it's making their search engine useless for certain queries.

And I don't like the idea that a search engine should be doing the critical thinking for users. That's more dangerous than the content they are supposedly protecting us from.

So your problem is that Google considers what you consider to be trustworthy sources to be neither relevant nor trustworthy? Have you considered the possibility that what you consider to be trustworthy sources are neither relevant nor trustworthy?

The problem is that I don't trust Google to define what "trustworthy" means. I'd prefer to get results that match on more clear heuristics and make the decision on trustworthiness myself.

The Google algorithm: sites gain reputation through the number of links to them from other high reputation sites.

I don’t want Google to have an opinion. I don’t want Google to second guess me.

We are a long way, I hope, from, “Hey, Siri, what’s your opinion about....”


The Chinafication of the US is complete. The gatekeepers know better than us and should give us the results that reinforce their echo chamber rather than the ones we are looking for, and the public has been convinced this is the right thing to do.

If this were happening 15 years ago, people would be outraged instead of supporting it. What kind of nonsense is this that the search engine should reinforce political stances? My political beliefs are my choice, and if I want to reinforce them, that's my prerogative. The search engine should not make that choice for me. Well unless you are baidu.com and backed by a one-party system.

It's not the responsibility of search engines to engage willful ignorance with a plurality of perspectives or debunk fringe political nutbaggery that stands opposed to the precepts of civilized society, but I'm glad they mostly do.

Regardless, that should be a personal choice to make.

Google has to rank its results somehow, in order to limit the number of results returned to some sensible amount (rather than returning literal hundreds of GBs of links for common queries, if it considered them all to be equally "first.")

Other search engines are no different.

You make your "personal choice" of ranking algorithm, by choosing which search engine to use.

"if you don't like it then leave" is not helpful. There's no good reason a search engine couldn't provide an interface to let you choose how you want to rank and filter results. In fact there used to be an advanced interface in the past the provided a crude version of this.

A great benefit to Google would be that SEO would be hard to game because everyone has different filters. That is already sort of the case but it should be the user, not Google, deciding which narratives and aspects to filter.

The trick is to include a variety of results early. Sources often can be grouped into blocks which return mostly the same results. Better to reduce the quantity of returns from the top block in order to include on the front page a few from the second and third tier.

Or maybe a text search equivalent to Yelp’s “re-search in this area” (after narrowing the map). Perhaps to be able to select some results and hit a “more like this” button.

>For instance, search for "has Infowars ever been correct" on Google and then Duck Duck Go. F

I suspect Duck duck go's results come from Bing based on the result similarities.

"For instance, search for "has Infowars ever been correct" on Google and then Duck Duck Go." Wow. Completely different results.

Right, the assumption is that if Daily Caller, Kos, and Breitbart became more popular then Google would surface them as authoritative.

First, there's a popular belief amongst their readership (for whatever reason) that this is not the case and that Google actively works against sites like these. Furthermore, if it doesn't currently work against these sites, it's taken for granted that if (for some reason) Zero Hedge became more popular than WSJ Google would work against it.

Secondly, if it were the case that Google would surface Zero Hedge over WSJ purely because of popularity is that ok? Is that what should happen? If Google does want to curate rather than using a "blind" algorithm does that change the way it's regulated and what it's responsible for?

I think this is something we haven't quite figured out as a free society. We recognize that an educated citizenry is a vital requisite for our survival as a free people (please don't over index on who the quote is attributed to).

Yes there are a multitude of media outlets on the internet which the citizenry can use to educate themselves. Are there moral hazards when the majority of the citizenry arrive at one or two doorsteps in-spite of the multitude of alternatives? What is the risk if what's served from that doorstep is detrimental to the republic?

> First, there's a popular belief amongst their readership (for whatever reason) that this is not the case and that Google actively works against sites like these.

Breitbart in particular has run a long series of articles featuring leaked discussions from within Google where there are calls internally to remove or penalize Breitbart from organic listings, and deactivate their AdWords.

They also discussed censorship on Facebook and Twitter amoung others.

They run these under the umbrella “Masters of the Universe”:


They should do so. Breitbart isn't really a news site; if you wanted to dignify it, you'd compare it to Salon, which also doesn't belong in news summaries on Google.

That wasn’t meant as an endorsement of Breitbart, to be sure. But parent raised the question of why people think this, and trust in media is a crucial point these days.

Breitbart articles discussing Google censorship aren’t radical alt-right preaching. You can strongly believe that Google is making informed and conscientious decisions about censorship and rankings on their platform, and then read a different perspective on Breitbart to understand the other viewpoint.

For example, in response to a question as to why when doing an image search on Google for “idiot” that Trump is the result (Currently he is 3 of the top 10 results. I mean, that is pretty funny actually) Sundar Pichai swore to Congress that Google does not manually intervene on any search result. Is that really credible?

Maybe it’s a little hard to quantify what is a “real” news site anymore. Some people assume this is self-evident, but I am skeptical of everything I read.

I sometimes find it entertaining to see what slant opinionated news sites will put on a story and compare it against the MSM.

>For example, in response to a question as to why when doing an image search on Google for “idiot” that Trump is the result


Is Sundar and Microsoft working together now? But wait, are they also paying off Duck Duck Go??


These conspiratorial arguments that search engines are politically biased stems from technical illiteracy and/or a lack of critical thinking. The arguments are used as a distraction by certain people who want Americans to disregard any new information that may paint them in a negative light - and it is working on a subset of Americans.

That example was meant to be funny. That’s not to say there is not evidence of direct intervention in search rankings when the algorithm results were undesireable.

It’s beyond question that Google manually intervenes in Search results in some cases. That could be entirely innocuous or concerningly dubious. The question is exactly how, how often, and should they be accountable for it?

>It’s beyond question that Google manually intervenes in Search results in some cases.

If it's "beyond question" then it should be easy for you to provide evidence, right?

As with most conspiratorial arguments, you're trying to protect your opinion by using a false premise (and prevent people from questioning it).

Why is it beyond questioning?

> Breitbart articles discussing Google censorship aren’t radical alt-right preaching.

That may be narrowly true of a particular article but it isn't true of the newspaper as a whole.


Bannon himself called Breitbart "the platform for the alt- right".

Real news sites post reported news. Breitbart digests news from real news sites. It's essentially a glamorized group blog, like Daily Kos. In fact, Breitbart is basically the right-wing version of Talking Points Memo, another group blog dressed up to look like a news site.

Fox News is a sharply right-wing news site with a very strong editorial bias. I don't like Fox News, but I can't argue that it doesn't belong in the same bracket as CNN. Breitbart does not.

This is mostly true. But, for example, they’ve interviewed Trump in the Oval Office. So not entirely un-credentialed.

You’ll say; This says more about Trump than it does about Breitbart.

No, I'll say that group blogs are a legitimate thing, and sometimes they get big-ticket interviews, and every once in awhile they even break a story. But that doesn't make them news organizations.

I'm not saying that sites like Breitbart should be buried (boycotts, though, seem fair game). I actually like Talking Points Memo every once in awhile. But I don't use blogs as my primary news source and Google shouldn't promote blogs as if they are.

CNN, MSNBC, WSJ, WaPo, and the Washington Times are not news sites either. They are mouthpieces in journalist clothing.

Interesting contention - who are they mouthpieces for?

One easy example is WaPo and the time they posted 16 negative Bernie Sanders stories in less than a day during the Democratic primaries:


Ever since Bernie has made Amazon a target, considering Bezos owns both. For example he was able to berate them into increase their minimum wage. Some sources:




Just like most big tech, the leadership is in bed with the corporate democrats, who are in turn in bed with the military industrial complex, big pharma, and Wall Street.

Do you have any examples of what you would consider to be legit journalism?

theintercept.com, therealnews.com, Max Blumenthal, Dan Cohen, Aaron Mate, Abby Martin, Caitlin Johnstone, Rania Khalek, and Kyle Kulinski are just a few.

Which they should

While it isn't quite as bad I'd compare Breitbart more to Der strummer the Nazi Propaganda rag then to partisan newspapers or blogs.

Racist and bigoted propaganda is what you'll see over and over in it's pages. Stuff like claims islamists burnt down Germany oldest church.

I'd say it's Google's societal duty to censor them

> Yes there are a multitude of media outlets on the internet which the citizenry can use to educate themselves. Are there moral hazards when the majority of the citizenry arrive at one or two doorsteps in-spite of the multitude of alternatives? What is the risk if what's served from that doorstep is detrimental to the republic?

As it turns out, we have figured a lot of this out. It turns out that a good way of deciding what is worth people's attention is to treat a link as a vote. We shouldn't be surprised or alarmed when that surfaces a few main choices, as most of everything is crap.

However as happens in almost every sphere of life, there's a good chance that the good stuff rises to the top (that would be that meritocracy that right wingers say they're fond of). Outranking the BBC for news is hard - it should be.

What its not for you to decide - even if you do suddenly start to decide to write in the style of John Stuart Mill - is that you like some ideas better than others and that you want to start forcing things on people that they neither want nor asked for in the name of 'balance'

Unfortunately, that’s not a good system. It would be an okay system (the phrase “worst possible except for all the alternatives” refuses to leave my mind on this) except for the fairly major problem that links and clicks can be automated, turning that into a bidding war. Propaganda of the rich, by the rich, and for the rich… and whoever they want to manipulate.

And that’s even without the problem that the people running the papers/channels are themselves both powerful and capable of having agendas that don’t need to be aligned with those of their readers.

Unfortunately, “what is true (news|science|morality|politics|economics|history)?” is very much not a solved problem.

> except for the fairly major problem that links and clicks can be automated, turning that into a bidding war.

I mean no disrespect by this but I'm inclined to think Google knows more about the issue of link and click fraud than you do.

Oh indeed, and I didn’t mean to imply otherwise. Most of what I know about it comes from their engineers explaining how they fight it — and that includes my awareness of how catastrophic it would be if they did nothing and only used the original PageRank algorithm without compensating for it.

Counting links isn’t a meritocracy, it’s a popularity contest. Those are not the same thing.

There is a reason that publications in Nature aren’t decided by Reddit votes, and it’s not because nobody can figure out how to integrate the two.

The question is: Is a simple popular majority a good way to decide what news all people should see?

It’s a great way to see what the majority echo chamber wants to hear, but that’s not really a good way to have a well-informed population.

> Secondly, if it were the case that Google would surface Zero Hedge over WSJ purely because of popularity is that ok? Is that what should happen?


Obviously discard any SEO trickery. But if Zero Hedge is being cited more frequently, being chosen from search results more frequently, etc. then people have found it more helpful, and presumably other people would find the same.

The whole premise of democracy (both the form of literal governance and the general philosophy) is that most of the people are correct most of the time. There exist other algorithms to combine human opinion, but they are wrong more often.

Yes, most political pundits were wrong in predicting Clinton wound win in 2016. But I don't think it makes sense for Google to try and "manually" curate that.

Zero Hedge is a conspiracy blog published by anonymous authors. If people are clicking it in search results more frequently than actual news sites, that is in itself a problem.

What's an "actual news site" though? Major news outlets also regularly publish unfounded conspiracy theories (e.g. russia gate, WMD in Iraq, etc). The question is do you like yours flavored with anonymity or government funding?

The point is that its not a search engines job to make that determination. People will decide for themselves and whatever the algorithm is, it should seek to remove the personal biases of the implementers and be blind. It should seek to serve the asker not change the asker.

I feel like there could be a term, similar to "uncanny valley" where instead of detecting the in-human traits of something attempting to appear human we see bias in something attempting to be unbiased.

If you pre-suppose that news orgs are the same thing as conspiracy-mongers, you're going to have a hard time defining news sites as distinct from not-news sites. Most people don't start from that assumption and find the distinction useful, though.

I don't believe justinmchase pre-supposed anything.

They fairly pointed out the evidence of mainstream news sources promulgating conspiracy theories.

I missed the evidence, let alone evidence.

> russia gate, WMD in Iraq

That's not evidence of anything, it's just an outline of someone's screwball opinion. Just like 'NASA is a hoax-making organization (Moon stuff)' does not contain any evidence but simply outlines, in shorthand, a screwball opinion.

They're not the same thing but sometimes they do still peddle conspiracy theories that turn out to be false. Many are susceptible to delivering fake news as if it was real due to a conflict of interest related to their funding model.

It is useful to make a distinction between news organizations and non-news organizations but it is not useful to apply any sort of value judgement of either based on that fact alone.

Again, you're trying to conflate getting things wrong or inadequately accounting for bias or expectations with 'peddling conspiracy theories' or 'delivering fake news'. These things are not the same and that's one of the key distinctions between real news organizations and ones that aren't.

Consider the search phrase "Is the Earth flat?"

If Google detects more engagement towards "Yes" and flatearther conspiracy websites, should their search engine prioritize results that say the Earth is flat?

> If Zero Hedge is being cited more frequently, being chosen from search results more frequently, etc. then people have found it more helpful, and presumably other people would find the same.

> The whole premise of democracy (both the form of literal governance and the general philosophy) is that most of the people are correct most of the time.

1. Google's clickthrough metrics are subject to severe sampling bias.

2. The premise of democracy is seeking compromise across diverse opinions. American democracy has explicit protections against tyranny of the masses.

If NYT is regarded as an authority by a diverse audience across broad search domains and cited by other similarly broadly authoritative sites, while ZH is cited and viewed by a high-engagement but narrow and isolated audience, then NYT should be ranked above ZH. As it is.

> If Google detects more engagement towards "Yes" and flatearther conspiracy websites, should their search engine prioritize results that say the Earth is flat?


Realize however that this is entirely hypothetical, as there is enormously more round earth information with extremely popular sites like wikipedia.org, nav.gov, etc.

My faith in democracy is strong that I believe this will always be the case, so long as the earth remains round.

You're setting up a straw man. Google's 'favorite' site was CNN, ending up as 11% of all impressions, nearly twice as high as any other single site. Quantitatively CNN has seen their viewership plunging to the point that they now have well under a million primetime viewers. [1] Their results on Alexa [2] show a similar online trend, and that is very much in spite of the fact that Google is constantly 'recommending' them which provides is providing a major inorganic boost to their traffic.

[1] - https://www.forbes.com/sites/markjoyella/2018/05/30/fox-news...

[2] - https://www.alexa.com/siteinfo/cnn.com

It's almost as if there is a discrepancy between an audience that uses television as their primary news source as opposed to searching for it via Google. As if they might not be the same people

And you are also straw manning.

> "Their results on Alexa [2] show a similar online trend, and that is very much in spite of the fact that Google is constantly 'recommending' them which is providing a major inorganic boost to their traffic."

And it's not remotely attempting to use legislation to surface all sites.

It's a push for a narrow set of extreme sites who are extremely unpopular because they are conspicuously fact free misdirection toward scapegoating. They naturally fall behind reality based sites.

Nor is it unusual for that group to try to use the force of law to elevate, legitimize and eventually mandate particular propaganda. The only variation is which countries go down this route and which don't.

You are claiming “reasonable people”, which is a legal term, and googles algorithm, which is a computing term, are the marketplace. Neither of these are the market.

The whole idea of googles algorithmic approach is wrong and it’s real purpose is to let google manipulate what people see, rather than letting them choose for themselves.

People should be able to dictate to google how to find information, but instead, google wants to dictate to them how to find information. As has been mentioned already in this thread, and can also be seen on YouTube, google actively overrides users own specified preferences.

A clear example on YouTube is the “trending” section. It is basically googles “we want to present a Disney like site, with a focus on mainstream media”. This goes against the actual marketplace. Actual videos that are trending but not advertiser friendly or whatever else, will not be shown in this section.

Google wants everyone to see “authoritative” sites, because it is a propaganda model. The ministries of truth, even though they are clearly not even valid news sites. Most of what cnn puts out is politically-motivated propaganda. I still remember “the stock market will crash if trump is elected”. Hah..

> Most reasonable people -> Most ideologically progressive people

Not that there is anything wrong with being ideologically progressive, but both sides of the political spectrum view themselves as reasonable because they are starting from different set of axioms.

WSJ is not progressive.

WSJ historically has not been, but these days it's a bit of a mixed bag.

Kos is not right-wing.

It is if you go far enough to the left

What about the fact that if you search: "The donald reddit" then reddit.com/r/the_donald is not in the first page (or 10) of results?

edit: I do see it first if I search "donald reddit"

You have to remember your search results are personalized for you.

Top result for me.

Interesting, for me the top result is r/TheDonald. The trump supporting one (r/the_donald) doesn’t show up at all.

This is totally fake.

Google has to MANIPULATE it's search results to put NPR first.

The smaller sites are linked to more often but Google doesn't want you to see them for political reasons.

The smaller sites don't want regulation - they want and end to manipulation and censorship.

And most reasonable conservative people see NPR,BBC at least as garbage extremist left wing outlet, and would not want to see them at all.

> The smaller sites are linked to more often but Google doesn't want you to see them for political reasons.

Google has never claimed to order search results by the number of links to them. Otherwise the Flash installer (or whatnot) would be at the top of every search results page. :)

Err...that's the exact basis of the original google search algorithm, and what made it so good. You should read up on it, you might learn something.

Reasonable conservative people see NPR and BBC as extremist left wing?

I am told I am annonyingly centrist. While not 'extremist' in the casual sense, I very much see a hard bias to the left that often seem unfettered by the facts by NPR. BBC seems to have its own agenda, however not quite 'left' by american standards. Read any recent Brexit coverage to see the worrying doublespeak that seems to be de rigour today nevertheless. Before you ask, I honestly am unaware of any news org without bias, although many have enough writers and/or a soft hand to obscure systemic detection by casual readers. Everyone who works hard to publish something, by proxy seems to have an agenda beyond factual reporting. If they well-meaning or not seems almost a bygone conclusion. Information has never been more weaponized.

Yeah, Infowars may have the best coverage of a story, but that doesn’t do Google any good if it’s abrasive to profitable users’ expectations. By pushing good content down they can keep the users happy who 1) click ads, 2) shop more online, 3) subscribe / engage on platforms. The algorithms aren’t designed to find the best content, but that which aligns with Google’s goals (eg profit).

> The algorithms aren’t designed to find the best content, but that which aligns with Google’s goals (eg profit).

And it so turns out that finding and surfacing the best content aligns well with the number of people who link to that content, which creates a great product for Google, which aligns with Google making a profit.

You can't argue from a hypothetical where Infowars offers the best content, as Infowars doesn't offer the best content. If Infowars started offering better content than BBC, people would start linking to Infowars, and Infowars would rise in the search results. It's not Google's fault that a lot of Infowars content is low quality that people don't link to.

> And it so turns out that finding and surfacing the best content aligns well with the number of people who link to that content, which creates a great product for Google, which aligns with Google making a profit.

The lines between Google "watching the market" and Google "commanding the market" are very blurry. If you don't get featured, nobody knows that you exist, nobody links to you, you don't get featured. Otoh: if you get featured, everybody sees you, people link to you, you get featured.

This is a hugely important point. Google’s algorithm was genius at the size they were 10 years ago.

Today Google is the tail that wags the dog and a purely popularity based algorithm is more concerning.

However it’s also unquestionably true that Google results are not purely a popularity contest, and then the question becomes whose hand is on the tiller and which way is it steering?

Clearly in this case it is steering towards mainstream media outlets which all slant a particular way. This does not have a small impact.

>> And it so turns out that finding and surfacing the best content aligns well with the number of people who link to that content,

It's like saying Burger King offers "the best" food, because it's popular.

And there's not really a reason(besides a little bit of money) for Google, The global monopoly, to insist on doing that. It's not that hard to have an easy option for quality content.

For example: Google's forum search, which was cancelled.

This argument might fly better if people were posting actually good but small news sources as examples. Lets not pretend that Breitbart or Infowars have any sort of integrity here.

For news stories that are posted on high quality forums(like hn or /r/science), offering a forum link seems like good value.

> It's like saying Burger King offers "the best" food, because it's popular.

Not really; no-one goes to Burger King because they think they are getting the "best" food. They think they are getting convenient, affordable and quick food. You certainly could argue that Burger King is enormously popular because it is very good at offering those three attributes.

Depend on how you define "best". For me it's value, and Burger king is up there.

I'm reminded of what Rory Sutherland once said about McDonalds: the one guarantee that you get with McDs is that it isn't going to make you ill.

The one thing I can guarantee you is that McD's makes me ill. Every bloody time I ate there, mostly on the road in the US where there is nothing else for a long way around. Only the fries are safe.

Well, at least everyone agrees that it is _consistent_.

Not always :( the maccas on the freeway stop near me has horrible chicken nuggets, while the one in the local shopping centre has good ones.

who has not had the bubble guts a time or two from Mickey D's?

The analogy doesn't work because all the content on google is free to users. I would argue that in a world where 5-star restaurants cost the same as Burger King, they would experience more demand than Burger King.

Putting aside who offers the best content entirely, that simply doesn't logically follow.

You can observe that an objective unbiased media source in any realm with a biased and obfuscated view of reality would do less well relative to one that catered to that biased and obfuscated view of reality, which is why media sources even in the modern age are localised and tailored to the audiences which they are aimed at, prejudices, unfounded beliefs and all like MEMRI in the middle east, christian fundamentalist media like WND, etc. If you expand the scope of enquiry to all of possible human history you can very easily imagine that the media sources which would and have historically done best in the more ignorant periods thereof are obscenely biased and not at all "the best content".

If you think that BBC / NY times / WaPo ad et al are not guilty of the exact same kind of thing merely from a different perspective, well that's quaint and charming, but simply completely inaccurate. Nobody has the full story, everybody is wrong, and putting together the puzzle pieces on any issue requires extensive survey of a broad variety of perspectives and sources, and even then, you're going to fall victim to your own biases and simply become another part of the tapestry to boot.

There is no winning. There's just losing less badly.

This is basically backwards

Infowars never has the best coverage

Because it's a at best a crazy conspiracy site, and at worst a far right conspiracy site.

This is actually a massive problem with YouTube in which Google optimizes for time spent watching and ad time which drives a lot of people to far right and conspiratorial content. And a lot of people are complain I that Google needs to fix thay

Comparing views held in 1994 with views held in 2017, it is clear that nothing is driving a lot of people to the far right. Quite the opposite is happening. A sort of relativity may be in play here, making it seem as if other people are moving to the right if the observer is moving to the left.

Histograms of the change, for each party and year: https://imgur.com/QJd2gRB

> Infowars may have the best coverage of a story

Has there ever been an example of this?

Can someone explain why this is any different from the fact that almost every Train Station in the USA has the same couple newspapers available (NYT, WSJ, USA Today, etc.)?

If the problem is the monopoly on how people get their news, then shouldn't that be what is solved?

> Can someone explain why this is any different

No they can't because it isn't any different.

The article is suggesting bias. There is likely some bias at play but I doubt it is anything nefarious. And it isn't enough to account for the natural bias inherit in the system of news parroting.

> shouldn't that be what is solved?

I don't know. All I know is it likely isn't something I trust anyone at Google or the Government to get right.

These algorithm on the scale of Google seems to have a feedback loop, i.e. what Googles links is what will be linked much from elsewhere favoring big players.

>seems to have a feedback loop

How is that different counting quarters in the newspaper machines? Or copies sold?

Physical spaces are much more limited real-estate wise, whereas Google has as much space as they do page loads. Surely with so many consumers and producers, a much higher breadth of news can be exposed than is currently.

No they don't.

The fundamental challenge in search is that you have two limited resources: display place and user attention span. Google won vs other search engines because they were better at figuring out what users were looking for and putting it higher in the listings.

Anyone arguing the reverse - that Google is driving opinion, not the other way around - is saying that they're massively sabotaging their core feature yet no competitor has been able to exploit this.

Fox News obviously has a large TV audience, but I'd be curious how that demographic maps to google usage. And it's worth remembering that it's not a "fox news vs cnn" comparison vs a "fox news vs the sum of the less-right-wing channels" comparison.

Why can't it be both? Google won because they were the best, yes. But there's always the possibility of feedback loops. Just look at YouTube - discovery on there of smaller channels has become almost impossible, with bigger channels being reinforced constantly. Granted, video is much harder to understand and surface to users than news. Surely there should be enough content that it's more than a tiny minority which succeeds?

Every major platform is struggling with this.

> Surely there should be enough content that it's more than a tiny minority which succeeds?

Isn't this what has always happened historically with any new media platform, from print to radio to online? Storage space got cheaper, but human attention remains limited, so popularity and feedback loops are a human problem, not a technical one.

One issue I see is that it's much more pervasive. I have no idea what the USA train stations' newspapers are, since I seldom take the train. Also because I live in Europe.

Google on the other hand, has a much bigger reach.

There's very little marginal cost to Google having more news sources.

Huge downside risk of ferreting out "fake news" and borderline-disreputable outlets in the name of diversity in news, however, which they already get blamed for.

Which is exactly what increased transparency can help with. Make the ranking algorithm or data public, or at least explain to users why some content was ranked as it was.

Downvoters: What is bad about transparency? Fake news proves what we have now isn't perfect. More eyeballs on an algorithm could be a stepping stone to solving the problem.

They just demonstrated that HN comment scoring needs a bit more transparency too.

(edit) I think every downvote should be public with their name and reason behind it.

Critical thinking skills are what we need to focus on, most people don't know what an algorithm is.

Why not both?

Something Steam has been doing is breaking down why a particular game is recommended (reviews, popularity friends, interests, other metrics,..) why not expose this data to users for news?

We build these systems to filter out the noise for us, but that doesn't mean we should be beholden to a black-box deciding what content we should see. The filter should be ours to tweak and knowing how it works is a first step to that.

Usually when someone breaks out an analogy like this it's because the comparison is not like for like and there is something they want to obfuscate.

Quite simply, because Steam is not Google. Steam's value is in all the games you want to play being on Steam, not in its discovery algorithm. It loses nothing by exposing that algorithm because even if you had it, you would not be able to build Steam as you do not have the games that are on Steam. It has vendor lock in.

There's no such lock in for Google. Google doesn't have exclusive rights to the websites it indexes and anyone could, it they had an algorithm as good as Google's (and the computing resources) build Google.

The only tech factor holding Google in place as the #1 search engine is that its algorithm is better than everyone else's. The algorithm is intellectual property that Google has invested billions of dollars in developing.

Providing the weightings (and allowing as some people are suggesting the ability to hobble Google's algorithm by using your own) would be the first step towards someone reverse engineering Google and be a material business risk for them.

Google's search algorithms are their secret sauce intellectual property. Why would they share them with you?

But then SEO firms and everyone else would know how to game the system with certainty. Obviously not what you want.

The train station is not algorithmically deciding what to show you. The station has brokered a business agreement with the newspapers in the newsstand. I am unaware of any deal Google has with CNN to make sure a percentage of their articles show up in search engine results.

What is stopping this from happening? What is stopping Google bombs?

What is preventing us from being smarter? From going out and educating ourselves? Thinking more deeply? Finding the truth? Introspecting?

Where do we get our thoughts from? Who guides our behavior?

There is a struggle going on right now to regain control of the human mind.

It used to be just a few major news/media outlets. Now it's the entire world.

Someone has to be in charge of the information we are given. Else chaos will consume us all.

No one is in charge any more. We have to fix that. People must be told what to think again. They are thinking too many different things to predict the future anymore and that is disconcerting to those in the old power structures.


> The train station is not algorithmically deciding what to show you. The station has brokered a business agreement with the newspapers in the newsstand.

....what do you think a business agreement has in it to describe the costs and revenue sharing?

Because Train Stations don't double as the largest search engine in the world.

It's not a monopoly. There is an acceptable amount of competition. There is nothing to solve.

I think it reflects the way the sites manage SEO.

Fox optimizes for user browsing more than the other major outlets.

How about an analogy with recruiting:

Should googles tech workforce resemble that of the broader US? i.e. Would having a workforce that resembles the US signify an elimination of bias in the hiring process.

If you believe that Google’s workforce hiring should be free from bias, than can’t you extend the same argument to search results?

For those in favor of bias in search results, may I ask you to justify why?

That's not an analogy, but making an bias argument (spreading FUD) without evidence.

An analogy is the train station newspaper example. What you're doing is arguing that all of the Google employees are inserting their own personal bias into search results, without evidence.

I want to spur a discussion on whether bias in search results is ok or not. I’m trying to provide a counter example to argue the point that I believe that google search results should be free from bias by arguing using a corollary that hiring should be free from bias. I do not intend to spread FUD.

Can you argue in favor of bias in Googles search results? In an ideal world what should it be?

The search results are already biased, and they should be. That's the basis of Google Search, that it provides a biased, more useful result based on how many pages link to it.

I'm not sure what you're asking. Do you want me to make an argument for bias? Why?

Good. The 21st-century internet has created a crisis of credibility. We've learned that "information wants to be free" and "The Net interprets censorship as damage and routes around it" applies just as well to lies as it does to factual information. We are in a storm of misinformation, and have been for many years.

I don't love the 500lb gorillas of mainstream media, but they have skin in the game if they get caught pushing falsehood. Look at men who've trainwrecked their careers over getting caught in a lie - Olbermann, Dan Rather, etc.

Can you imagine smaller no-name media sources facing any real consequences in that kind of situation? No.

We need credibility again. Until somebody finds a way to algorithmize credibility, I'm happy with saying "only the big boys are assumed to be credible by aggregators".

People who have a different worldview are probably just going to bury this, but in case it is useful information to anyone, I will write it anyway.

To make it easier for people to accept the premise, let's suppose that we are not concerned about the United States. Say we are worried about some other countries.

The issue is that when only a relatively small number of the largest media companies show up in the results, that makes it easier for state or corporate interests to control the information stream that citizens receive. If smaller, independent outlets were surfaced more often, it would mean that in order to push any particular narrative effectively, there would be that many more media outlets to control. It could make it much more difficult for a government or special interest to propagandize, since they would need to influence a very large number of independent outlets.

And again I realize that many people here may not believe that government propaganda still exists in American media. But I think that most can agree that it does happen in many other countries at least.

This assumes that an endless array of small media sources is more difficult to control than large ones. I'd say that's false. An endless array of small media sources can just as easily be all traced back to a single controller. That's actually the business model of Sinclair media, for example.

But because smaller media sources are forgettable and have fewer eyeballs checking them, there's no record when they do something unethical like the large companies.

Rathergate would not happen with some small, forgettable news source. The falsehood would be pushed and unquestioned because nobody wants to play whack-a-mole.

Boiling down credibility to outright lies is far too simplistic.

The 'big boys' have more than enough bias, and tricks up their sleeve to bend the story to their liking.

The problem is that "the big boys" lack just as much credibility in 2019 as anyone else. The fact that they constantly push government narratives doesn't make them any more credible. Google pushing of the "big boy" outlets is an attempt to make them seem more credible.

You're implying NYT has the same credibility as Infowars. Don't know in what dimension such a statement could be true, but it certainly isn't ours.

Also, the current government under Trump despises most of the big boys for obvious reason, so pretending they constantly push government narratives again highlights quite a distorted sense of reality.

NYT certainly doesn't have the same credibility as Infowars, but their name has certainly been tarnished.

Just look at Pulitzers awarded for the Collusion story that's been proven to be fake. Or the anti-semitic cartoon they were forced to withdraw last month.

If you go to the Pulitzer site, they link to the stories that the NYT was given the prize for. You should probably go read them yourself. None of them are fake. It was good reporting.

Somebody may have told you they were fake, but you should stop listening to them and go read the stories yourself.

Former NYT columnist and Pulitzer prize winner Chris Hedges now works for RT. Does that mean that RT is credible, or that Hedges has become less credible? Neither. It means that neither Google or the US government should be deciding which news outlets are credible any more I would trust Google to work in partnership with the Russian government to decide who is credible.

No I'm claiming that the NYT is less credible than they used to be, and not more credible than many other, smaller news outlets. Perhaps "establishment narrative| is more accurate than, "government narrative". If you are interested in understanding what this narrative is, go read Manufacturing Consent again or any of the dozens of books that have been written on the bias of US corporate media.

Bloomberg pushed the likely false China hacking story. Most mainstream media pushed the likely false Russiagate story. Doesn't look like they have much "skin in the game if they get caught pushing falsehood".

> IN THE LAST WEEK OF APRIL, nearly 23 percent of all traffic to news sites tracked by web analytics firm Parse.ly came from search engines. Google alone accounts for nearly half of external referral traffic...

Is this surprisingly low to anyone else?

Depending on how you parse it, only 10% or 20% of news sites traffic comes from Google.

When I worked in comparison shopping, 80-90% of our inbound traffic came from Google, as we failed and failed again to cultivate any loyal, direct users.

Hey, CTO of Parse.ly here. Might be surprising, but news sites get their traffic from 5 broad categories: (1) search engines (mostly Google in US); (2) social networks (Facebook, LinkedIn, Twitter, Pinterest, etc.); (3) editorial & recirculation (homepage promotion, article-to-article promotion); (4) direct, text, & email, which covers things like WhatsApp and manually shared email links; and (5) the long tail ("other"), which covers things like Google News, Flipboard, blogs, and site-to-site links.

These 5 categories are roughly equally split in aggregate traffic -- somewhere between 15-25% per category. You're right that certain kinds of sites, like e-commerce, are heavily weighted toward search -- but this is not broadly or necessarily true for the whole "content universe" of news, information, & entertainment sites, including blogs and so on.

Our data reveal all sorts of interesting patterns that go against mainstream assumptions about how people read/watch content online. For example, a measly 1% of traffic to content publishers comes from Twitter, even though Twitter certainly seems like it drives way more than 1% of the conversation, especially in certain categories of content. I wrote about that phenomenon here:


If you care to go deeper, one of our data analysts, Kelsey, did a nice deep dive on the different kinds of traffic sources that resonate with different content categories here:


Thanks for posting here pixelmonkey! I had a question; I asked the Parse.ly help/marketing people a while ago if you guys were tracking the rates of invalid traffic on articles and they said you guys don't track it. Is that accurate? Or are there any estimates on how much noise/dirt there is in the data?

Hmm, that's an interesting question, but I'm not sure I fully understand it. By invalid traffic, do you just mean, non-human (bot) traffic?

If so, I can say that over time, we have improved our use of bot lists, though that's just an IP blocking thing. Non-human traffic detection is not presently a strong focus of the company, though people have asked us to invest there. The issue is that non-human traffic detection is a somewhat gnarly problem in its own right, with its own vendors (mostly cybersecurity vendors) trying to figure that problem out.

We do know we are missing some traffic due to ad/analytics blockers and pi-hole style VPNs, which is fine.

One way we have thought about guesstimating "noise/dirt" in the data is to use one of our universally measured metrics, engaged time. So, we could separate really short page sessions from the rest, under the assumption that if a page session is super short, it's either a mindless human click or a JavaScript-enabled bot crawl. I discussed this on our blog awhile back when we did a data study on the subject:


In that study, we found that 32% of visits to pages were "bad visits" (page session <15s), a pretty high number, but that would include not just bots, but also humans queuing up tabs, Instapaper/Pocket saves, and so on.

Apologies on the terminology - by invalid traffic I'm referring to bots as well as click farms and other issues as used in the Media Rating Council's definition (they divide it into general and sophisticated invalid traffic both of which have a lot of types of traffic, http://mediaratingcouncil.org/101515_IVT%20Addendum%20FINAL%...).

I'm just a bit concerned that the Russian malware dudes were re-purposing their click fraud for astroturfing way back in 2015 and they had no problem just sitting and building dwell time instead of bouncing (https://www.trustwave.com/en-us/resources/blogs/spiderlabs-b...). I haven't been able to find anything indicating that US media companies have any kind of tracking to defend against or even identify a similar strategy being used to hit their article analytics to influence article production/placement, especially when it's now known that a Russian information campaign against the US was going on at the time.

I'd love to have us do better here and you sound very knowledgeable on the subject. Willing to reach out to me by email? ~email redacted~

When comparison shopping all anyone really cares about is the price, and you're comparing like for like in most cases so Google works really well. News is something where I at least rarely find myself wanting just any old opinion on a story, so I'm more likely to go to a news source that I trust already for their take on that.

? When comparison shopping all anyone really cares about is the price

I disagree - people care about delivery times, quality of goods (is it what I ordered), and in some cases, they are open to alternatives (I would like a cheap android phone, If you can give me a Nokia instead of a Huawei, I don't care).

Similarly for news, people may have some preferences in their browsing, but ultimately for breaking news it doesn't matter whether it comes from CNN or the India times if there's no opinion involved in it yet, and lets be honest, most short-form journalism has very little research/opinion

For long form/blog-like content I agree with you though.

> When comparison shopping all anyone really cares about is the price

While I certainly also care about price: it's not the only thing I care about. I care about length of warranty, user-serviceability and/or ease of return for defect or repair, and reliability.

Why should I spend $800 on something that the manufacturer only warrants is useful for 30 days? Why should I spend $80 on something that I can't repair using my own tools? Why should I spend $8 on something that breaks within a week and it's more expensive to return than buy again?

I take pride in the things I've acquired. I provide care and maintenance to them. I think only the poorest and/or un-savviest and/or lavish of people worry only about price.

> and you're comparing like for like in most cases so Google works really well.

I've found that most "marketplaces" provide really poor experiences for customers like me. Amazon is right at the top of the poor experiences. Google Shopping comes next in line. Even something like Newegg will frequently have some pretty iffy deals going on.

It makes you (me) really wonder about other "marketplaces" such as the ACA.

Not really, I would bet that twitter, reddit, facebook, and links from other news-like content generate the lion's share of traffic.

For news I go to a small set of specific sites combined with links from an number of social media sites like Reddit and HN for wider coverage.

I think it's very different than comparison shopping - for news what someone you trust considers important and what communities you care about matters.

I usually only look at Google if I want to dive into more sources about a specific news item, which is fairly rare, and usually indicates I have reason to mistrust that something is covered properly by my usual sources, or if it's something that for other reasons will not be covered by my usual sources (e.g. let's say some local news item in a location where I don't know what the trusted local media is).

For comparison shopping on the other hand, I want to find who can sell me something cheapest - if your site shows up in Google, then I don't have a reason to go to you directly vs. going to Google and getting others too.

Who needs to go to the news site anymore when google shows you the content already? I bet they are losing visits from that. With the other visitors to the news most likely bookmarking their favourite/trusted sites.

Because headlines are _not_ a substitute for quality journalism and despite falling ad revenue, people will still seek it out to some extent. It's why the EU link tax is a terrible idea - when Google News pulled out of Spain it damaged the online news industry badly[0] with 6-14% drops.


I certainly felt like Google does this for their 'Google News' product. I selectively blocked half a dozen sources I found to be excessively clickbaity and it really cut into the amount of news that was shown.

My guess would be that there are three things going on a least;

One is that news outlets are loath to let Google crawl them and post snippets, they argue that Google gets the 'benefit' without paying for the content. (this was used in a couple of lawsuits)

One is that Google may provide the advertising feed for news outlets that are shown, thus a double benefit to Google is that you go to their news aggregator, and when you pop off into a news story you continue to be served Google ads.

And finally, there is the polarization of people's news intake through a limited number of sources. Per my experiment I found after deleting the new sources I found offenses Google was unable to come up with additional "interesting" news sources, they just didn't crawl them or have access to them (or they don't exist I suppose). Regardless of the reason, the additional wasn't in their index to serve up.

Interesting you should say this, as I've found Google News to be "desperately local", to the point of excess. I'm all for regional news and mixing things up, but typically when I've traveled somewhere within the UK perhaps for a weekend, Ithen get inundated with totally inappropriate news from the area for ages, until I proactively block it!

Their algorithms can suggest amazing insight at times, but when you suddenly get stuff like "Local council changes bin days in Huddersfield" it's clear they have a long way to go to understand news relevance, especially when you were just passing through.

Before we get too deep in arguing about supposed favoritism by Google, let's back up and look at the types of search terms that were entered in this study. Just about all of them relate to the sorts of Washington-focused headline news that's primarily covered by "a small number of major news outlets."

Searching for "Rex Tillerson" is the perfect example. (Remember him?)

Typical news searchers are looking for a much wider swath of news than just the latest on the former U.S. Secretary of State. In my own experience, that includes local news (what were all those police cars doing last night?) and lots of sports news, business/tech/entertainment news, scientific news, etc. Those articles come from a much wider assortment of news outlets, and I think Google does a decent job of finding them.

There's a limit to how many news organizations can generate meaningful coverage about Rex Tillerson. I'm at peace with the idea that Google has found most of the ones that matter.

You'd be at peace if Google favored news and narratives that overlap with your world view. Hypothetically, how'd you feel if 65% of those articles were from Fox news and the greater Murdoch empire?

I wish they simply skipped narrative driven, opinion filth and stayed with factual reporting from AP, Reuters, NPR, ...

In the last 5 years, NYT, WaPo,.. have wholly embraced narrative driven opinions instead of sticking with plain news, because anger sells

65% of articles in my Google News (not logged in) are from Fox News and/or Washington Examiner.

I think geography affects search results a lot. I'm in a blue city, but red state.

Are you positive about that? I just tried to VPN from a few different locations in the US (Texas, Georgia and Seattle) with a fresh private browsing session each, all of the headlines appear the same and doesn't change by region.

I'm not positive about that. All I can tell you is that I never see WSJ, but I often see Fox News, WaPo, Politico, and Washington Examiner. The rest of the sources are a mixed bag.

I've been hitting Google News without a logged-in session (but with my real IP address) multiple times a day for 5+ years.

I noticed the increase in Fox News as a source around the time conservatives were complaining about censorship a year or two ago. Interestingly, though, Fox News is rarely the source for a major political news story -- it's often their fluff pieces that hit my feed.

The data presented in the article is the exact opposite of your claim. If you'd like to refute the data, please cite a better source than your personal anecdote.

I wasn't refuting the data. I was saying that my anecdote is an exception, and I have a theory about why.

The article's data was far from closing the book on this topic.

I would add BBC to that list.

Dear Google, I suggest that you embrace this kind of audit in service of transparency and credibility: Create an automated news.google.com/audit page so that readers can easily evaluate the sources and track how they evolve over time.

As a user I would also greatly appreciate the ability to toggle news sources on and off, and/or to weight them, and to easily share those filters.

That would be great, but you are asking for them to incriminate themselves.

I don't know about now, but during the Obama administration Google actively participated in spreading narratives. It's not even a secret. Ex: Syria.

>during the Obama administration Google actively participated in spreading narratives

I haven't heard of that, could you link me to where I could read more?

>As a user I would also greatly appreciate the ability to toggle news sources on and off, and/or to weight them, and to easily share those filters.

IDK. Do we need more ways for users to create bubbles for themselves? Is the solution to avoid Google's preferred bubble just to let you build your own?

I mean, yeah? You buy newspapers that you trust, you don't buy all of them.

What Fox News thinks is completely irrelevant to me since I'm looking for facts. I should be able to tell that to someone who's in the business of curating my news for me.

To mix metaphors, bubbles are essential sanity equipment when drinking from a fire hose.

Are you implying that to protect yourself - that you need to only see news sources that you think will already agree with your idea of how things are?

There is such a project. Proof media beta (https://proofmedia.io/) allows readers voting on the truth value and bias of news stories. It uses a betting scheme as an incentive.

I wish you could just block entire domains, regardless of search or subject.

> As a user I would also greatly appreciate the ability to toggle news sources on and off, and/or to weight them, and to easily share those filters.

I was literary thinking of that last week.

Major traditional news outlets are shown to be major news outlets online.

News at 11..

This study is just proving the Pareto principal and claiming its some big thing. Of course these outlets get most of the Google love, they're all linked to as sources by all the little news outlets, how is anyone surprised by this?

Spoiler alert: so does everyone else (see subscription numbers).

There are a few niche and highly specialized publications which I view (GQ, Men's Health, etc.) but for major news and in depth reporting I'm more likely to frequent WaPo, NYT, Vox, etc. There is a huge drop off in quality when viewing work from the mid-tier news providers.

It's true, they're the only ones actually putting people on payroll to send them out to areas, do interviews and original investigation. Everybody else is just reaggreggating others from other news outlets, social media and public documents with their own analysis. But even the big mainstream companies suck at local reporting for their own towns. For things that are not national federal news you have to really get it from other people.

I'm not positive but I feel like this is getting mistakenly lumped into the issue of censorship by big tech companies?

I could be being naive but this seems like an echo chamber problem that faces websites attempting to surface content to you based on what you like or your network likes. This doesn't appear to me to be same issues of political-based censorship of certain people's accounts that we have been hearing a lot about over the last year.

It's not difficult to understand:

Google shows users what other users like, and most of Google's current users favor a small number of major outlets.

I can appreciate that you may love RC Cola. But it shouldn't surprise you that most gas stations mostly stock Coke and Pepsi products.

If you think gas stations need to be more "fair" about which pop they sell, then I'd love to hear you explain how you think that should be done.

I think your logic is perfect. As long as you agree to treat Google as a publisher and not as a platform like it is currently being treated now.

Section 230 [1], which in theory was supposed to protect free speech, is actually doing the opposite and giving Google and other major media companies the power to censor free speech and decide what you can see or not.

I'm sure the gas station would be held liable if RC Cola made their customers sick and they kept selling it.

Google is not liable for anything they publish/display, but still have the power to choose what they publish/display.

You can't have your cake and eat it too.

[1] - https://www.eff.org/issues/cda230

> Google is not liable for anything they publish/display, but still have the power to choose what they publish/display.

Choosing how to respond to a query string is the very nature of what a search engine does. It's utterly impossible to have one without the other. The more charitable interpretation of your argument is that there should be more "neutrality" in the algorithm, but that's also difficult... Should they be neutral to whether or not people are trying to manipulate algorithms with dishonest SEO? Should they be neutral to what other users like? Neutral to who else links to the page? Who decides which neutrality prevails where they conflict? Who decides if a certain kind of neutrality is more important than user preferences or market demands?

This is a rabbit hole of mutually exclusive goals and inevitable trade-offs that requires complex judgment. Government regulation in this area would be extremely difficult and troublesome, not just for economic reasons but also for first amendment ones. Free market competition in this area is going to be limited due to economies of scale, so that isn't perfect either.

And if you think that imposing traditional publisher liability for search engines is likely to make search engines censor _less_, you might want to think a few steps further down the strategic consequences.

The only way to resolve this is to make the search algorithm public, so that we actually know how the results should be interpreted.

Are you in general violently opposed to Intellectual Property, or only in this case?

Only in this case. Search rules our entire economy and deserves a special status.

Maybe we should start with easy problems.

I'm not proposing that we have a "fair" algorithm. I think two simple actions would suffice:

1) Google (and other platforms) should stop removing content from people they do not agree with in their platforms.

2) They also should publicize why one result was chosen over the other. That would give users the necessary information to decide if Google is biased or not. Everyone would make decisions based on publicly available information.

That alone would make things way more transparent and level the playing field a bit.

Unless Google doesn't want that. If that's the case, them they are a publisher and not a platform.

Google News has no concept of 'agree with' it simply tries to find a news result based on what it thinks you want to see.

There are no politics baked in.

Google news will display what its users want to see.

Your issue is not with Google News. Your issue is with the users of Google news.

How do you know this?

> Google (and other platforms) should stop removing content from people they do not agree with in their platforms.

Citation needed.

> They also should publicize why one result was chosen over the other.

Seems like this would make it much easier for the SEO spam sites to manipulate.

Here is a list from wikipedia on twitter suspensions, not sure how up-to-date it is. https://en.wikipedia.org/wiki/Twitter_suspensions

I believe it's factual to say that significantly more "conservative-related" accounts are banned than liberal ones? Is that because "they don't agree with" the conservatives more than the liberals? I dunno.

What does Twitter have to do with Google?

@deelowe, sure.

- https://boingboing.net/2019/05/02/facebook-just-deplatformed...

- https://mashable.com/article/deplatforming-alex-jones-2018/

I can probably find a lot more, but I guess this is enough to make the point, right?

> Unless Google doesn't want that. If that's the case, them they are a publisher and not a platform.

Would you rather fight spammers and keep free-speech alive or give it up over to Google and other platforms?

I know where I stand.

> Would you rather fight spammers and keep free-speech alive or give it up over to Google and other platforms?

Spam is a form of censorship. Too much of it and it becomes a denial of service attack that makes publishing impossible.

Let's say you claim you have free speech, because you can go into any park, stand on a soapbox, and give a speech. Only, I have blanketed every major park with loud speakers that overwhelm what you say with noise, or GPT-2 generated fake speeches run through WaveNet.

I used to believe that 'the solution to bad speech is just more speech', but this was a naive pre-mass internet view of how speech and publishing works. In a word where robots can literally saturate every one of your media channels, spam up your email, your phone with robo calls, your text messages, the idea of these platform providers NOT filtering and censoring is itself a form of assault on free speech and public conversation in the town square.

I think now we are getting somewhere with the discussion. I like your points.

What you described is a real problem and I believe that’s the problem we have to solve.

I’m not a lawyer but I’m pretty sure that situations like the one you described where free speech is negated are already addressed on the law. So spammers would be treated by it.

In that case, platforms would work to identify the sources and work with the justice department to prevent specific actors from performing crimes such as spamming and DDOSing.

This would be fine, in my opinion, because now everybody would be working against known criminals with specific and objective criteria as to why they should be “deplatformed”.

However, what Google is doing today is removing or de-prioritizing content that diverge from its opinion without any recourse for the person affected because they are not treated as publishers and THAT is not okay.

Free speech has never been about having free unrestricted access to private platforms. The bandwidth of private platforms for media has always been limited, be it broadcast TV bandwidth, or the the editorial policy of news papers and their disproportionate audience sizes (sucking up all the oxygen and providing a huge megaphone). Fringe views rarely even got airings a few decades ago, and it wasn't until cable TV gave us 500 channels, did the lower scarcity of outlets bring out more diversity.

Alex Jones, for example, can run his own video hosting site if he wants. He's rich and could surely hire someone to build a site one on of the cloud providers, it's getting cheaper everyday. A big chunk of the HN audience could probably build it in a few weeks sprint on AWS on the cheap.

The whole point of the internet is that anyone can run a server on it. He is not entitled to publish from YouTube. And in fact, the argument that he has a right to be heard on there is only strengthening Google's monopoly and centralizing the internet by arguing almost the only way to host video, and exposing him to risk of deplatforming.

If you were a regular writer for say, the New York Times, or Readers Digest, or the Wall Street Journal, and had a huge audience, you had the same risk of sudden deprioritizing and deplatforming if the editor decided you were toxic. You don't have a right to free speech and free audience on private media platforms.

And let's be frank, what the tech companies are doing is responding to the public pressure, because the vast majority of people object to the content that's being deplatformed, and their advertisers object to advertising on platforms where most of the customers object to the content being associated with the product. A small percentage of angry political zealots on the extremes of both political wings are turning themselves into "victims".

The internet and web is the real public square here, the real printing press, NOT YouTube, Facebook, et al. And just like decades ago, if you found yourself unpopular and couldn't get your fringe magazine distributed on the shelves of major book stores, you'd self publish and self distribute to build your audience.

We need to get back to the way the internet used to be anyway. More people need to run their own websites.

I can't agree enough with you. Everything you say makes sense for a private company.

What also makes sense for a private company and, in this case, a publishing company is that they are liable for what they publish.

That means that people can take action in the justice system if they think a contract was broken or any sort of harm was made.

Unfortunately, that is not the case with these platforms. You can't take action against them because they are not categorized as publishers, but as platforms instead which do not make them liable for things that they would otherwise be.

Make Google, Facebook and other "platforms" into publishers and this problem is solved.

I know I will probably get downvoted by this comment, but I find it really interesting that in 2 hours I got 4 downvotes just in this comment and no argument to explain it.

I know downvoting is a tool that is available to us, but it really strikes me when someone is willing to cast a "negative" vote anonymously and most likely with no fundamentals attached to it instead of engaging in a respectful argument where all opinions are presented.

The all-too-sad answer to this is that HN, like any other forum, has a particular demographic, and the demographic here (predominantly well-to-do urbanites) has a certain political lean. People, generally speaking, can not divorce their political beliefs from their actions and act purely objectively, which, for example, is why we have laws and company rules about even creating an appearance of impropriety.

You can ask, beg, threaten, whatever all you want to for people to be objective and measured and to follow the stated principles, but people will simply vote you into invisibility simply because they have a knee-jerk, primate-brain reaction to the ideas being posted. This is not a problem that can be solved, here, or anywhere else.

It's a human problem, not a tech problem.

On that note, it's best to never complain about being downvoted. It only attracts more downvotes.

That's the sad reality I see as well.

If you look into my comments, you will see that most of them are downvoted simply because I differ from the main opinion of the demographics.

While it makes me a little sad to see an unjustified downvote I think that in the great scheme of things it doesn't really matter.

I'll keep voicing my opinion, for as long as I can even if that means I'm going to be downvoted. Ah, that includes "complaining" about downvotes as well just to fulfill the profecy ehehehe.

For what it's worth, I get downvotes for political posts too, and I'm the sort of person who thinks Nazis should thank people for the punches they get.

The notion that you're somehow victim of a political bias is not well supported. You're essentially shouting, "Debate me you cowards" and that's why you're going to get downvoted more.

As a courtesy to you, I will explain that it's extremely likely you're being downloaded because your argument is essentially (although perhaps unwittingly? who can say!) an argument to censor all tech companies as publishers. This is an argument to use the government to compel conservative speech on private industry, while at the same time not once asking questions like, "Should we then force 20 minutes of Marxist/Leninist or Anarchist (or even just a scientist calmly explaining the reality of climate change) content onto Alex Jones's or Rush Limbaugh's show?

Indeed, throughout the course of this argument you're simultaneously arguing that Alex Jones should have unlimited speech but if Google does the same thing (highlighting what they consider to be reputable 3rd parties) they should be treated as a "publisher" and subject to fairness doctrine and stripped of safe harbor.

That's an awful idea. But what's more, it's patently obvious that you're arguing a double standard to anyone who is not eagerly to use the State's authority as a literal-and-figurative club even as the State itself skirts freedom of speech laws. And as such, it's likely folks are reading your argument, assuming it's offered disingenuously, and hitting downvote.

Thank you!

Here’s my point of view. The purpose of discussion, I believe, is to convince people of a given point of view.

That is done through rational and logical debate.

So when you say that I’m inviting people for a debate I have to agree. After all, isn’t that the purpose of a public forum?

If we can’t go deep and have meaningful discussions what’s the point?

Just do some virtue signaling and bash whoever is the person we are bashing this time?

When I invite argumentation I’m honestly looking for people to present arguments that can convince me. I’m open to be convinced, just not by empty argumentation and fallacies. What I’m getting, instead, is downvoting which reinforce the bias you said I have. Right?

Also, you mention that what I’m proposing it’s an awful idea with no counter argumentation. Is this how we suppose to convince people now? Especially the ones on the other side of the isle?

I used to do that too, but then, through argumentation I was thought that it is not how it should be done and in the journey I was able to help some other people to see that too and I’m proud of it.

I think that as long as there is respect we all can have a discussion and learn from each other so for whoever reads this and see any of my comments in future threads, please know that all my positions and arguments are made in good faith and only have the goal to promote intellectual challenging.

Lastly, I have not defended Alex Jones in any comment I made. To be honest, I read and hear a lot about him but I have no idea what his thoughts are. I’m just pointing out the fact that people are being “deplatformized” arbitrarily and I don’t agree with that.

> That is done through rational and logical debate... So when you say that I’m inviting people for a debate I have to agree. After all, isn’t that the purpose of a public forum?

This is what you'd like from this forum. Actually, political debates are not what I'm here for even though I participate sometimes.

> If we can’t go deep and have meaningful discussions what’s the point?

You're not really entitled to a sympathetic audience here. Nor is it necessarily obvious that you're debating.

> Just do some virtue signaling and bash whoever is the person we are bashing this time?

If you're mad that humans tend to influence one another, why are you here? Does it not infuriate you?

And what is virtue signaling if not this very post you've written, trying to appeal to a Hitchens-ian notion that only through debate of your style and terms can we reach truth. You're constantly reminding people of the importance of debating you foe the sake of the public forum and talking about "profecy" that no coward dares to debate you. Is that not itself a kind of virtue signaling?

> I’m just pointing out the fact that people are being “deplatformized” arbitrarily and I don’t agree with that.

The guy literally went to court, swore and oath to tell the truth, and told the court his entire show was an act to sell soy supplements. He's not news, he's not a content creator, he's a weird surrealist advertiser of postmodernist medical products. Advertisers don't get free rides, they have to pay.

>deplatforming Alex Jones

Just as a datapoint, I just googled "Alex Jones"[0] about 5 minutes after your comment and his Infowars website is prominently shown on page 1 as the 3rd result[1].

I don't have a Facebook account but it doesn't look like Google Search in particular has deplatformed Alex Jones.

(To the downvoters, this thread's topic is the Columbia Journalism Review story and it's about referrals from Google Search and not Youtube. The comment by liara_k is also specifically about the Google search engine and that's what tucaz was replying to. If my screenshot of Google search engine results does not contribute to the discussion, please explain why.)

[0] https://www.google.com/search?q=alex+jones

[1] https://imgur.com/a/KDVBHcJ

This is a false dichotomy. We're not actually required to make this choice. Literally nothing forces you or anyone to use Google. The reason folks flock to their products despite costs and the occasional awkward depreciation is because they execute pretty well and they have one of the best security track records in the industry.

Attempting to co-opt that for "free speech" by injecting stories onto their pages that are more agreeable to a "balanced" interpretation is not a free speech approach.

What's more, many of the headline proposals in this article text (e.g., Kaepernick and Tax Reform) are tough to give the rightward view on without facing direct misinformation, because the politicians on the right have engaged very directly in misinformation about taxes and deliberately tried to intimidate free speech in atheletes (above and beyond what influence the average American has). Saying that any news outlet should make a special exemption for calling these out to be "balanced" is to propose that telling the truth is a liberal approach, and I find that idea pretty repugnant on it's face for a variety of reasons (not the least of which being that it gives way too much credit to the center left of America).

As for Alex Jones, I don't know how many brazen and horrifying lies you need to literally go to court and plead that you're a comedy show for before you lose the right to use the veil of serving the public interest. Is a transparent desire to lie with the intent to deceive not a DQ for being a "journalist" now? At what point DO we say, "You do you, Alex, but we're not a part of this anymore?" And if Alex gets that privilege, are you going to fall over yourself to similarly defend the Democratic Socialists of America and antifa actions? I suspect for most people that is a "No."

The two links you provided were cases where Google acts as the publisher (e.g. youtube). This doesn't apply to the premise of the article, which was specific to search and curation of news outlets.

> Would you rather fight spammers and keep free-speech alive or give it up over to Google and other platforms?

You appear to be mixing search and youtube. If I search for Alex Jones, the second hit is infowars. First is wikipedia.

I'd love to know what knobs are turned at those companies to alter results, it's something of a trade secret I guess.

Just to see for myself, I picked a search that should churn up a guy that they might find irritating and did 10 seconds of anecdotal research.

Search was 'vox day blog'.

Google, Yandex, Startpage return his blog as the first entry (perhaps after an ad). Goduckgo it was the sixth entry. Bing, it never showed up. I got bored after the first dozen pages.

What exactly does this prove? Not being a web/seach engine guy I don't know.

What I'd like to find is a search engine for which I could build a great big list of curated sites with zero results from others. I tend to look up the same things a lot and after a year or two of additions it might be a really nice thing to have.

Neither source you link even mentions google. Yes, please find a lot more. That was not enough to make the point.

Those don't appear to be examples of Google removing content.

Thus if you have more examples, I would be interested.

@deelowe, By using the word "mixing" up you are implying that I have confused two separate things, which I didn't.

They are not two separate things. Google enjoys the status of a platform in all of its products.

The links I provided talk about YouTube and other platforms and demonstrate how they act as a publisher and not as a platform and the submitted article demonstrates how search results are skewed towards a direction.

Can you please point where my argument is flawed?

Your argument is flawed because there was no evidence given of search results being censored. This is the premise of the article; some sort of biased, preferable treatment being given to certain news sites. Throwing youtube and facebook into the mix, which are curated platforms (AKA publishers) and one of which is clearly NOT Google, is disingenuous. Those platforms are not expected to be unbiased in their results.

Even if you give that Google is a platform, as Tyler Cowen artfully put it this week:

> You might be worried that, because of deplatforming, the remaining sites and writers and YouTube posters have to “walk the line” more than ideally would be the case. That to me is a genuine concern, but still let’s be comparative. Did you ever try to crack the New York publishing scene in the 1990s, or submit an Op-Ed to the New York Times before the internet was “a thing”? Now that was deplatforming, and most of it was due to the size of the slush pile rather than to evil intentions, though undoubtedly there was bias in both settings.

What do you think was happening before?


This is a very weak argument. The internet has created a new environment for speech to occur, and this new environment needs to be treated fairly. Just because things were worse before does not justify the misuse of the tools we currently have to continue letting things be "worse".

Before humans figured out agriculture, people regularly starved. Would you walk up to someone starving to death today, shrug at them, and say "we used to starve all the time"? Your position is inheritly anti-progress.

This is the list of the world's top 50 news sites.


Hardly any conservative sites make the cut - Fox News is 10th, but the Drudge Report is 23rd and the NY Post is 26th.

Meanwhile, CNN is third, trailing only Google News and Reddit. NYTimes is 4th, WaPo 11th, Bloomberg 17th and USAToday. Meanwhile, 222,000+ sites link to the NYT. Fox News has just 52,000.

Google’s algorithm lets the Internet vote on which news sources it trusts the most. As it turns out, very few right wing sites make the cut.

Perhaps not the biggest surprise, as right now, according to Breitbart, the biggest story in the entire world right now isn't Zuck, isn't Trade Wars with China, it isn't even Julian Assange, it's that Jussie Smollett's 'Empire' just got cancelled.

Your argument is essentially "the internet is fair". I was arguing against my parent, who basically said "the internet does not have to be fair". They are different arguments.

That said, the obvious counter-point to your point is that conservative sources are ranked lower because they are intentionally deprioritized by platforms like Google and Alexa. I'm not sure that I believe that, but the current state of affairs is not always self justifying.

Really cancelled? That's news to many like myself who have been following the case.

In fairness the China back/forth is static quo for the Trump era, the fact the Sweden reopening is just nonsense fake politic showboating. Empire being cancelled is really interesting and speaks to something deeper / culturally important. The: you must believe everything that comes out of my mouth because I identify as these groups who have historically been oppressed narrative has changed. People who normally would be afraid to question his story have started to. The change happened after the Saturday night live bit a few weeks ago. His responses put in that context made his side so absurd his supporters couldn't defend him.

> The: you must believe everything that comes out of my mouth because I identify as these groups who have historically been oppressed narrative has changed.

Right, but clearly not so much that Fox et al aren't running under 25% of more respected outlets numbers. If it had have changed that much, the vast majority of people wouldn't be flocking towards the latest micro update in a macro story that by and large hadn't changed in weeks over the latest update on a culture wars proxy story.

Those are overall numbers but it may increase engagement within the 25% which makes the comparison between apples and oranges. If x engages me more more products will be sold.

You're thinking about Section 230 wrong. Censorship is a government only thing. But even if we accept your definition of censorship, in google's case, there is no censorship taking place. Because google always lets every user know about every hit it found for their search at the very top of the first page of results. So if google knows about 1,690,000,000 instances of, say, "chocolate cake", it tells you that up front. Crucially, given enough time, google will always present to every user all 1,690,000,000 instances of which it is aware.

You are not required to present all of the information on your platform at once in order to be considered a platform. Indeed, it is generally accepted that platforms, by their very nature, are not able to do so.

I don't want to argue semantics, but even if we do, I would state that you are wrong. [1]

Censorship can come from anyone and anywhere. Especially when a private company like Google has more power that many governments in the world. Google could even be considered an un-elected government depending on how you look at it.

What they are doing by de-platforming people is censorship.

They offer no clue as to how their algorithm works or why some people get up on the list while others don't.

While they could theoretically show all 1,690,000,000 results, we know for a fact that just the first or second page at tops is what matters. So, in all practicality, the other results do not exist.

If the other results do not exist, then they are a publisher and should be treated as such since they pick the winners and losers.

If there's nothing wrong with what they are doing, what's the problem of calling then a publisher? Why not embrace it and take full responsibility?

[1] - https://en.wikipedia.org/wiki/Censorship

You seem to be suggesting that by curating the content on their platform, Google is losing (or should be losing) their section 230 protections. This is simply wrong.

If I have a blog, and I ban someone from commenting, I don't lose my 230 protections. Even though I've 'de-platformed' someone from my blog. Google is no different than a blog, just scaled up.

But the guy is even more wrong than that.

He is saying that any hit not on the first two pages of google's search results, doesn't exist, and is therefore being deplatformed?

That's a ludicrous position. Those hits do very much exist, and google will display to you each result in turn hoping you find something to click on. If his argument is taken at face value, the vast majority of hits, on every search term, for every user, is being deplatformed. That's just not the case. You can't really say you're being deplatformed, because you're not on the first page.

@bilbo0s I’m sorry. Let me be more specific.

When I refer to deplatforming I’m talking about users and opinions being banned from the plaftorms (YouTube, Twitter, FB, etc).

When I’m referring to search engine results what I’m saying is that Google is promoting a few selected sites over others based on a undisclosed criteria. The undisclosed criteria here is what makes a world of difference.

By not making it clear they are picking winners and that should not happen.

On my blog I'm free to delete any comment I like, and I'm not beholden to the government to explain why I did it. That's simply the Constitution at work. How big, in your opinion, does a publisher need to be in order for their First Amendment protections against forced speech to be stripped away?

Right, because your blog is tiny and meaningless. When you control the eyeballs of a country it is a different story.

Can you direct me to the clause in the Constitution that limits free speech rights if you attract a big enough audience?

Anti monopoly laws, would be one. You can say whatever you want, you'll just have your platform split in smaller pieces to avoid harming the average individual.

If you disclose the criteria, people game it. Any ‘objective’ scheme you come up with will be overrun by scammers in weeks without constant updates from google.

So the solution is to give all the power to one company and hope they do well by us?

Just because a problem is hard to solve it doesn’t mean that an authoritarian solution is acceptable.

Perhaps we should come up with a system where people are free to use whatever search engine they like, and the government can't tell you what search engine to use, and the government can't tell the search engines what to publish. It would be so much better than the current system where we're legally required to use Google.

Because Google, along with Facebook, control the advertising market. They integrate well together, but it presents a long term problem.

You forgot Amazon, but it sounds like we've moved the goalposts from free speech to fairness for advertisers now. That's an antitrust problem, not a free speech problem.

That's what I'm suggesting @mullingitover.

It seems like you don't agree. Maybe you could make the case about the reason you don't agree and we can go from there.

"Crucially, given enough time, google will always present to every user all 1,690,000,000 instances of which it is aware."

Just on a technical basis, no, it won't: https://www.google.com/search?q=testing&start=1090: "Sorry, Google does not serve more than 1000 results for any query. (You asked for results starting from 1090.)"

Arguments based on that are therefore void.

> Censorship is a government only thing

To nitpick: no it's not. Government censorship is worse than other forms because the state is an entrenched, nearly irreplaceable power structure with lots of inertia, but that doesn't mean non-state censorship is perfectly fine.

Any platform that curates content is censoring. Sometimes that's fine (illegal content), and sometimes it's just not acceptable.

I'll add that its also worse because of government monopoly on violence.

Though corporate lobbying can co-opt government violence, which creates many dangerous scenarios.

"I see you edited our software in violation of the DMCA, the police will arrive shortly"

Are you saying that all websites protected by section 230 should be required to be neutral, and not be allowed to filter out content they don't like? Or just Google because they have so much power?

Google is supposed to be nothing. They are here to make money, and they will follow whatever rules that let them do that as long as we let them.

They are not a public service.

That's true. The problem is the monopoly position they have and the massive adoption has put them in a public role.

Everyone needs to get from point A to point B and roads are a key way that we accomplish that.

In a similar vein, everyone needs to search for information and search engines are a key way that we accomplish that.

With Google, it's almost like one company owns 90% of the roads. On multiple continents.

It is sort of like a utility company, except way beyond that.

I personally think that platforms with this scope should be built on public, open, decentralized protocols and networks. Something like YaCy might be the right direction.

Then maybe we could have companies built on top of that platform to add value. But the core functioning of the platform would be auditable.

What free speech does Google explicitly censor?

One of my big fears is the fragmentation of society and a conservative vs. a liberal internet. The echo chamber could be ratcheted up without exposure to a different viewpoint. I understand catering to customers, but I don’t see the cola example as analogous to this. With the influence that google has on search, I think they have a duty to present a balance of opinion to end users and to be unbiased.

> One of my big fears is the fragmentation of society and a conservative vs. a liberal internet

That's already true - consider a site like Reddit and the perceptions of /r/politics compared to /r/the_donald or the insulated bubble you can easily find yourself in on YouTube if you follow the recommended videos rabbit hole. The bubbles can, and already do, exist on the same platform.

1 - Too late to stop liberals and conservatives from enjoying their echo chambers. (In fact, the entire implication of the First Amendment is that different people should enjoy their echo chambers if they choose to do so.)

2 - Claiming that google influences people to search for "FOX", or "CNN" is as ludicrous as claiming that google is influencing people into searching for "how to kill blacks", or "how to make bombs". My own suspicion is that people decide what they're going to search for long before they come to google. It's certainly not google influencing people to type in certain terms on its famously empty homepage.

I don't know if Google news still has the feature, but when I used it, there was an option to raise and lower weights to certain news outlets. I found it worked less and less over time to the point where the outlets I specifically filtered on didn't even show up.

I don't believe this is simply an effect of giving the crowds what they want.

The controls have gotten less granular, and I don't see a way to add new sites to the filters without an article showing up in your results first. But they do still seem to work on personalized news such as the "For You" section.

- Blocked sources: https://news.google.com/settings/feedback

- Favorite sources: https://news.google.com/my/library

What I have noticed is that the filters don't seem to apply if you click on "Full Coverage" of a topic anymore (via the little icon with the colorful rectangles), perhaps in response to those echo chamber complaints.

If gas stations cared about the customer they would offer totally refreshing natural tap water for free, offering the sugar-drink products as a secondary option. But they don't do that, do they?

Google and the rest are pretty much the same.

Returning to the soft drinks analogy, why don't they sell RC Cola?

They buy cola by the crate and there is a better margin selling a case of the presumably more expensive crate of Coca Cola.

If RC Cola retails at 50c and Coke retails at $1 then they are not going to stock RC Cola. The supermarket are in a different situation, they are selling multi-packs and they have their own brand stuff, the RC Cola and the real Coke product for their customers, Coke having the best shelf.

If the gas station had demand from customers for RC Cola then they could stock it. However, 50c of revenue would be walking out the door with every sale of a cola-style beverage. Even if RC Cola have better margins when buying at trade that 50c difference is a big one.

The supermarket can increase footfall by offering RC Cola at a competitive price, people coming in for it might buy other stuff where the real money is made.

With the gas station a reduced price RC Cola offering is not going to increase footfall. Plus gas only has slim margins to it there is no elasticity there. The beverage and candy options are therefore really important for profit. Hence only certain products can be sold, the business cannot afford to stock anything that gets sub-par revenue or put that tap there for people to obtain limitless quantities of wholesome tap water.

Google shows users what other users like, and most of Google's current users favor a small number of major outlets...

If you think gas stations need to be more "fair" about which pop they sell

Not a good analogy. With social media and the desire to exert control over what's trending, it's more like, in the earlier days of 5 Hour Energy, when sales were growing rapidly, gas stations and convenience stores refused to try it out for reasons beyond sales. In reality, they're fairly enthusiastic about trying such things out. In reality, the space next to the register is fairly market driven.

“Getting it next to the register isn’t hard. Keeping it there? Very hard. Anything that sells, the stores will try. To own that space is really hard.”


Sometimes I see the entire pantheon of legacy media news channels on YouTube presented to me, when it should be obvious that I never click on those links. Funny that.

Isn’t selection like this always self reinforcing?

It’s like when Facebook or Instagram only shows me certain posts, and I react to those posts because they’re all I am presented with. Then, their logic is that I just love posts like that, and it’s all they should show in the future.

You would use a little randomness to counteract that, I suppose. Like how "heat" is used in simulated annealing.

Exactly. If some users perceive Google News as somewhat biased, it's likely to be self-reinforcing-- by nature of how Google's algorithms evolved over their corporate history.

That is a logical hypothetical, but we don't know if it's accurate.

This provides data about what the gas station is stocking, but we do not have data about what Google's current users favor. It may well be that the users do prefer RC Cola.

And setting aside user preferences, the gas station doesn't stock what people want to buy. They stock what will make the most profit. The users may want RC Cola, but by refusing to stock it they may be able to force users into buying their higher-profit-margin Coke.

This is completely off topic, but I do love RC Cola. To the point that I buy Keurig Dr Pepper stock (formerly Dr Pepper Snapple co). In many places Dr Pepper and their other sodas are bottled by the regional cocacola bottling company. I don't know if it's fact, but I always assumed the reason it's so hard to find is because Coke refused to bottle RC. Kind of like Apple refusing to let an app in the App store that competes with something they make.

Kind of like Apple refusing to let an app in the App store that competes with something they make

Well seeing that for every product Apple sells or give away, there are competing products in the App Store, there is an existence proof that this isn’t true....

There are no other browser engines in the app store.

No but there is WhatsApp, Fantastical, VSCO, Camera+, Halide, Google Photos, Evernote, Bear Notes, Google Maps, Spotify...

That's for security reasons. Apple doesn't want outdated web engines becoming vulnerabilities.

How does a browser “compete” with a non revenue generating Safari?

Because you can't use any other browser in iOS. It's not AppStore rules. https://stackoverflow.com/questions/11259152/chrome-ios-is-i...

Siri, Apple Pay, the Lock screen, the Home Screen, AirDrop, AppleTV, the Phone app, the SMS app, the App store itself.

Siri - Alexa App is on the App Store as well as Google assistant

Apple Pay - there are plenty of QR code based pay apps on the store

AirDrop - any one can write a share sheet extension to duplicate the functionality.

Phone app - there has been a voip API that let third party apps integrate with the native phone screen forever. Hopefully you don’t consider third party apps being able to intercept phone calls a “feature”? How many security issues have arisen because of that on Android?

AppleTV - you mean a remote app? Streaming from your phone - the Roku app lets you stream video from your phone and Apple just announced third party AirPlay integration.

SMS App- Google Voice. If you mean allowing third parties to intercept your native SMS phone messages, that has been one of the many security nightmares on Android.

Also “as far as the App Store itself”, the original poster said that there are no products “in the App Store”. Are you expecting Apple to sell an App Store within the App Store?

You make a compelling argument that Google is biased against new entrants into market(s).

How are people supposed to know about these products/services if Google is showing what people already like? i.e. incumbents. They are forced to buy ads!

This is just one example why it's so important to show "neutral" results to everyone. The good news is that politicians like Warren understand these issues. Regulation and possibly break-ups are coming.

Relying on corporations to be the arbiter of fairness is dangerous. I certainly welcome the new progressive agenda of regulation to keep markets free and fair.

The disingenuous bit here, is that they are not technically censoring, in the traditional sense. Instead, what Google/YouTube and others are doing are manipulating discovery/virality for their own ends.

As shown in Manufacturing Consent governments and powerful organizations can get the effect of censorship by manipulating visibility. A social climate where virality essentially belongs to just one political faction is dangerous. It would be the 1950's equivalent of only Republicans getting to appear on TV and radio.

How do you define “neutral” results?

We could start with what they had before they started biasing results based upon user activity.

i.e. What search engines like DuckDuckGo do.

People make web pages. Web pages link to each other. Page Rank finds out how important a page is, based on links to it from other important pages.

That idea is probably at the core of all modern Search Engines (including DDG).

Thus, all modern Search Engines (including DDG) are almost certainly based upon user activity. That being the users who build web pages and links to them.

Correct. I guess I wasn't clear. I'm talking about the personalizing of search results based upon search history, search terms, personally and in aggregate. Not based upon the structure of the web itself.

How is that less biased, as opposed to less personal? Or do you mean that companies shouldn’t be biased to your preferences as a customer?

Search engines could bias towards our preferences as consumers if they weren't a monopoly and didn't sell/place ads in the search results.

That's the problem with Google and I think we're too far down this path for them to fix it. We've been watching this slow walk to where they are for a long time and I think people have just become to used to it to notice how insidious the situation has become.

Is the company biased towards their interest or towards the customer's?

If Google was simply a provider of information then the answer would be that those are one and the same. But they obviously are not. One conflict that already provides an irreconcilable conflict of interest is the fact that they are primarily an advertising company - and the world's largest one at that. There are countless other issues that could also be brought up to emphasize conflicts of interest, but that's not really necessary. That their primary business is selling advertisements creates too large of a conflict of interest to ever expect "customized" search results to be anything but manipulative.

In the past year Fox News has been getting much more exposure than they ever did before. They stand out because their headlines are either the direct opposite of the other services in the same topic group or some form of irrelevant rage baiting for their base. I don't want them there and didn't ask for them.

How do you feel about conservatives ignoring news sources because they’ve been labeled “left-leaning”?

The problem is that one "gas station" is owned by iHeartMedia/ClearChannel. And if you drive onto their lot worth an RC Cola in your car, you can't get gas.

In fact, if they find out that you've ever endorsed RC Cola anywhere ever, you can't get gas.

Users often favor what is advertised to them frequently.

Google presenting particular results influences users.

Pretending that Google is merely passively reflecting user preferences rather than actively shaping them makes no sense.

Consumers favor sodas that are advertised to them frequently, too.

We are where we are, because most users have gotten us there.

> Pretending that Google is merely passively reflecting user preferences rather than actively shaping them makes no sense.

It has content and advertising just like a gas station does.

It's a market. Literally, it's a market.

Yup, a market where Google makes editorial decisions about what ads and results they show.

Isn’t that just like Safeway or the NYT? Except these companies probably exercise way more control than Google, and their customers likely enjoy the curation.

It’s just like that,

Except that the NYT and Safeway don’t pretend that their product isn’t the result of curation.

There is nothing ‘organic’ about organic search results.

I am sure both of those companies do exercise more control than Google, but they also do not have over 90% market share.

It’s not literally a market in any way.

But I agree that there are comparisons with a Gas station.

A gas station is there for one reason - to sell you what the owners find profitable. What is sold and what is displayed are chosen for that reason only.

Google is exactly like this.

What it is not, is a responsible organizer of the world’s information or a search service, or in any way responsible to the interests of searchers unless they align with the interests of advertisers.

"Consumers favor sodas that are advertised to them frequently, too."

A) Information is not Soda, it's not an arbitrary choice

B) Soda is also a physical distribution business - Coke and Pepsi dominate physical distribution, shelf space etc. - these problems don't really exist in news, or rather, in totally different ways.

So - 1) they are different kinds of 'products' in terms of their civil relation (i.e. we don't care about gossip information), and 2) the businesses are actually different.

> and most of Google's current users favor a small number of major outlets.

With what level of certainty do we actually know this? Sounds like an assumption to me.

How can this be true if their top source, by a ratio of almost 2 to 1, is the cable news network with the lowest viewership? CNN doesn't even have a show that cracks the top 25 cable news shows[0].

[0] https://www.adweek.com/tvnewser/the-top-cable-news-shows-of-...

There are people who use cable news, and consume cable news through cable.

There are people who like cable news web sites, and search for them on Google.

Let's say for instance that 2/3 of the people who search for news on Google are people who like cable news web sites. Then, by a ratio of almost 2 to 1, cable news web sites will be favored.

Rephrased: Perhaps people are actively consuming CNN through the Google searches on the web, and people are passively consuming Fox through cable.

Not that it's strong evidence, but consider: "According to Nielsen ratings, the median age of Fox’s audience was 66 in 2016." [1] [2]



That would explain CNN being at the top, but wouldn't explain why NYT and WP outrank Fox in the audit, by a 2 to 1 ratio, when Fox outranks them on Alexa.

Some people open Fox News directly, and then Alexa thinks they're awesome.

Some people come to Google, and go past the Fox News result, and seem happy with the CNN result, or NYT, or WP, so Google thinks they're awesome.

And if Fox News viewers think Google is biased, and then avoid Google, then Google will become more biased. It's a vicious cycle, or virtuous cycle, depending on how you look at it.

I think it is partly because NYT and WP cover much larger ground and produce many more stories than Fox News. So a large number of visits to Fox News itself for stories belonging to small set of topics doesn't help Fox News with Google search results. Also, maybe reputation matters. Due to many reasons (and this is the elephant in the room I guess), Fox tends to be less objective than WP and NYT. Fox News also had to redact many fabricated stories in the past.

> Fox tends to be less objective than WP and NYT

I get where you're coming from, but if you're not a card-carrying progressive then this statement feels egregiously false. The Washington Post was at the forefront of the Covington debacle. NYT pushes editorials about how mixed race dating is bad (from a "woke" perspective), among other crazy, far-left ideology.

What gets me about this whole subject is that everyone I know who watches Fox News knows that they're watching partisan spin, even the elderly ones. It's kind of a theater of overblown, right wing outrage that they're willing participants in. Most people I know who watch CNN think that it's objective, which couldn't be further from the truth.

Because television viewership != online readership.

New flash: people aren't getting their news from TV as much as they used to. CNN's audience being mostly online would be a reflection of that.

Then why does Fox News outrank The New York Times and The Washington Post on Alexa, but both are represented at a ratio of 2 to 1 to FN in the audit?

Maybe their website has more users than their cable network has ratings.

The problem is that gas station is owned by Google and in theory the faceless algorithm that runs everything says all cans are the same but in reality the RC Cola can criticized Sergey a few years ago and keeps making jokes that reveal some uncomfortable lies and you now have to go in the bathroom and climb a ladder to get the RC Cola.

Hackers drink Jolt cola ...

You can have my Shasta when you pry it out of my cold, fat, fingers.

I guess it would be an open and shut case if Silicon Valley, Google included, wasn't either actively censoring conservatives or if "algorithm mistakes" always happened to apply to them. I'm not a conservative - but I'm not so short sighted to see that censorship could turn around on me quickly.

... Consider this from yesterday, where one of the foremost experts on the field of transsexualism, a major contributor to the DSM-V was suspended from twitter [0] for explaining transsexualism.

[0] https://pjmedia.com/trending/expert-psychologist-blocked-on-...

If "the algorithm" is ready to block anyone for anything that could be offensive to one group but actively allows threats, doxxing, and attacks on another - you aren't talking about RC Cola vs Coke.

It's not at all unreasonable to say that Google and SV as a whole are clearly biased. We don't need more ways to make a bubble, and we certainly don't need Google deciding the narrative themselves. I think the analogy to physical sodas don't really hold up to digital delivery of news. Sodas range has a phsyical limitation, digital news does not, if sodas were a digital resource you would sometimes get to try RC Cola.

Try and consider how fast it could turn around on you. Cool, today you agree that coke and pepsi are better than RC Cola. What happens when RC Cola is politically tied to Google's fate and they push that instead?

> Google included, wasn't either actively censoring conservatives or if "algorithm mistakes" always happened to apply to them

I disagree with your assessment about "always". Frequently Google results are very anti-liberal as well.



I can find more examples if you like. YouTube suggests awful right-wing content ALL THE TIME.

You claim falls apart that, joebiden.info seems to have better SEO than the real site. Just because you or I disagree with it, doesn't make it Google pushing an anti-left narrative, it could just be organic.

The other article from 2017 doesn't give me the fake MLK result, so does that mean Google DID adjust their search results?

Should Google be manually currating results? Sometimes, but I think they've proven they don't draw the line exactly straight.

You're stating your opinion in an unfalsifiable way: mistakes that follow your narrative are a conspiracy, even if later fixed. Mistakes that don't are glitches, yet to be corrected, even if they never are.

What would convince you that you were wrong?

You might be presuming too much.

Look at the examples given. One is Google doing nothing to "fix a problem" (http://www.joebiden.info) and the other demanding Google fix an issue which they seem to have done - as being anti-left. Google being agnostic and not protecting Joe Biden is not them being Anti-Left.

I will not be bullied into thinking that Google doing nothing is them being anti-left. I disagree with many of the conservatives Google that have been deplatformed, but I recognize how quickly that pendulum could swing.

A few months ago I would have dismissed what you are saying, but now I could have written it myself. It's getting easier than ever to test the bias for yourself.

I actively wanted to read some of the extreme right wing stuff. (I believe strongly that we do everyone a disservice by silencing people. It harms the rest of us by isolated us from anything not "mainstream" and it further pushes the silenced into radicalism and vindicates/bolsters their argument that they are mistreated). I literally could not find it on Google, but just moving over to Duck Duck Go I found what I was looking for right away.

To everyone down-voting the parent, try it yourself. Think about some polarizing political topic and plug it into Google and then do the exact same search in Duck Duck Go (and maybe Bing, I haven't tried that but I will). You'll be shocked at the difference.

> (I believe strongly that we do everyone a disservice by silencing people. It harms the rest of us by isolated us from anything not "mainstream" and it further pushes the silenced into radicalism and vindicates/bolsters their argument that they are mistreated).

I absolutely agree that being "silenced" and feeling "persecuted" only serves to make extreme views more extreme. But I'm also not sure the bias is only directed one way as many of the far-right pundits would have people believe.

I also use DDG and avoid all things google as much as I can but the other day I searched "atlas" on YouTube (looking for the Greek mythological figure) and stumbled across a clip from the movie "Atlas Shrugged." I had read the book in high school and I know it's a favorite among conservatives so I figured it couldn't hurt to watch so I did, without thinking much of it. When I went back to my YouTube homepage later the recommend videos were very different from any content I would ever seek out: Jordan Peterson, Ben Shapiro, the Ayn Rand Institute, and even one called "Why Immigration is Bad for Britain." From watching one movie clip.

I think Google's algorithms, for better or for worse, can very quickly discern what content a user "wants" and filters out the rest. I'm the last person to have anything nice to say about Google but I do genuinely believe it's more a case of algorithms enforcing the unconscious bias/preferences of our brains than anyone at Google maliciously "silencing" certain points of view.

Thanks for the story, that is indeed interesting. I've noticed exactly the same thing on YouTube actually. I watched a Dennis Prager video to see why he was being threatened with banning. The video itself seemed fine, but afterward all my recommended videos are "person x gets DESTROYED by Ben Shapiro". I have never clicked on one afaik.

I do wonder if these are different problems tho. One could make a good case that they aren't, but I tend to think of search results as more neutral whereas video recommendations would be highly personalized. If I use the YouTube search box I'd expect similar results to everyone. But my expectations are based on nothing except my personal subjective sense of the way things ought to be.

> ... Consider this from yesterday, where one of the foremost experts on the field of transsexualism, a major contributor to the DSM-V was suspended from twitter [0] for explaining transsexualism.

Which was revoked as soon as a human got a look.

This happens all the time on Twitter. Here's a liberal Twitter user who got suspended for saying "you really wanna die on this hill huh", which the algorithm interpreted as a death threat.


The current AI state-of-the-art is extremely stupid, and once you get past that the first level of human review isn't much better - it's unrelentingly boring minimum wage work. Confirmation bias turns this random capriciousness into conspiracies.

> This happens all the time on Twitter.

What also happens all the time is that accounts from conservatives get banned, while people like Louis Farrakhan can be openly and virulently anti-Jew.[1] Bruce Carroll gets banned for referring to Chelsea Manning as Bradley Manning. Apparently that's more terrible than comparing an entire race of people to termites as Farrakhan did.

[1] https://nypost.com/2018/10/18/twitter-has-a-huge-louis-farra...

Many would buy rc cola or water companies not owned by Coke. Coke/Pepsi payoff/threaten to keep shelf space away from anyone else.

Drinking Coke/Pepsi is probably worse than smoking short/long term.

>Google shows users what other users like, and most of Google's current users favor a small number of major outlets.

I would say that's partly true. The other part comes from all the flack that Google receives when they include niche or non-mainstream sources (which admittedly are more sloppy or more biased in their reporting). And it is a valid question whether Breitbart articles should be displayed alongside results from CNN or NYTimes or WaPo for generic news queries.

It costs Google almost nothing to carry another electronic news outlet, unlike with physical goods (they probably still cache all of them anyways). Very bad analogy. As an example, the cost difference from picking the best result from 1000 sites or 10 sites is probably not very great.

It just tells me that Google controls what news that you are able to see and that they are not very objective.

edit: of course I am not talking about showing bad results on your prime screen real estate... just choosing the best results from more sources (which is what I thought this article was about).

Yes it does: it costs space on the page and a lower clickthrough rate, since people aren't going to click links from sources they don't recognize as often.

Should Google start preferring some CS undergrad's uncited poorly-formatted blog instead of stackoverflow as well?

a) " a lower clickthrough rate" - the click through rate to news is not the only concern of Google.

b) It's not an argument to suggest that Google should be carrying some random student news paper. As if Google can't tell the difference between 'The Burlington Vermont Times' and random noise.

I am not sure I can tell the difference between 'The Burlington Vermont Times' and some random news aggregator which decided to filter on location.

And there are individual blogs which have higher quality / better reporting than some regional newspapers. Should those be included as well? Which ones?

Once you start making decisions like these, you are quickly getting into controversial territory. So I can see why Google prefers to abstain and say "it's all users, not our fault!"

"it's all users, not our fault"

I don't think it's 'aggregate users choice' they use, I think it's ultimately 'page rank' reputation at the core of it.

Also, I think if they were using 'user choice popularity' the news would be HuffPo, Buzzfeed, TMZ, Breitbart etc. - we on HN don't read this stuff but it's very, very popular stuff.

Consider that HuffPo for example is more popular that most news orgs on that list, and they do actually have a lot of in-house content ... so why are they not added? Clearly Google has made some kind of editorial choice.

This is not just regular search, it's news, so this becomes more important.

I don't think there is a way around having some kind of editorial opinion, so it should hopefully be transparent, maybe pliable, possibly even regulated.

So now it's a good thing that Google prefers reputable and well-known news sources over random unknown noise?

First, I did not imply that reputability should not factor into the results.

Second, the criteria for 'reputability' is a great deal of what's at stake here.

It's not about the cost, it's about presentaiton space. A gas station is limited to one fridge (say), and has to choose what to put there.

Similarly, Google has a ~5" device with about 15-20 slots to fill in.

The actual cost of buying a fridge/extra slot is marginal, the issue is the attention of the consumer.

Of course I was not talking about it in that sense. I meant that for about the same cost, they should be able to show the most relevant results from the best source, and not just a select few.

They are doing that already. What do you think "most relevant" means -- it means "the link that most people want to click on".

Now, if I am looking at the google new results, and I see results from bbc.co.uk and "Random Guy's News Site", which one am I going to click? I'd choose bbc.co.uk every time. So they will be marked as "relevant" and will rise to the top, while "Random Guy's News Site" will be marked irrelevant and fall to the bottom of the list where no one will ever see it.

It takes up room on the shelf.

It's a very good analogy.


Unless someone comes up with a method of somehow listing all 18,000 or whatever entries on the front page, the algorithm has to choose which ones are most popular.

Well of course I was not talking about it in that sense. I meant that for about the same cost, they should be able to show the most relevant results from the best source, and not just a select few.

This is the wrong analogy.

1) "Google shows users what other users like, most of Google's current users favour a small number of major outlets"

This is only partly true, and we have no evidence of it. Google can do whatever they want their results and there are surely other factors. What are they? Why do they exist? etc.. There's room for all sorts of bias in there.

Also, I suggest it's actually not really true either: 'What people want' is BuzzFeed, HuffPo, Breitbart and TMZ. Unfortunately, that's the reality of the world. 'Most people' read that stuff. They want to know what Kim Kardashian thinks about it.

2) "But it shouldn't surprise you that most gas stations mostly stock Coke and Pepsi products."

This is a physical inventory and shelf space problem, google really doesn't have this so much: results can be tailored, represented in different ways, possibly randomized etc..

This situation is a great case for how systematic biases etc. are reinforced.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact