Wow, thank you for writing out such detailed feedback!
Your initial assumptions are correct on the inputs to the algorithm.
>the extension's rating of an "objectively good" outlet and an "objectively bad" outlet referencing themselves in a followup article would differ only based on some databases credibility ranking?
I may not understand your question. But there's an algorithm that factors in count of sources, diversity of sources, political leaning of sources, credibility of sources, and (soon) author depth of experience in that topic. So it's more than simply referencing a database. Does that give you more confidence in what the rating or not enough?
>If the user only trusted this extension, I could as an outlet easily trick it by linking in external sources such as quackblog.biz and truthiness.zone, while decrying a link from biasedrag.com to present the veneer of offering counterpoints.
True. You'd have to link to sites that are rated by Mediabiasfactcheck (MBFC) so you can't link to random stuff and get a high rating. And if the author was trying to game the rating at least they did decry the points from an alternate viewpoint; that's better than what happens with the news today.
MBFC is not perfect of course but their methodology follows the Fact-checkers’ Code of Principles developed by Poynter Institute so is generally consistent and high quality.
>I applaud the effort, but I don't think looking at sources for news works, as outside of science news, they are usually the first to post on things, and thus the only "news" linking to sources, are second-hand outlets or blogs.
Fair point on breaking news. We wanted to first solve for non-breaking-news (i.e. issue centric), which do have sources and those range widely in quality. i.e. most things discussed on HN are not breaking news but deeper news. On breaking news we plan to have some calculations based on source info like tweets/posts/videos as well as author's expertise in that topic. What do you think?
The Reuters article has no links hence we cannot rate it. Not great but maybe helping people to get accustom to looking at sources, or lack there of, is good in and of itself?
>Without transparency, you as the extension author becomes the central arbiter of news, which is IMHO more dangerous than a status quo of outlets.
Agreed. I'll update our FAQ to be clearer on how we do the rating. (Just rolled out yesterday so a bit slow on the documentation; sorry). I hope we can make clear that there is no objective arbiter of bias but rather such tools can help you get smarter about evaluating the news you read.
>A lot of news that is factual, unbiased, etc don't require a counterpoint or opposing views as a fact at its core is a fact. A lot of topics are of course aggregates where things get muddy, and statistics involve a lot of assumptions.
Agreed on both points, though I might counter that there's no such thing as unbiased news just because of topic selection and language used. Anyway, our algorithm doesn't require that counter-points be considered though it does evaluate that. So an article with high quality sources that lean one way politically can score ok but we'll notify the user of this lean preference.
We are unlikely to be able to say a claim is true or false, or that statistics are selectively being used to prove a point. That's pretty tough to do with an algorithm. But maybe the counterpoint sourcing can at least hint at if the author has considered this?
>I like the idea, but I'm unsure of the possibility of the venture.
We wanted to see if the idea has merit and your sentence is exactly what I was looking for. So thank you and thank you for all the feedback. As for the venture... I hear you, we have more to do. Please let me know of any other comments. I'm at amoorthy at civikowl dot com.
Your initial assumptions are correct on the inputs to the algorithm.
>the extension's rating of an "objectively good" outlet and an "objectively bad" outlet referencing themselves in a followup article would differ only based on some databases credibility ranking?
I may not understand your question. But there's an algorithm that factors in count of sources, diversity of sources, political leaning of sources, credibility of sources, and (soon) author depth of experience in that topic. So it's more than simply referencing a database. Does that give you more confidence in what the rating or not enough?
>If the user only trusted this extension, I could as an outlet easily trick it by linking in external sources such as quackblog.biz and truthiness.zone, while decrying a link from biasedrag.com to present the veneer of offering counterpoints.
True. You'd have to link to sites that are rated by Mediabiasfactcheck (MBFC) so you can't link to random stuff and get a high rating. And if the author was trying to game the rating at least they did decry the points from an alternate viewpoint; that's better than what happens with the news today.
MBFC is not perfect of course but their methodology follows the Fact-checkers’ Code of Principles developed by Poynter Institute so is generally consistent and high quality.
>I applaud the effort, but I don't think looking at sources for news works, as outside of science news, they are usually the first to post on things, and thus the only "news" linking to sources, are second-hand outlets or blogs.
Fair point on breaking news. We wanted to first solve for non-breaking-news (i.e. issue centric), which do have sources and those range widely in quality. i.e. most things discussed on HN are not breaking news but deeper news. On breaking news we plan to have some calculations based on source info like tweets/posts/videos as well as author's expertise in that topic. What do you think?
The Reuters article has no links hence we cannot rate it. Not great but maybe helping people to get accustom to looking at sources, or lack there of, is good in and of itself?
>Without transparency, you as the extension author becomes the central arbiter of news, which is IMHO more dangerous than a status quo of outlets.
Agreed. I'll update our FAQ to be clearer on how we do the rating. (Just rolled out yesterday so a bit slow on the documentation; sorry). I hope we can make clear that there is no objective arbiter of bias but rather such tools can help you get smarter about evaluating the news you read.
>A lot of news that is factual, unbiased, etc don't require a counterpoint or opposing views as a fact at its core is a fact. A lot of topics are of course aggregates where things get muddy, and statistics involve a lot of assumptions.
Agreed on both points, though I might counter that there's no such thing as unbiased news just because of topic selection and language used. Anyway, our algorithm doesn't require that counter-points be considered though it does evaluate that. So an article with high quality sources that lean one way politically can score ok but we'll notify the user of this lean preference.
We are unlikely to be able to say a claim is true or false, or that statistics are selectively being used to prove a point. That's pretty tough to do with an algorithm. But maybe the counterpoint sourcing can at least hint at if the author has considered this?
>I like the idea, but I'm unsure of the possibility of the venture.
We wanted to see if the idea has merit and your sentence is exactly what I was looking for. So thank you and thank you for all the feedback. As for the venture... I hear you, we have more to do. Please let me know of any other comments. I'm at amoorthy at civikowl dot com.