Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: I made a platform for journalists to “open source” their fact checking (sourcedfact.com)
593 points by YazIAm on April 4, 2019 | hide | past | favorite | 298 comments

I appreciate the recognition that there is a problem in journalism. But I'm afraid that the fundamental problems are problems in the nature of humans and aren't ameniable to this sort of solution. Those problems include:

1) Many people choose to believe what feels right to them, rather than dispassionately seeing things for what they are. Unfortunately there are a great many of people that do this regularly, and they have practiced this their entire lives. Richard Feynman cautioned us in saying that "The first principle is that you must not fool yourself - and you are the easiest person to fool." That caution was aimed at scientists who are already among the most objective of us, yet they can still suffer from this ailment.

2) When people talk about the facts, they rarely just lay out the facts dispassionately and without judgement. Humans need a motivation to tell a story, and most people's motivations include convincing other people to support their ideology, or else berating those who dont. Facts are organized within a mental framework of the ideology of the teller, wrapped up in beliefs, desires, and biases andcarry additional information about which ideological position you ought to hold -- sometimes they are even intended to mislead and manipulate the listener.

The modern news media is composed of businesses in search of profits. That goal is often not aligned with a straight reporting of facts. By pandering to an audiences ideology they increase ratings. By reporting on salacious and exciting news, even if incorrect, they get ratings. Buyer beware.

> Unfortunately there are a great many of people that do this regularly, and they have practiced this their entire lives.

A certain percentage of the population does not have critical thinking skills and never will. However a project like this is useful for those of us who do. If journalists don't disclose their data and reasoning its hard even for those of us inclined to verify or fact check anything, leaving us susceptible to propaganda.

False. Trust in journalism is built by proving reporting to be of quality over time. Not by providing a bibliography before the report ever dries.

Even if that were true, I don't see why providing a bibliography would be a bad thing.

The answer to 1 most likely lies in education and (more broadly) reduction of economic segregation. You will not save everbody with this, but the majority.

I would divide media roughly into three classes:

1. The worst: those who make up things with no base in reality at all

2. The standard: those who don’t make up things willfully, but manipulate through selective focus (leaving out certain topics while pushing others)

3. The good: those who carefully try not to lie and try to avoid selection based manipulation, while clearly marking the border between facts and opinion.

This concept would only really prevent the behaviour of the worst – IF they would care about getting caught in a lie. The fact that there _are_ watchblogs that point out lies with no effect shows probably that only technology is not a practical solution.

What kinda works is having a national press council that has the power to sue media plattforms to print corrections when they get things wrong. Even the most uneducated would think about changing the newspaper when ever other issue starts with a correction notice. Let alone the advertisement partners.

Technology like this is cool, but must always be emmbeded in some powerful institution.

What kinda works is having a national press council that has the power to sue media plattforms to print corrections when they get things wrong. Even the most uneducated would think about changing the newspaper when ever other issue starts with a correction notice.

Actually, people have been calling out media organizations and getting them to print retractions and even suing them. When they do retract, they do it in a way where it doesn't get much space and much attention. (The mainstream media is particularly bad about covering this sort of thing.) Some (even mainstream) organizations in 2018/2019 would simply ignore the claims. Some (even mainstream) even quietly edited articles without retraction.

In any case, I don't think most media organizations care, because most of the time, the retraction isn't nearly as viral as the initial clickbait story, and even if there was outrage, that would still ultimately benefit them by ultimately generating more attention.

I grew up in Austria, and over here they usually have to print it on the front page and not too small. Same goes for web stuff.

For an excellent book on your points, see "The Righteous Mind" https://righteousmind.com/

3) Whenever there is a chance to gain wealth and/or influence without immediate disadvantages, humans will usually take it.

Another fact, in some domains I don't think reporters can ever write anything useful, it would take very high viewpoint and years of experience in the field to say something ..

Reporter reports

Indeed. The heart is deceitful above all things, and desperately sick; who can understand it?

> desperately sick

How so?

The common sentiment that marriage takes hard work on the daily or else it turns sour, ideas like that point to a fundamental difficulty with the human heart, no?

Well, this seems very subjective. Personally, I believe there are many ways to soothe one's heart.

Though without any care, you're probably right.

Awkward question here... can anyone cite an example of a major-media news story that was actually factually wrong in a way that "open source fact checking" could have detected it?

Just because a bunch of people believe it's a problem doesn't mean it's a problem. I find the bias, when it exists, is in the presentation, not the facts themselves. And mainstream journalism is really committed to getting the facts right.

Claas Relotius, the Der Spiegel journalist who won CNN's journalist of the year award who was revelead to be completely inventing stories about people to make enticing click bait.


This is actually a very good example of press failure. But also possibly one which shows how difficult it would actually be for "open source" fact checking based on consensus voting to pick up blatant lies. Relotius got away with his spiel for a long time and won awards because none of the people he claimed to have interviewed read his articles (especially not the people who didn't actually exist!). It's also reasonable to imagine that Relotius wouldn't have volunteered to put his stories on a fact checking platform, but more importantly even if somebody else did, very few online volunteers would have been in a position to actually validate them. Most of the stuff he made up was personal details about individual ordinary people, and even the trivially falsified stuff like a town he visited not being surrounded by forest and not having a town sign emblazoned with "Welcome to Fergus Falls, home of damn good folks" isn't the sort of stuff fact checkers like to get their teeth into. And the latter would be the sort of thing you'd want to establish by asking one person who lived there, not getting a consensus of netizens.

How I see it playing out:

1. Individual journalists do not sign up for this, publications would, as an integration for every article.

2. A casual reader from Fergus falls would notice a few weird things, and start raising questions.

3. That would start the snowball into all the other things the 'fake journalist' would be doing, at least for the article. Once you get popular enough, the internet haters will follow through, since anyone with something of a following starts attracting internet haters into their life eventually.

This will have all the general problems all internet comment sections do currently although, like twitter.

A site would have to get quite big to have many casual readers from Fergus Falls (population ~13000) preapproving its German articles though, even taking into account some people remember meeting him and eventually got round to blogging about it.

Actually, it's interesting how little attention internet haters paid to Relotius, considering he was writing on polarising subjects, winning awards and he was arrogant enough to throw in details that were trivially proven wrong like election results as well as random deniable lies about the individuals he interviewed or said he interviewed. Perhaps writing in German and on things happening overseas helped shield him. But yeah, agree that internet haters are the way forward for crowdsourced factchecking (at least on subjects a bit less partisan than politics or sport where the haters can generally be relied on to reach some sort of consensus)

He either invented sources or made up quotes from real people he never met. How do you expect this platform to remedy that?

There were a few tells one could find, at least after the fact. For example, he described a women locking her door „with three turns of the key“. That’s something that works on German locks, where the bolt keeps advancing. But the incident was supposed to have happened in some southern US state. And in the US, most locks only allow a quarter turn. I asked specifically about the prevalence of different lock types in the state in a HN threat at the time.

That said, this is a singular and decidedly rare event. The insinuation of widespread whole-cloth fabrication is almost certainly wrong, and probably more damaging to democracy than any actual fraud will ever be.

The three turns is only a salient mess-up in light of other, much more damning evidence. On its own, it's credibly explainable as an honest mistake (I was there but who counts these things? I just assumed).

Yeah I don't know how well this tool could remedy that, or whatever else it hopes to solve. I don't see how this is hardly different than just sourcing your own facts with Google.

Digital signatures from sources would be a nice feature, maybe some kind of trust system such that sources could be kept anonymous.

My favorite example, much clearer even than Covington (and never corrected in the major media): the bus driver who drove drove his bus off of a bridge: https://www.bbc.com/news/world-asia-china-46068412

For whatever reason, the official police report implied that a passenger struggled with the driver causing the disaster. The next day, video footage of the entire event was released (from the security cam inside the bus), showing a different story. The driver pretty clearly decides to go over the edge, turns the wheel, and does not flinch or turn it back as they go over.

After the video came out, dozens of major outlets covered the story. Almost all ran with "passenger hits driver causing crash", one or two did not specify a cause. No major outlet suggested [what seems to me to be] the most likely account, which is "driver deliberately drives bus off of bridge during argument with passenger".

In contrast, on youtube / other social media, most commentators pointed out that the driver seemed to crash deliberately, changing course many seconds after the passenger hits him on the arm with her phone.

This one fascinates for a few additional reasons, which a third-party site that supports revising and completing stories could address:

* It shows how automated the echo chamber is. The real crash cause is a more interesting + rare story. (though needs A/B testing) * It shows how short-lived major coverage is. While most passengers unfortunately perished in the crash, _both the driver and the arguing passenger survived_, and will end up in court once they recover. This was mentioned by very few outlets, but will eventually help inform understanding of the event.

What's interesting about this is that there isn't even a narrative or a partisan bias here. The media just lies because there's no reason not to.

The most recent and obvious case in my mind: Covington Catholic Incident: almost every major mainstream media outlet ran with the narrative that the boys were harassing the Native Americans which was based solely on one video clip. No traditional investigative work was done. No one was interviewed (until the damage had been done). Nothing was fact checked. It was just one video, and boom: instant headlines.

Excellent example- it will be interesting to see if they win their defamation lawsuits against CNN and the Washington Post.

I disagree. Whether they were harassing is a matter of opinion. Whether there was a confrontation is a matter of fact, confirmed by the video and photos.

If you want to claim the "narrative", you need a link to an actual article with actual quotes. That's how fact-checking works, and your response is failing it badly.

>Whether they were harassing is a matter of opinion

The entire event is on video. The boys are waiting for their bus. They are repeatedly insulted by a group of adult men who use racial and homophobic insults against the boys, who do not respond in kind but do talk with the adults harassing them. Into the fracas enters Phillips, who approaches the boys, and bangs a drum in the face of one boy. Most mainstream news outlets then smear the children as racists who surrounded and harassed Phillips without regard for the events on film.

If you want to describe this as a matter of different opinions, then I think you're basically in the camp of just disregarding reality and making up whatever is most pleasant to your perspective.

"If you want to claim the "narrative", you need a link to an actual article with actual quotes"

This is essentially what you're looking for: https://www.theatlantic.com/ideas/archive/2019/01/julie-irwi...

The short story is that when news about the incident first emerged, it was lacking the context about the Black Hebrew Nationalists antagonizing the boys. Early coverage made it sound like the boys sought out Mr Phillips to antagonize him. When it turned out that he had inserted himself into the situation, rightly or wrongly, it changes a lot. This doesn't necessarily exonerate the teenagers--you might still find stuff to criticize about their behavior--but it does provide a more sympathetic context, one that was missing with the initial press coverage.

So yes, whatever your final takeaway, the initial coverage was incomplete and people did form and share very strong opinions based on that incomplete coverage. Once more information came out, many people changed their minds while others stuck by their initial takes. Had the coverage been more complete before people starting making opinions, we might have avoided a lot of unnecessary kerfuffle over this.

> Whether they were harassing is a matter of opinion

But when you report it as a matter of fact in an editorializing headline, then it is a problem.

> Whether there was a confrontation is a matter of fact

This was not the narrative portrayed, in the slightest bit. Had this been the headline and narrative, I would have had no issue with it, either. The stories that day were "Boy in MAGA hat harasses Native Americans" period.

See previous post. If you can't produce links and quotes, then that's just, like, your opinion, man.

Firsthand quote from the VOX article...

“It was getting ugly, and I was thinking: ‘I’ve got to find myself an exit out of this situation and finish my song at the Lincoln Memorial,’” Phillips told the Washington Post in a separate interview. “I started going that way, and that guy in the hat stood in my way, and we were at an impasse. He just blocked my way and wouldn’t allow me to retreat.”

The article is explicit. It notes that Phillips initiated contact because they were harassing another group. But it also makes it clear, with quotes, that he believes they began harassing him instead.

Now, if you want to say Phillips is lying, you can do that, but unless you do that, at least one of your articles does not support your contention.

The article was modified, and the sections in your quote added. I should have used the internet archive originally.

See: https://web.archive.org/web/20190120030012/https://www.vox.c...

Greatly looking forward to how beat twists himself in knots to interpret this one as "factually accurate".


Neither your, nor my politics have absolutely nothing to do with the accuracy of your information, though if anything, we should hold our own side to a higher standard. There are so so many easily verifiable things to write about, that its beyond pointless to trash our credibility over things that are proven inaccurate with a little bit of fact checking. I mean, Phillips being the wrong age to have been in Vietnam, as he claimed to Vox, would have been trivial to detect with any fact checking at all, and should have raised flags about the rest of his narrative.

No one here is even trying to advance narrative high school boys are angels, with a bit a of digging it's pretty easy to find things that Covington Catholic does may well deserve the level of public shaming that this story got.

However in this case, the entire narrative has been pretty widely disputed, Caitlin Flanagan does it much more elegantly than I could.


As for the source, why not go to the original, like any responsible journalist should have, copy of the full video: https://www.youtube.com/watch?v=QwNyOD8FIQk

Feel free to tell me when you hear them chanting "Build a wall", I haven't heard it.

The first widely reported, and easily proven incorrect fact is that Nathan Phillips is a Vietnam Veteran, see fairly through coverage of that here:


So really you're asking me to believe the narrative of someone who is proven to be lying about his military service, with no corroborating video of his claims, which just seems a bit ludicrous.

edited: spelling

Even after you have fallen victim to the very thing you are arguing isn't a problem - you still don't see it as a problem? It is a very big problem that these articles are being destructively modified. They are whipping up people that are already so freaked out that they think the President is both Hitler and a Russian agent, and people like you are left wondering what the big deal is when you look at the results of stealth edits.

   -- a young man wearing a self-assured smirk and a red “Make America Great Again”
   ++ a young man wearing a red “Make America Great Again” hat

   -- More video shows Phillips wiping away tears in reaction to the harassment.
   ++ More video shows Phillips wiping away tears after the incident.
You can be sure that a huge number of tweets and facebook posts, by politicians and journalists/pundits promoting the story, frequently found in the sidebar of articles like this - got memory holed as well.

Don't bother with the "you don't know my politics" shtick, your overly sensitive reaction leaves the writing all over the wall on that point. You wouldn't be so defensive if it didn't strike close to home.

This would be less silly if what you were pretending is so obscure and nuanced was not actually as plain as day.

I took your invitation to reread, and only needed to go as far as the second paragraph: "In some shots, the teens appear to be shouting “build that wall, build that wall.”"

This is false. Not an opinion, not a narrative, a plainly false statement.

My "contradictory sources with at least the same level of journalistic respectability" is any of the videos of the event themselves, is that a credible enough source for you? The full-length video was available at the time of the writing of this article and Vox did not make any effort to verify what they wrote against it.

They lied, period. This is not that difficult to understand. Do try to keep up.

I believe that you are making a mistake when you say "not an opinion" in your (parent) post.

The words "appear to be" in one of the above quotes precisely indicates that this is a statement of opinion, and not a statement of fact. When we say "the wall is blue", we are talking about the color of the wall factually. When we say "the wall appears blue", we are talking about how someone sees the wall, which is a subjective observance; the wall may appear to be a color other than blue to someone else with a different pair of eyes.

At least one word, "is", refers to things as a matter of fact. Other words like "appears", "seems", or the sub-phrase "I think", are indicators that we're probably talking about a statement of opinion. This may be a useful rule to follow when reading news stories. I think it seems relevant here. (SwIdt?)

Your argument would be more interesting to read if it didn't include personal attacks (which are against HN rules).

This is an annoying sentiment for a recent national news story.

This has been debunked thoroughly. That guy went up to the kids and started beating his drum right in their faces to intimidate them and somehow the media painted the kids as the harasser.

Was his intent to intimidate? Why do you say that? How is that a "fact"? Where did you even hear that story?

Geez, y'all don't understand fact checking at all.

Does it matter what he says his intent was? He can paint whatever story he likes in interviews, but nobody would appreciate a random person going right in their face and beating a drum, especially a young kid. You can even see him flinching because of the proximity.

Not sure what you're talking about "hearing that story", the video evidence is clear as day.

[deleted unpopular opinion]


I recommend against deleting unpopular comments. Editing in something like "Update: I disavow!" if you've changed your mind is harmless, but in general there's a benefit to treating public statements as a sort of permanent record.

This practice exposes people to potentially underappreciated viewpoints and allows others to track the development and popularity of ideas.

I have many poorly rated comments preserved here, but on balance I think people respect my personal commitment to the truth, however I see it. In the realm of public expression, it's honesty and integrity which matter most. Everybody is wrong sometimes; it's part of the process of discovering truth.

You are 100% right, I should have left it there. I only rarely edit/delete my own comments, and almost always it is because I realize I have waded into a political argument I don't want to be part of.

Fair enough! Cheers.

You assume the drummer was trying to go somewhere such that someone could block his path.

But he was exactly where he wanted, in this kid's face trying to provoke him. And the kid just smirked, earning great hatred and hostility from people who don't think there's any place for a smile in this cold, brutal world.

I assume the kid wanted the confrontation just as much, hence the smirk and refusal to make a hole for the guy to walk through. Even if it is exactly what the old man wanted, the kid is not blameless.

Again, you're assuming the kid had the option of yielding so that the drummer could "walk through." Watch the original video that includes the surrounding events.

The kid did have an option to yield: getting on his knees and begging for forgiveness for the crime of appearing at an anti-abortion demonstration as a white Catholic male. Then the drummer (who is a 'professional protestor', by the way) would have had a more dramatic story to sell.

Instead, he smirked.

Are you familiar with the story at all? You need to do more than "hear" about it from whomever you did. I watched a full recording of what happened, and the media coverage was absolutely ridiculously wrong. I hope the Covington kids win a defamation suit.

The confrontation was between the kids and the black Israelites. The Native gentleman was trying to use chanting to interfere with the conflict.

If you watch the video, that kid with the long face was just standing there looking goofy because he was uncomfortable as the drummer walked right up to his face. There is really no way to misread that interaction. He's a teenager for god's sake and he was being harassed by ideologues.

>Whether they were harassing is a matter of opinion

It really is not, further video evidence makes this a fact, not an opinion.

“Vietnam Vet” — that was published by the mainstream media. But the Native American wasn’t a Vietnam vet. He never left the US and, in fact, was AWOL twice. The media used “Vietnam vet” to elicit sympathy, but it wasn’t even factually accurate. The actual facts were ignored in order to promote a particular bias. “AWOL soldier” doesn’t have the same emotional value to the narrative as does “Vietnam combat vet.” The history of the Native American, including past protests and confrontations was highly relevant to the story but instead, the media focused on the the teenagers rather than the professional agitator as well as the hate-group the Black Israelites.[1]

The Covington story was “privileged, white teenage, racist Trump supporters got in a confrontation.” However the facts had to be massaged in order to sell that fiction was what happened. We all know that was the intent of the media and to suggest the reporting was either factual or objective is nonsensical. It’s delusional to suggest that the media treats Trump supporters objectively. Look, for example how the New York Times classifies Joe Biden’s repeated sexual harassment as “tactile politics,” while anyone affiliated with Trump gets no such delicacy in headlines.

It’s absurd to suggest that the Covington story was anything except the media attempting to make those kids look like evil puppy-killers. The vile, racist comments of the Black Israelites on the scene were ignored for the most part and instead media talking heads spent hours analyzing the alleged “smirk” of the white kid with the MAGA hat.

There is a narrative and any reasonably intelligent person can clearly see it.

[1] https://www.splcenter.org/fighting-hate/intelligence-report/...

> Whether they were harassing is a matter of opinion.

Perhaps. But the question of who approached who is a matter of fact. Initial reporting claimed the teenagers approached a group of Native Americans. The videos clarified that the Native Americans approached the teenagers.

And why couldn't media conglomerates get the facts right?

The assumption behind this idea isn't that there are no accurate journalism outlets out there, it's that by "open sourcing" their fact checking, these outlets will better stand out, adopting a standard that inaccurate outlets just can't meet.

Right now, outlets are basically asking you to trust their assertions based on their brand. Fewer and fewer readers are inclined to give their trust based on brand (I'm guessing this is because there are just so many outlets/brands now). The way forward is for journalists to earn their readers' trust by showing their work on as many fact checks as possible. It won't be possible for every fact check (such as those involving anonymous sources) but where possible, it'll add a great deal of value.

"An article on Sunday about once-dominant N.H.L. teams adjusting to an evolving league misstated the number of conference finals appearance by the Kings, the Ducks and the Blackhawks since 2009. It is 10, not nine. Because of an editing error, an earlier version also misstated the number of playoff appearances by those teams in that span. It is 24, not 27."

An example from the new york times from yesterday. Obviously not an earth shattering error, but there it is. Something trivially easy to be detected by anyone who cared to check.

For more check out https://www.nytimes.com/section/corrections

Yeah, and they corrected the error themselves and published the correction. This is why they are trustworthy.

Making a mistake is not the same as lying.

Making a mistake is not lying, but it is a moral failing to promulgate fiction as fact because it accords with your preferred narrative, especially when it's done by the media, who are trusted because they are supposed to practice restraint and good judgment.

And when has "mainstream media" ever "promulgated fiction as fact"? This thread started with an acklowledgement that intentional errors, exceptions, or mischaracterizations are exceedingly rare in well-reputed news outlets.

Frequently "the news" puts telling a good story ahead of Truth. That is, they don't tell the most informative version of the story but the most compelling version or up/downplaying a story based on bias.

This doesn't involve outright untruths or even significant manipulation of facts.

It does involve things like cherry-picking facts, quotes, and statistics that paint the desired picture without drawing conclusions while the reader's intended conclusion is obvious.

One narrative is that News outlets knowingly put out false or incomplete information because nobody cares about edits or retractions after the first day or so. They get clicks from outraged people and then after the cpm drops off they edit the story with the real facts, to claim that it was a "mistake". Kind of conspiratorial, but given the political bias of a lot of media outlets not completely out of question.

Corrections are often the result of external people pointing out errors.

Who said anything about lying?

"External people pointing out errors"? How is that any worse than this hare-brained software idea? You still have to wait for somebody with the corrected information to hear of the story and recommend an edit. This is just adding a middleman that doesn't add value.

>Can anyone cite an example of a major-media news story that was actually factually wrong in a way that "open source fact checking" could have detected it?

The Supermicro """backdoor""" story of late last year, where Bloomberg alleged that the company had surreptitiously placed small microchips on nearly all their products on behalf of China without anyone noticing

Was that ever conclusively disproved? By whom? And with what evidence?

That could well fall under he said she said.

That could well fall under he said she said.

That doesn't seem like a sensible standard, most reporting falls under 'our sources said vs what someone else said'. Many details of this particular story were later disputed by others, including Apple. Would a crowd-sourced fact-checking service have helped here? No, because it wasn't at all necessary. It's still a story with serious problems - it's 'the anonymous sources of the reporters' vs 'just about everybody else involved'.

>> And mainstream journalism is really committed to getting the facts right.

Is it? Maybe it once was but the collapse of local news and the massive drop of talented writers and support staff like editors and fact checkers seems to have changed the environment. Most news today is little more than Twitter posts cobbled together.

Because arguments about facts take place all the time when people dismiss citations, so having a source that exists specifically to document claims in open fashion becomes a useful reference.

I maybe misunderstanding your question, but this platform only accepts public primary sources as valid evidence. A media news story is never a primary source for proving the assertions within it, it's a secondary source.

Edit: I had originally misunderstood the parent comment. Responded again here https://news.ycombinator.com/item?id=19576185

Right, I get that. And my concern is that mainstream journalism that actually contradicts public primary sources isn't the actual problem. I believe that it is, in fact, vanishingly rare. Hence my question.

And my concern is that mainstream journalism that actually contradicts public primary sources isn't the actual problem. I believe that it is, in fact, vanishingly rare.

Take a look at niche journalism. The bias and narrative pushing is quite massive there. I think the "garbage tier" label given to such organizations is well deserved. Yet, when mainstream journalism gives its attention, it follows the niche journalists lead.

I find journalism that actually contradicts facts to be quite common. In today's climate, the mainstream can get away with complete contradiction of the facts, so long as the targets are obscure and/or unpopular, and the "right" narrative is being pushed. If you want to find this, then you need to start looking into the dissidents of mainstream culture in 2019. (You don't have to go as far as toxic people like the Alt Right. Rather, investigate the people who are being mislabeled as "Alt Right" as a tactic to marginalize them.)

That's an opinion, not an example.

I asked for examples. I'm getting opinions.

I cited a couple of examples in another comment. In any case, I didn't proffer that as just an opinion. It's also a prescription for how you could find tons and tons of evidence. Explicitly look into people who are not "Alt-Right" but who have been labeled that way as a means of quashing their message. This will often follow the pattern of the Boston Rally: Non white people for whom the media is pushing an anti-non-white racism narrative.

You should read through some of the fact checks, especially the first one for Net Neutrality. They address literally this "presentation" aspect.

This was recently posted on HN. It actually got me thinking how I might write a service like OP's.

I wrote a story that became a legend. Then I discovered it wasn’t true: https://www.cjr.org/first_person/tinazzi-motorcycle-mont-bla...

> I find the bias, when it exists, is in the presentation, not the facts themselves.

You mean something like a lie by omission? That is pretty easily solved with some form of source control and public pull requests. An equivalent to git blame would be pretty helpful (cue Sinclair Broadcast Group NPC script reading). Just watch http://newsdiffs.org/ for a while, these people regularly modify stories without editor's notes. It is kind of funny that further below you discuss the embarrassing conduct of the media in their coverage of Covington Catholic... do you remember what was in the news cycle right before they breathlessly pounced on a non-story? The media was getting embarrassed by a very rare direct denial from the DoJ in relation to their latest reporting on the Trump collusion conspiracy theory.

I agree with your broader point though: this won't improve the media. They're not even pretending to be unbiased and have no shame in telling laughably obvious lies about the most ridiculous thing - like koi ponds. Their motivation is clear: short sighted servicing of those needing confirmation bias conforming stories. This isn't a scheme hatched in a smoke filled back room, it is just a very foolish business plan.


Covington students.

justin smollet? 99% of the media?

You realize that "being wrong" is not the same as "lying" right?

Every news source reported on the inconsistencies in Jussie Smollett's story, and continued to update their reporting as the facts turned against him.

They're newspapers, not oracles.

it is. they reported that it was a racist attack - because of trump - before checking the fact. it is fake news.

They reported on the claims that were made. Once again, newspapers are not oracles. Facts change from day to day and stories develop over time.

There's a reason that the Times puts out a new edition every day, instead of releasing just a singe one 2000 years ago that we all worship as indisputable truth.

"Fake news" is reporting that Barack Obama, Black Lives Matter, and Anonymous are planning riots in 37 city centers at 7 pm on July 15 2016, as Breitbart, the Daily Caller, and other far-right news sources reported two days before. The story was entirely made up, and most of the text was copied from a chain email from the 00s.

hence why you do not report on something as being a fact before it is proven and before you investigate the claim. Its like me reporting the earth is flat because someone said it.

No other facts were available. The claim was itself the news, with updates pending a police investigation. This isn't complicated. Investigating crimes is the police's job, not every single newspaper in the world's.

Should all of the world's newspapers have held off on reporting the MAGA bombs until Cesar Sayoc's conviction?

Should we all have pretended that nothing was wrong about the world trade center until the 9/11 commission filed their final report?

not my point. but don't say it is a terrorist attack before proven. don't say it is a racist attack by two white men wearing a maga hat before its proven. can you imagine implications of people lynching to people with red caps in the street thinking its them?

And at what point did the evil lying media say those things? They said that "Jussie Smollett claims he was attacked by a group of racists late at night. He was relased from the hospital with minor injuries. Police are investigating it as a hate crime". Those are facts.

Here are some examples, quote the statements that were verifiably false at the time.

Police report offers chilling details on Jussie Smollett Attack - https://abcnews.go.com/Entertainment/newly-revealed-report-o...

Jussie Smollett attacked in possible hate crime - https://www.cnn.com/2019/01/29/entertainment/jussie-smollett...

Jussie Smollett releases first statement about attack - https://www.nytimes.com/2019/02/01/arts/jussie-smollett-atta...

Police say Jussie Smollett attacked in possible hate crime - https://www.npr.org/2019/01/29/689758253/police-say-empire-a...

Jussie Smollett hospitalized after attack, police are investigating as a hate crime - https://www.foxnews.com/entertainment/empire-actor-jussie-sm...

can anyone cite an example of a major-media news story that was actually factually wrong in a way that "open source fact checking" could have detected it?

There have been many of these recently. One striking one was the Boston Rally for which the mainstream news pushed a "white supremacist" narrative. This was odd, as the few people who managed to come out included a large proportion of brown people, including one South Asian senate candidate. The fact checking seems to go out the window, so long as the story pushes one of the "sacrosanct" narratives.


Another example was the coverage of the public harassment incident at VidCon between Anita Saarkesian and Sargon of Akkad. The way it was reported, it sounds like Sargon did something, whereas videos online show that all the harassment was done by Anita Saarkesian. You may not like Sargon, but the fact that the news narrative goes 180 degrees against the truth on the ground should disturb everyone.

I find the bias, when it exists, is in the presentation, not the facts themselves.

Manufacturing Consent covers how western media entirely fits a "Propganda Model" of operation through presentation alone. However in recent years, even facts are thrown out in favor of narratives which run entirely counter to the facts.

And mainstream journalism is really committed to getting the facts right.

Not so much. I recall reading a NYT article about a storm-in-a-teacup thing involving who gets to walk first as valedictorian. The article was amazingly nearly fact free. It seems that articles and reports get a free pass, so long as they are pushing the "right" narrative. It has been this way for a very long time (re: Manufacturing Consent) but the quality of fact checking and the quality of the writing used to subtly spin the facts has gone down.

People complained mightily about how all of the "Weapons of Mass Destruction" evidence was one big circle-jerk of "Citogenesis."


Now, the same media organizations are involved in the same circle-jerks of "Citogenesis," but it's given a pass for tribal reasons, because the narrative supports "our side."

Regarding the Boston rally and the south Asian candidate: you're talking about Shiva Ayyadurai, the guy who (in my understanding of reports, which could be erroneous) sues people who contradict his claim of having invented electronic mail, has (to my understanding, which could be flawed) appeared on white supremacist podcasts, and used (as I understand it, based on reports) a caricature of himself as Pepe the Frog in his campaign pins? Who appeared (I have read in sources) in a video with a Charlottesville alt-righter and "blessed" a statue of "kek"? That's the basis of your claim that the media got this story wrong?

Help me understand why you were even paying attention to this drama? Do you really find Shiva Ayyadurai to be a sympathetic figure? I assume no, right?

Regarding the Boston rally and the south Asian candidate: you're talking about Shiva Ayyadurai, the guy who (in my understanding of reports, which could be erroneous) sues people who contradict his claim of having invented electronic mail, has (to my understanding, which could be flawed) appeared on white supremacist podcasts, and used (as I understand it, based on reports) a caricature of himself as Pepe the Frog in his campaign pins? Who appeared (I have read in sources) in a video with a Charlottesville alt-righter

None of that is evidence this guy is a white supremacist. Which podcast did he appear on?

and "blessed" a statue of "kek"? That's the basis of your claim that the media got this story wrong?

That's hardly evidence of anything. The "Kek" character meant a lot of things to different people. Again, this is an example of narrative pushing and Citogenesis.

Yours is the side that has to explain away facts with narratives. If you look at the Hard Bastard video, you'll see a number of brown faces in that "white supremacist" rally. To explain that away, you need more narrative.

In any case, that narrative isn't even sufficient to avoid the charge that the media is being misled by narratives. By the network's own telling, they "didn't have access" to the Gazebo and what was going on. So on what did they base their narrative? The answer, is that they just took the narrative from far left people and didn't check.

Also, note that the VidCon event fits this pattern. Video evidence from facts on the ground tell us: Some people are attending a convention together and just sitting there peacefully, then a speaker starts harassing them. Those are just the simple facts on the ground. To explain this away, more narrative is needed -- that of a harassment campaign and shadowy conspiracies. There is no evidence for this campaign. Investigation by Patreon and earlier law enforcement concluded there was nothing there. Instead, video criticism is spun as "harassment" by a narrative, which gets taken up into the mainstream media unchecked and unverified.

There is a bubble englobing SV CEOs and the surrounding circles, and it's mostly narratives supported by emotions and vice versa. (And not a little dishonest spinning and editing of dialogue and interviews, for which there's ample evidence as well.)

Sure, in this case, "white supremacist" might not be the technical term for the bigotry involved. And, regarding the many meanings of kek, Pepe, and all that (and aside from the fact that the person who asked him to "bless" the statue is a white supremacist) --- sure, if you say so, dude.

might not be the technical term for the bigotry involved.

If you look at the signs, there was a black man concerned with the identitarian nature of Black Lives Matter. Are you trying to tell me that he's bigoted? Rather, I think he's concerned with bigotry masquerading as activism.

aside from the fact that the person who asked him to "bless" the statue

Again, this is weak "guilt by association." I have no idea if that other party is a bigot or not. It doesn't matter. Again, it's narrative generation, with the purpose of effective silencing by tarring.

Silicon Valley is largely encased in a bubble produced by and consisting of accusation, name calling, and narrative pushing. The purpose of this is to gain the power to demonize and silence people. The devious thing about it, is that more outrage is generated when the accusations are false, and that fuels more noise and breaks down discourse even further.


sure, if you say so, dude.

In history, there have been many examples of arrogant groups who achieved economic success and power, became arrogant, and got trapped in bubbles of perception.


They all had disparaging names for outsiders/underlings and thought of themselves as "betters." Most of the time, they thought of themselves as benevolent and benefactors. History is pretty clear about the lessons when those groups start imposing their misguided will on the people.

If you think agreeing to appear in a video with a white supremacist and joking about his "kek" statue doesn't paint at least part of a picture, and that suggesting otherwise is "guilt by association", that's on you. To me, it's more telling that your best, leading example of a place where the media got its facts wrong is Shiva Ayyadurai is also telling, and also paints part of a picture (not one I've described on the thread so far, just to be clear).

If you think agreeing to appear in a video with a white supremacist and joking about his "kek" statue doesn't paint at least part of a picture

"Paint part of a picture," Is just another way of saying "Narrative."

To me, it's more telling that your best, leading example of a place where the media got its facts wrong is Shiva Ayyadurai is also telling

So it doesn't matter that they got the facts wrong, because the people they are getting the facts wrong about are "bad" people. Sorry, but that's morally bankrupt. That's the classic moral stance to justify propaganda. Who in the past took actions against people they thought were generally bad, facts be damned? You appear to be in an epistemological calamity. You seem to be trapped in a bubble, where proof invalidating the bubble is sequestered through unsupported narratives.

also paints part of a picture (not one I've described on the thread so far, just to be clear)

So then you threaten to tar me by association as well? Again, it's this pushing of a narrative with no proof.

(Just to be clear, all identitarians are in the wrong. All politics by violence is bad. All politics by intimidation is equally bad, as is disdain for facts and the truth.)

I am simply going to point out again that your best example of the media getting its facts wrong is a video in which "Dr. SHIVA Ayyadurai,PhD(M.I.T.) Inventor of Email" --- a repeat InfoWars guest! --- complains about a BuzzFeed listicle-grade story that is itself at pains to point out that the (very few) attendees of an anti-left demonstration in Boston were not all white supremacists.

As you've no doubt surmised: not many people are interested in following you down "Hard News Network" video rabbit holes, for the same reason that even though the big-bad-scary-media reports it to be so, we're also not all that interested in what Alex Jones has to say about the Sandy Hook victims or the effectiveness of vaccines.

As I said, the picture you're painting of yourself is not the same picture Shiva Ayyadurai has painted of himself.

But I can sum both up as: "play stupid games, win stupid prizes".

By the way: it is my understanding, a personal, subjective understanding based on reportage and accounts from other people that I have read, presented here as a statement of my own opinion and not a relation of specific facts, that the claim that Shiva Ayyadurai invented email is laughable.

By the way: it is my understanding, a personal, subjective understanding based on reportage and accounts from other people that I have read, presented here as a statement of my own opinion and not a relation of specific facts, that the claim that Shiva Ayyadurai invented email is laughable.

I wouldn't disagree with that. None of that is really relevant, however.

It's not relevant whether the source you presented for your claim that the media got the Boston right-wing rally wrong is himself (a conclusion I've drawn from sources and reporting and, to be clear, a statement of pure opinion and not of fact, and I will remind you: opinion about a very public figure) habitually dishonest?

your best example of the media getting its facts wrong

You say "best." It's just one example. You are implicitly admitting that it is an example of where the mainstream media got the facts wrong:

a BuzzFeed listicle-grade story that is itself at pains to point out that the (very few) attendees of an anti-left demonstration in Boston were not all white supremacists.

Furthermore you are also admitting that the mainstream media narrative ran 180 degrees counter to the truth.

"Dr. SHIVA Ayyadurai,PhD(M.I.T.) Inventor of Email"

Then you cover it up by spinning a narrative. None of that is relevant. Also, thanks for demonstrating the mechanism of pushing narrative once again.

As I said, the picture you're painting of yourself is not the same picture Shiva Ayyadurai has painted of himself.

The point is not to convince you. The purpose for me is for 3rd party readers to see. So long as someone speaks out against the narratives, other dissidents will see that someone, somewhere understands.

But in the interest of hope, please pay attention to exactly what it is that you are saying. You have admitted that 1) the mainstream media got the facts wrong, 2) the media narrative was pushing in a deceptive direction in the opposite direction from the facts, and 3) nothing of the factual accuracy of the media, or the fidelity of its pushed narrative matters to you because the targeted people are reprehensibles.

Isn't this the attitude of every arrogant, out of touch elite, unknowingly doing injustice in history, ever? In what moral universe, is it alright that the news sources are so wildly wrong, and the narrative is so contrary to the truth, but it's alright because the subjects are so disliked? I know of extra-legal organizations from just a few decades back who didn't care too much about who exactly was guilty of what, or the facts, just that the people they were acting against were thought of as "reprehensibles" in their world view and also had the "correct" external attributes to be their appropriate targets -- in short, to fit their narratives. I've had such actions taken against me, personally, by such people. Such actions are particularly unjust, at root precisely because of the epistemological disconnect. When I engage in such discussions here, it's this very disconnect which most strongly motivates me.

Please pay attention that you are taking, in principle, such a stance. If your attitude is at all representative of SV CEOs in general, then that doesn't bode well for society at all.

I'm neither in silicon valley nor a CEO. Are you sure you're alright?

> The way it was reported, it sounds like Sargon did something, whereas videos online show that all the harassment was done by Anita Saarkesian.

He had been involved in a long harassment campaign against her, and then shows up in the front row at her panel. That is not an innocent act.

He had been involved in a long harassment campaign against her

There is no evidence of this. This is a Citogenesis fiction pushed by niche media. Making videos criticizing someone's videos isn't harassment. Patreon did an investigation just after the incident. My understanding was there was nothing. If there's evidence of him doing anything, it is actually of him attempting the "harassment" (really just insulting) white supremacists using toxic language.

then shows up in the front row at her panel. That is not an innocent act.

If there is no harassment of Ms. Saarkesian, that is an innocent act.

Note the pattern here. There is simple, straightforward evidence of people just sitting there while a speaker hurls abuse over the PA system. To explain this away, a narrative is needed, which includes some sort of conspiracy theory. Immediately following the incident, Patreon found nothing. Just prior to that, the GamerGate movement was investigated by law enforcement, and they also found nothing.

Just take the facts at face value. Occam's Razor. The simplest explanation is that mainstream media takes the narrative from biased niche media and does not fact check. Please notice that the evidence on your side is all weak "guilt by association" and easy to fabricate narratives.

Your cited examples are: a 40 minute youtube video that starts with a narrator saying 'Why I think today is a win for the free speech side and a massive loss for the left', some anecdote about a youtuber spat at a tubin' conference, Manufacturing Consent, an NYT article you 'recall' but don't link and an xkcd comic.

I don't think any of these are what the question was about.

Paul Graham: "If a journalist describes you as "embattled," they're saying you're guilty as charged. The word basically doesn't exist in spoken English. Its only use, except among castle nerds, is as a (seemingly neutral and presumably also libel-proof) press code word."


Paul Graham is entitled to his opinion. I am a professional journalist and I disagree.

Journalists use the word "embattled" the way Merriam-Webster defines it: "characterized by conflict or controversy."

Typically it applies to a person who has been heavily criticized, or accused of wrongdoing at length or by many people.

Accurately describing a person as such is not at all the same as the journalist declaring a person guilty, and it's certainly not a "press code word."

He is seeing bias that isn't there.

I think PG is getting at a larger point, which I heavily agree with, that journalists very often use words that imply a lot of "story" that is not supported by "facts". For example, I just went on to the NYTimes Economy section right now, and the second article says "Short of Workers, US Farmers Crave More Immigrants". Crave? Really? Cable news is horrendous about this. They do stuff all the time that's like, "Senator XYZ blasts Trump". Blasts? And that's just headlines. But you see it all the time in subtle lines throughout an article, such as, "Facebook's employees have been raising a stir". Or like, "The new law is leaving homeowners in a lurch". These words have no quantifiable meaning, and typically it seems journalists pick words that have more average "emotional valence" than other words, even if the facts don't quite support that level of emotional valence. And that honestly is the part that annoys me the most about normal journalism.

> I think PG is getting at a larger point, which I heavily agree with, that journalists very often use words that imply a lot of "story" that is not supported by "facts".

And he illustrates this larger point with specific claims that are, themselves, unsupported by facts and constructed around the narrative he is trying to push.

I agree, but dramatic presentation is what gets attention. A dry recitation of the facts performs poorly in the marketplace.

You're not engaging with the actual criticism at all. Nobody is disputing that "embattled" is in the dictionary; instead, he's saying that it's journalism jargon that is not actually used in real life.

No, no, no. pg is claiming that "embattled" is a journalistic euphemism for "guilty as charged". Sure, it's jargon and rarely used in "real life", but it's generally not inaccurate or deliberately misleading.

Someone who is accused yet innocent is still "embattled". Readers assuming guilt is not a problem of fact. It may not even be the journalistic intent.

For that matter, embattled is often used in a context where there's no crime at all involved. A CEO may be embattled because he hasn't been able to turn a company around and the shareholders and board are being restless. In that case, in a sense it's code for on the way out, but only because by the time you get to that point, the handwriting is probably on the wall.

I tried the word "embattled" in Google News. One of the articles that came up was about an embattled tree. Which I'm sure is every bit as guilty as the media suggests.

That, at least, is a response to the actual argument. I disagree: I have never seen a story where "embattled" was used to describe someone the rest of the story led you to believe might be unfairly embattled, and the nice thing about "embattled" is that you do not have to actually prove anything to use it. It's just "I hear lots of people are saying..." dressed up.

PG is blaming journalists for what is certainly an anecdotal interpretation of the word. There's no reason for anyone who knows the meaning of the word "embattled" to believe it's castigating.

The thrust of PG's claim was that journalists use the term as a synonym for "guilty" and in response, a journalist has argued that they actually use it in line with the dictionary definition of "characterised by conflict or controversy". A bit rich to accuse other people of "not engaging with actual criticism" when your reading comprehension falls that far short...

In my experience, journalists use it mostly for people who are likely to be fired shortly, many of whom are not "guilty" of anything other than being in a position of responsibility in an organization, department or team that is perceived as underperforming or being taken over by critics.

I actually would find embattled a bit of an unusual word choice for someone on trial for a crime outside of a broader context. If someone's arrested for murder, we'd call them a suspected murderer or alleged murderer, not someone embattled.

As you say, I associate embattled more with, say, an executive caught up in a scandal or who is likely to be dismissed for poor performance.

"alleged" or "suspected" is actually perhaps a better example of the sort of not-always-correctly-interpreted-by-the-readership journalistic phrase PG intended to highlight, since "alleged" can both mean "this person is [probably going to be] on trial for this offence, so because it's so serious we're respecting the legal process by not asserting they did it even though there's substantial or even blindingly obvious proof" or "we're just reporting something someone once claimed and because it's so open to dispute we have no desire to get sued by making the claim our own"

It would be interesting to add a journo jargon extension to this website so you can understand what certain phrases could be referring to. Kind of like "a stranger"


You'll notice that on this platform every proven fact is annotated. As a result, proven statements stand out from unproven statements. This forces the writer to be upfront about which assertions are proven, and which are speculation/opinion.

Well, yeah. And "fact checking" won't fix that.

Interesting you mention libel. Libel law, of course, is a tremendous motivation to not posting actual lies in journalism. And courts are more flexible than "fact checking".

Great project, but this seems like a classic example of a programmer naively assuming a technical solution can solve a social problem.

This is definitely worth trying, however.

I'm a bit jealous because I had nearly the same idea back when I worked in public radio, but didn't have the time to implement it. Basically it was about having facts and quotes exist as embedded units within articles, all with their own individual records of verification and relationships to all the articles in which they were used. I'm really glad someone is taking a shot at this, even though I have my doubts as to whether journalists will take it seriously.

YazIAm: If you want any further inspiration, here's my old github issue where I wrote about this concept: https://github.com/SCPR/SCPRv4/issues/718

I don't know what all the criteria are, but you should consider applying for a Knight Foundation grant: https://knightfoundation.org/apply/

Just read through some of your repo, ravenstine. Very cool! I'll be sending you an email :)

But it might be worth a try. Wikipedia does pretty well imho but if someone would told me the idea I would never thought that it would work.

Wikipedia, for a few high-profile current events each week, serves a function similar to the one proposed for sourcedfact; and it regularly results in deeper + more contextualized coverage than even the best outlets. However, WP's commitment to being a tertiary source means that _some_ secondary source has to properly interpret the primary sources, before that interpretation can be included.

Wikipedia has flaws,too. 1. Your account need to have good history or social repo for your edits to be seriously considered by Wikipedia admins. 2. Admin can have their own set of bias and can decide to ignore edits.

Remember Wikipedia is a centralized and under control of single entity, fact that they rely on donation doesn't change anything.

Just because something has flaws doesn’t mean it’s not a good idea to do it.

There wasn't anything said about Wikipedia being a bad idea, but rather that it shouldn't be your sole source for fact-checking. I've learned that people only trust Wikipedia to the point at which it confirms their biases.

Part of the problem is that Wikipedia is generally pretty darn accurate. Everybody always talks about funny vandalism stories, but generally it's pretty good. The issue of course is that there's parts of it that definitely aren't correct.

This makes it easy to say "Well if it's accurate 99% of the time then it must confirm what I thought!".

I have found that Wikipedia is pretty great, with the sole exception for anything that is remotely political. I would look elsewhere for such content.

The flow of the conversation was:

- OP made an open platform

- Someone said it could be a good idea because Wikipedia

- Someone else said that Wikipedia has flaws

Hence my comment.

This is more a communication standard than a complicated tech solution. A communication standard for fact checks from public primary sources.

Coding this wasn't actually that complicated, figuring out a design that enables readers to review the fact checking as easily as possible and enables fact checkers to collaborate easily took a lot of experimentation though. Although even that is still very much a work in progress.

Exactly. I would just appreciate footnotes in the articles I read that gave a detailed explanation of how much fact checking was done.

That way if a story needs to be expanded upon at a later date, the next journalist can dig deeper easily picking up the trail where the previous journalist left off.

I think to solve the problem, you need to do it in steps, kind of like how chained logic works. You start with something simple: "Can we verify what someone says with public record?" Yes, here's how we do it.

As one problem is solved, it starts hinting at solutions to subsequent problems. This is not a problem with a silver bullet. It's going to take a fundamental shift in how we understand and verify truth.

Agreed, it is also a question of goals. Journalists can bend "facts" to support the argument they want. Title are 99% click-bait anymore. Sure there are some people who want to present fair and balanced view points but that takes a ton of work and time. Most "journalism" these days is get likes and move on.

"These days" suggests that this is something new. Tabloids and magazines have been doing bunk journalism for decades. Now they are motivated by getting clicks to inflate ad costs instead of getting sales to inflate ad costs. The only thing that has changed is the delivery method. The reliable sources of journalism are unchanged as well. There are plenty of new online tabloids and "e-zines" but journalism still comes from the same places.

Right. Many prominent newsrooms seem to regard fact checking as an obstacle to avoid.

Still, it seems valuable to make it easy for readers to view the supporting or contradictory evidence. That might’ve stymied many notable media hit-pieces. Of course, many newsrooms wouldn’t want this (at least I can’t imagine The Guardian and it’s ilk would want to draw attention to their own impoverished fact:BS ratios), so it would have to be a browser plugin or similar.

People post contradictory evidence to journalism they don't like all the time. Their "evidence" is usually conspiracy theory crap that can't pass even logic checking, much less fact checking.

Sure, but it's not easy to find all of the debate about a given assertion in a particular article, much less contribute to that debate (given that it's probably distributed across a bunch of social media venues).

One can easily imagine hovering over a given assertion (e.g., "Damore posted an anti-diversity screed") and see the fact-checks, perhaps ranked by fact-checker's credibility in order to suppress conspiracy theory crap as well as contribute to the debate.

Anyway, that there is lots of noise among the criticism doesn't mean that there isn't a lot of valid criticism. It's only been a few months since nearly the whole American newsmedia willfully mislead the nation, sending an angry mob after innocent kids, after all.

> Many prominent newsrooms seem to regard fact checking as an obstacle to avoid.

Or they simply can't afford to employ fact checkers anymore, now that people refuse to pay for news.

> now that people refuse to pay for news

People (the audience, that is) have historically refused to pay a significant share of the cost for news, what has changed is that advertisers have stopped paying as much for news (because the share of people's attention debited to news, and therefore which news outlets have to sell access to, has dropped.)

Paid circulation used to often be important not so much because of direct revenue but because advertisers looked to it as an important measure of reach.

That's true as well. The effect is the same either way, though -- news organizations aren't exactly swimming in money these days. They have trouble just paying journalists, much less support staff for those journalists like fact-checkers.

Maybe that's part of it, but there are plenty of examples of the media deliberately misleading their audience. "Can't afford fact checkers" doesn't really explain that.

Fact checking was very effective for publications like Readers' Digest that routinely invited guest authors. The publication wanted a variety of authors to contribute, and they also had a reputation to defend.

If you've ever been interviewed by a modern journo, you know you just get the same question reworded a dozen different ways. The reason is that they decide what story they want to write up front, and then just go shopping for facts to support it. That story is usually dictated by market pressures, which presently reward sensationalism to feed the addiction of a dwindling number of listeners in an echo chambers.

And current fact checkers are typically checking material put out by others, which is completely broken. They have all the same incentives as anyone writing a hit / puff piece, except they put a phony "truth number" at the top and other sites will tally those up as though it means something.

Isn't that just setting a high bar? It might not solve an issue, but it could help.

I think there is a technical solution to this problem though. A structured graph of supporting facts (including an analog of identity for the sources and corroborators, pointing to the edges) is an excellent datastructure for maintaining a record of reasoning and evidence for a statement of fact.

There is, of course, the social issue of convincing people that this is important, but the underlying issue is in part a technical one. I don't think it's fair to disqualify a solution based on the fact that involves technical work.

I'm reminded of the oft-quoted phrase "you cannot reason someone out of a position they did not reason themselves into."

I am not trying to disqualify the solution, more like observing that the existence of provable facts has not yet been any sort of antidote for 'fake news' (whether that refers to the real deal, or the position many people take that any news they don't like is fake news).

I don't think this sort of effort has to be ironclad to be meaningful. The margins could be a big deal, here.

Some sort of attestation toolchain might enable people who care about evidence to produce more strongly-attested cases, and to get a better handle on what is and isn't sourced from others. It seems likely that using good attestation toolchains could create feedback loops for researchers, journalists, archivists, authors, film-makers, educators, students, and other people involved in knowledge production.

These feedback loops might change how they research, where they start, how often they have to re-invent wheels, and raise standards by enabling these people to compete on metrics that might drive more social value than clicks, engagement, shares, or ad revenue.

These effects could be important, even if you never budge the kind of person inclined to consume the information equivalent of junk-food.

Technical solutions solve social problems all the time. A lock is a technical solution to the social problem of theft, and much superior to a social solution.

Why does it seem like that to you? Do you assume that the project should solve the problem entirely in order for it to be of any help or have any value?

IMHO reputation is important to build into the base level for a platform like this. Allowing anon registrations and commenting will eventually lead to a lot of noise and abuse of the platform for misinformation. I think some sort of verification of a person's qualifications on particular subject areas and then allowing them to only "fact check" on those qualified areas is required here to have an effective platform that will scale well.

I'm not 100% sure that lack of specific subject matter expertise should prevent people from commenting. In many cases experts are the best people to go to, but in other cases they can represent entrenched ideas that are precisely the ones that need rethinking.

For an example, I have read writings of many experts in mental disorders that completely ignore changes in the DSM V. I do not know if the experts are not aware of the changes or don't agree with them, but the way mental disorders are classified underwent some significant changes and I can quote direct lines from the DSM V that strongly contradict the claims of experts.

> For an example, I have read writings of many experts in mental disorders that completely ignore changes in the DSM V.

Nitpick, but it's DSM 5; roman numerals were dropped after DSM IV.

> I do not know if the experts are not aware of the changes or don't agree with them

Almost certainly the latter. Lots of DSM V changes were, and are, controversial (even among people who acknowledge the existence of the problems the changes were intended to fix, the aisle solutions chosen are sometimes controversial.)

Equally important : Don't display reputation to users other than themselves. Humans will attributes higher trust level to individuals with higher reputation, despite them occasionally dropping false claims.

Does it really make sense to call it "reputation" if you're the only one that can see it?

I think so. You could probably pick something better.

Its more of a Pavlovian reinforcement than reputation. You see what got you lots of points (what people liked) and you're more apt to do that in the future.

By keeping everyone else blind to someone's scores, you don't have as much popularity bias creeping in.

I think it is better to record the facts about people's opinions, than to count reputation points on identities.

For example, as ternary predicates:

1: water is wet

2: identity_1 corroborates 1

3: 2 by_way_of blind_faith

4: identity_2 denies 1

5: 4 by_way_of blind_faith


You can go as far with this as you want, and the consumer gets to decide what establishes or hurts credibility, and which statements of fact corroborated by what methods and by whom will either count toward or against the credibility of a specific form of statement on a specific topic by a specific set of identities.

For example, you could have predicates describing conflicts of interest, and identify the relevant conflicts of interest by querying for interests in the predicate in question to find what people you trust will say about the interest of individuals and organizations in corroborating or denying a particular claim.

You could also have predicates which show ways in which a statement is controversial in its general form, but is uncontroversial when refined. For example: liquid water can not be wetted with liquid water.

> Does it really make sense to call it "reputation" if you're the only one that can see it?

Sure, if those with higher reputation have their claims promoted above others with poorer reputations.

On HN they call it Karma

You can see a user's karma on HN though, just not on their individual posts.

Isn't that the purpose of reputation? Some people are more trustworthy then others, even if they aren't perfect.

Currently (for the prototype), you can start contributing, but your reviews only become public once I've vetted you and given you a fact checking test.

Long term the goal is to have an open reputation score for every fact checker. I'm currently experimenting with different implementations of this, and seeing good promise.

What credentials do you have to properly vet people?

Well first, the point of this type of open fact checking is you can always verify everything for yourself, so you don't need to trust my credentials.

Working on this over the last few years though, I've personally fact checked over 500 assertions.

Yes, but that's not the point. That merely shows that we still need to trust someone -- you.

Specifically, let's say you support belief A, and I want to add a whole bunch of facts that support belief B. You're still the gatekeeper to decide whether I get to be a contributor. There's no way for someone to know that you're going to approve equally-qualified contributors on different sides of an issue, so they're always going to wonder whether there are voices or facts being omitted.

(But if that's just the plan for beta, that's fine.)

Indeed, manual vetting is only for alpha. I should reiterate - the point of this type of open fact checking is you can always verify everything for yourself, so you don't need to trust my credentials.

Long term every review, even those that are flagged, will be accessible. Albeit, those that are flagged will be in a separate section, but the point is they will always be accessible, so that you never have to trust me or the community, you can always review for yourself.

> I should reiterate - the point of this type of open fact checking is you can always verify everything for yourself, so you don't need to trust my credentials.

Again, not for the omission of facts, or the omission of counter-evidence.

E.g, suppose you only let in contributors who agree with you. An article is posts with "Fact A," which you happen to agree with. A bunch of your hand-picked contributors put up evidence to support "Fact A." No one ever posts the evidence debunking "Fact A."

So there's no way to "verify" anything, because we can't see what's not there. Most readers will be satisfied that "Fact A" is well-sourced.

So it's not enough to say that the reader can verify everything and not need to trust you. I mean, they could do their own research to Google the evidence debunking Fact A, but they could do that anyway and the platform doesn't support them.

I would respectfully disagree. Introduce enough noise into the system and the cost of verification becomes so high that doing so is unrealistic for many or most people.

Also, on first glance the app looks great, well-polished, and the UX of reading an article and looking at sources for claim is nice :). Thanks for working on and sharing this!

This sounds like a job where people don't get paid.

Like Wikipedia contributors?

And Wookiepedia

Just like this one site ... Wikipedia?

Disagree. The point is to link assertions to publicly available primary sources like court documents or suchlike and articulate the linkage clearly.It's well designed for that.

Because anon users won't link to "primary sources", which are really nothing of the sort? Because "primary sources" are always correct and trustworthy and never disagree?

I have no trouble maintaining such distinctions.

Wikipedia talk pages are a perfect example of facts ignored and removed from articles to push an agenda. You can see how a narrative can be pushed with half-truths and selective editing. Same thing is happening in News, its agenda driven, not truth driven. Entertainment opinion pieces that pretends to be news. People are just bending the facts to support their personal politics.

If Wikipedia can't be factually accurate due to editors personal politics, how do you keep those personal politics out of a fact checker?

You're 100% correct in that one can cite completely valid facts and still push an agenda by omitting other facts. I'm actually working on a "fact comment section" feature where any reader can pin facts to an article (facts that still go through the same rigorous and open fact checking process as everything else). Facts which readers deem are important extra context.

However, I'd argue that even without such a feature, having an open standard for facts openly verified from primary sources will add a lot of value to current public discourse.

What problem does this solve that existing reputation systems don't? When I read a New Yorker article, for example, part of what my subscription is paying for is considerable skilled labor by a group of in-house fact checkers. How does moving this work to an outside free platform improve the situation?

Moreover, how do you replicate work like calling sources to verify statements before publication?

I wrote a short piece for Wired once and had to go back and forth with their fact checker on the most picayune details. In an article about balancing online work and travel I had mentioned in passing searching under a hotel bed for an outlet, and she wanted to know what hotel it was, what country it was in, when I stayed there, etc. The process was exhaustive and not something I would go through with a bunch of Internet randos. It was also done before publication and involved a fairly high level of trust.

The goal is to capture the fact checking process and enable any reader to review it for themselves. Certainly some publications currently have an excellent fact checking process, but very few make their fact checking open and transparent.

As for "how do you replicate work like calling sources to verify statements before publication?" Good question. Currently this is only designed to open up the fact checking which involves public primary sources. I think that a platform that focuses on doing that subset of fact checking really really well can add a lot of value to the public discourse.

Congratulations on this; it looks great.

Currently this is only designed to open up the fact checking which involves public primary sources.

Limiting the scope to this is wise. It's a tractable problem and it's better journalism. The types of journalists who write "this unnamed source implied that this other unnamed source had implied to another unnamed source..." articles won't like this sort of tool, but those journalists should be encouraged to change careers.

Some might not realize that articles like that are fairly common. For an example, I went to NYT homepage and this was the first article: https://www.nytimes.com/2019/04/03/us/politics/william-barr-...

First sentence: "Some of Robert S. Mueller III’s investigators have told associates that Attorney General William P. Barr failed to adequately portray the findings of their inquiry and that they were more troubling for President Trump than Mr. Barr indicated, according to government officials and others familiar with their simmering frustrations."

This sort of crap gets swept under the rug within a couple of weeks, but we're certainly paying attention to it now.

What's the problem with that sentence? People speak to the press off the record all the time; it's part of the process. Unless the NYT doesn't actually have these sources, there's nothing wrong with it. Of course, all we have for proof is their reputation as a newspaper for integrity, and the word of any reporters involved, but reporters have often gone to jail rather than reveal anonymous sources.

>Unless the NYT doesn't actually have these sources, there's nothing wrong with it

It doesn't even have to come down to bad intentions or journalists intentionally lying about their anonymous sources, it's the fact that I have no way of reasonably knowing whether the NYT got tricked or lied to. Maybe their so-called "anonymous source" is just taking credulous journalists for a ride. I'm expected to just put blind faith in their opaque verification process and accept that anonymous sources are equally as valid as someone going on record?

Interestingly enough, the credibility (or lack thereof) of anonymous sources is an easily tractable problem that could be easily solvable with ring-signature cryptography [0] if there was the will to do so. And the trustless nature of that solution does not require journalists to be deemed the anointed arbiters of The Truth(tm). Alas, the social and cultural factors are just not there such that every government official, business leader, etc has a public crypto key.

[0]: https://people.csail.mit.edu/rivest/pubs/RST01.pdf

Crypto aside (I think it's useless nonsense in this case, but that's me), my gripe with journalism isn't the facts... it's the stenography. Some serial liar (coughcoughTrumpcough) says the sky is falling because the Sun rises in the West and the idea that it rises in the East is liberal fake news, and the media dutifully reports the quote and doesn't call them on their shit. And then the debate over whether the Sun rises in the east or the west is treated as he-said-she-said opinion by the media.

That is a real problem. Not anonymous sources or fact-checking.

1) I've never heard of Nicholas Fandos, Michael S. Schmidt, or Mark Mazzetti. I don't suspect them of anything horrible, but neither do they have "reputations" on which I can rely. Taibbi in his recent essay observed that reporters without established reputations get lower-quality anonymous tips (i.e., "lies") from insiders than better-known reporters get. In this case multiple no-name authors are actually less reputable than one no-name author would be, because we don't know which of the authors received the anonymous tip at the heart of this article.

2) The basic proposition addressed by that sentence is that "investigators are uncomfortable with Barr's summary", but the implication is that there is a problem with that summary. The rest of the article addresses the potential meaning of this "big, if true" implication but gets no closer to the question of whether that implication is true. The alleged feelings of randoms are not interesting to the average reader. If investigators or anyone else has specific knowledge of wrongdoing that should be made known to the public, they should contact reporters directly.

3) Even if the basic proposition were firmer, we're approaching it through too many layers of indirection. The reporter didn't talk to the investigators. He didn't talk to associates of the investigators. He talked rather to "government officials and others familiar", and not about what those officials and others had heard from the investigators but rather about what associates of the investigators had heard from the investigators. How the government officials and others heard about it is not specified. We're at least four layers deep here, and nothing believable is four layers deep. These reporters are under no threat of "going to jail", because even if the basic question were important no judge could decide that the reporters are close enough to the truth to justify contempt.

4) The basic question is not important. Eventually politicians of both parties will read the entire un-redacted report. The public will not, because grand jury testimony will be redacted to protect the reputations of Republican operatives whose transgressions were not judged to merit indictments while "sources and methods" will be redacted to protect Democratic FBI (and possibly NSA?) agents whose transgressions were under color of law. It's not surprising that Barr's report had a political slant. Everything in DC has a political slant. Maybe if they found the Lindbergh baby, this tottering tower of misdirection could be justified. Nobody here found the Lindbergh baby.

5) That this happens "all the time" is actually a serious problem. Every day, well-placed people insert self-interested opinions into the public discourse not as self-interested opinions but rather as journalistic fact. This has terrible effects on our society and on our military victims around the world. Soi-disant journalists should stop aiding this terrible process, and this tool could help them do that.

I've never heard of Nicholas Fandos, Michael S. Schmidt, or Mark Mazzetti. I don't suspect them of anything horrible, but neither do they have "reputations" on which I can rely.

Two of these people have multiple Pulitzers between them. Schmidt is on tv practically daily. These are not low quality tips to no-name reporters - that 'fact' you made up to fit your preferred narrative. Or at least, did not bother to fact-check!

Thanks for the update; it's really hard to figure out who has won a Pulitzer! Schmidt, in particular, seems to show up in the pictures without his name appearing at the top of the prize announcement. I would have recognized the name "Ronan Farrow". [0] Is it meaningful that the intern is the first author? Might he have written the first line in the story, which as we've established is the only claim of any substance, such as it is?

[0] https://www.pulitzer.org/winners/new-york-times-reporting-le...

I haven't heard of the journalists who wrote the article. I have, however, heard of the New York Times. I trust the NYT to vet its reporters carefully, and to review relevant facts. If someone wants to go off on "liberal media fake news", well, fuck them. They're not interested in facts anyway, and there's no point in debating such idiots.

As for "specific knowledge of wrongdoing"... well. What's "wrongdoing" here? Barr presented a summary of the Mueller Report to the public. Apparently, people in the investigation itself are unhappy with that summary and consider it deceptive (yeah, no shit, saying the Trump administration misleading people is like saying water is wet).

Yes, this is multi-layered, anonymous leakage, BUT. BUT. Is it untrue? Are investigators actually fine with the Barr summary, and the sources are lying to the journalists? Probably not. So despite the abstraction layers, the gist of the story is most likely true. And the NYT certainly doesn't have to run a story that says "Mueller is totally fine with the Barr summary" next week, so they're not going to print it unless they believe it true. So smell tests are fine here.

And the basic question is important. It's not whether Barr lied, but rather whether he intentionally misled - or more to the point, if Mueller's team believes he misled. That is relevant to the public, and to me.

>Apparently, people in the investigation itself are unhappy with that summary

But the article doesn't even say that they heard the investigators say that. At best it's hearsay and gossip of n-th degree, at worst it's outright misrepresentation of the facts.

Look at the weasel words: "according to government officials and others familiar with their simmering frustrations."

Who are government officials in this usage? And what qualifies someone as "familiar" with the matter? How many people work as a government official in some capacity and could claim some vague familiarity with the Mueller report? Does a park ranger in Wyoming who follows political news qualify? It wouldn't be technically lying to cite them as a "government official familiar with the matter", right?

Stories are leaked through cutouts like this for good reason, politically. What I'm saying is that the layers don't make it untrue, and decent journalism will catch most misrepresentation.

You say "weasel words", I say "We think you should take this with a grain of salt". They're being explicitly clear that their sources are secondhand. What's your answer? That they shouldn't run the story at all?

>What they shouldn't run the story at all?

Well, I think as you get more and more degrees removed from a primary source, the closer the article gets to being gossip and/or somebody's opinion and not factual news. Which is fine, there's a market for stuff like that. But I don't think that's very sustainable if you want to maintain a reputation as a purveyor of fact-based reporting.

There's a reason hearsay isn't generally admissable in a court of law.

> There's a reason hearsay isn't generally admissable in a court of law.

The main reasons are that courts have compulsory process to attain testimony of primary witnesses which makes it generally unnecessary, and that courts are by design intended to work via adversarial process including cross examination, and a indirect witness can't be cross examined.

Needless to say, there are lots of exceptions to hearsay exclusions, too.

It's not hard to see parallels between the two situations. Both the court and the average consumer of news are trying to figure out what's true and what isn't. A real human being who is quoted in the paper will certainly be "cross-examined" when subsequent events show him to have been completely mistaken. DC-insider ventriloquists with no names will not be cross-examined so.

So basically, you're saying that lies on the record is good journalism, but truth off the record is not.

Got it.

Wow, that's, like, meta-weasel-words. Well done.

Absolutely, a story based entirely on odious unattributable anonymous innuendo like that we see here should not be published. If the "government officials and others" want this important information to get out, they can still talk to Rep. Schiff. He will repeat anything that hypes up RussiaRussiaRussia, and he's on TV all the time. If, in fact, he isn't the "government official" to which this piece refers in the first place.

I don't say this because I support Trump or voted for him (never did, never would). I say it because this whole multiyear journalistic dumpster fire has guaranteed his reelection. TFA is all about putting out the flames, which I do support.

> What problem does this solve that existing reputation systems don't?

To turn its slogan into an analogy: the New Yorker is high quality closed source software. If you disagree with the New Yorker, you can't file a bug report.

You might say, "I wouldn't want to," either because of something on your end, or something on the New Yorker's end. Continuing the analogy, there are closed-source models, open-source-but-closed-development models, and open-source-community-driven models.

I'm typing this because the problem this solves for me is I often disagree with the news, and I would like to fact check them. Perhaps not--as you mention--by calling sources, but by citing other evidence.

Do you not think that closed source software allows bug reports? That's news to every closed source op in the world.

Every news source has bug reports too. They're called "letters to the editor".

If the New Yorker makes a factual error, you will get a lot of public attention for correcting them, and it will be a big deal.

I truly believe that post-publication fact checking is a losing battle. If you take a sampling of redaction tweets by media outlets and compare their retweet counts to the retweet counts of the original posting you'll see that that is very much the case.

So, your examples are all "reporting about court cases" and you use court documents as your citations for facts.

Court cases are as good as we have for determining facts, I suppose. (Though the error rate in court cases is quite high!) But 99.9999% of reporting isn't about court cases. Most reporting isn't about issues that will ever be litigated, and even those things that will be litigated, haven't been at the time of the reporting.

"Mitch McConnell had oatmeal for breakfast this morning, according to sources."

There, that's a fact. Now, tell me which court case I can refer to in order to verify that fact?

Fact-checking is an impossible endeavor. The essence of journalism is reporting things no one knows. That's what news is! This is completely at odds with "fact-checking". A good news story CANNOT be fact-checked by the public because if the public knew it, it wouldn't be news. There is an extremely limited genre of "news by trawling public but little-known documents", such as reporting on old court cases. Almost zero news fits into that category.

Call me a pedant, but I'd say OP is enabling "consensus checking"

I find the present day's fact checking wave is a misnomer. In a world where something is false, but sourced, most of what we call fact-checking would only reinforce mistruths.


  The ratio of circumference to diameter is exactly three to one [1][2][3]
  [1]: the Bible.
  [2]: pious scholars.
  [3]: low-tech experiments.
Would be cool to see someone think carefully about what "real" "fact checking" would be. First-hand sources? Links to instructions for replicating findings?

Consensus checking is of course also an interesting problem, albeit a pretty different one.

Offtopic, but the Bible does not actually use a value of 3 for pi:


Nice to see more work in this area.

However as others have said a reputation system for participants is absolutely key. Any platform like this has to assume it is going to be flooded by bots trying to influence opinion on behalf of corporations, foreign governments and activists.

A reputation where users gain reputation slowly (through work, by verifying sources, etc..) become more able to influence opinion, but the reputation drops quickly if they are found to be misleading people. How this actually works is harder than it seems if you want a truly robust system.

When I think of platforms like this I think of Galileo[0]. He was perceived as a troll or heretic in his time, but of course he was right. How do you create a platform that allows correct ideas to flourish even when they go against conventional wisdom?

[0] https://en.wikipedia.org/wiki/Galileo_Galilei

I 100% agree that the reputation system will be they key to scaling this. I'm currently seeing promise with my experiments with reputation scoring.

As for "How do you create a platform that allows correct ideas to flourish even when they go against conventional wisdom?" Great question, however my goal here is a bit more modest, it's to create a platform that allows facts which are provable by public primary sources to stand out from everything else. I think a platform that focuses on simply that can add a lot of value to political discourse.

The bots would be a serious issue if this ever became big enough, since they can add to each other's reputation. Reddit probably has thousands of code-hours devoted to this, and bots are still a problem.

There is also the problem that humans will probably tend to upvote "facts" that they agree with, and visa-versa. I'm not sure that reputation is the solution.

And yet "expert" verification introduces trust. I'm not sure what the solution is.

Yes it's a super tough problem, not to be underestimated by anybody working on such a platform.

If there is any easily automatable way gaining reputation, then the bots will use that, and if you can easily pass reputation too, then the rep system won't hold.

> I'm not sure what the solution is.

Ditto, and I've ben pondering this on and off for years :/

Been experimenting with building tools to make fact checking more transparent and reusable for a few years now, this is the latest iteration of that project. Feedback appreciated!

Is the primary idea for journalists to share the checks they did, or to have contributors help them check whether the things they mention are factual? The former is what I'd imagine when I hear "open source fact checking", the latter would be called "crowd-sourced fact checking", I think.

Just some feedback that might or might not be useful for your messaging, depending on your goals :)

For this to be useful at scale, you will need to not just design this for good-faith users, but also against bad-faith actors. If this catches on, it will become a target of coordinated disinformation campaigns. Have you been thinking about what to do against that?

Indeed, currently all "verified" reviewers are manually vetted.

Long term the goal is to have an open reputation score for every fact checker. I'm currently experimenting with different implementations of this, and seeing good promise.

How would the open reputation score mitigate bad-faith actors trying to exploit the system?

For example, I believe HN has a vote-ring detector, but I think there's also a decent amount of manual intervention from the mods. Amazon's reputation system is under constant attack through a variety of means (paying teams of people for fake reviews, going to the length of having real transactions on Amazon between fake-reviewer and the seller buying the review).

Perhaps I am just being nitpicky, and I have no comments on the content or intent of the site (not my industry / area) but grey/blue text on the night sky blue background with white for emphasis really threw me whilst trying to read the text on the site. It may also have to do with Linux's crappy font rendering but I found it unreasonably hard to look at.

In my experience, there are few good colours that remain readable on text, and I personally find serif fonts visually displeasing in most cases, although there are some exceptions (https://www.nytimes.com/) and even then, the light grey on white background is pretty displeasing.

I wish you all the best in your endeavour!

The goal is to enable journalists/bloggers to annotate their content with their fact checking. Allowing their readers to audit to and to contribute to the fact checking for themselves.

In other words, enabling journalists to "show their work".

Thanks for the feedback about the colours, I'll play around with it!

It doesn't look like "fact checking". It looks like "citation checking". More popular but wrong opinion will take over less popular but correct one.

Fact checking need a court like system, with judges. Contact me, if you interested. I have lot of real world experience with that.

So this differs from regular citations in that:

1) only primary sources are accepted 2) the claim and primary source must be verified by a community of fact checkers

Definitely happy to chat though, will send you an email!

Looks like you've done great work on this.

It's clean and easy to use and I don't necessarily see a huge problem with "noise" diluting the "facts".

I certainly appreciate both the goal and work you've done to accomplish it.

This is pretty awesome. Any time I open any article, the first thing I do is try to follow the trail of links/sources back to the primary sources and read that instead. If I can't find one, I don't trust the article.

I'm the exact same way, that's why I built this :)

Hey everybody, I just want to point out that I'm working on a similar project to document evidence for and against claims: https://www.wikiclaim.org

The tech is dumb (MediaWiki on Heroku), but it's been sufficient to allow me to spend several hundred hours creating probably about 200 hundred wiki pages so far.

Thanks for posting this, I really enjoyed browsing Wikiclaim and it feels to me like I've never seen anywhere on the Internet that centralises this kind of content (commonly heard claims with info for/against)...

Having said that, the usual strategy when I hear a new claim I'm not sure about is to google it; so is the Google search engine itself the competition for this site? It does try to give an "answer" at the top of search results these days. Then there is Quora.

The main advantage Wikiclaim has is that, like Wikipedia, the info can be moderated and updated, and hopefully can become an improving source of truth. Godspeed, devinplatt!

Thanks Smenis!

In time I hope that Google will surface Wikiclaim higher in its search results (whether URL or even answer box), so in my mind the competitor is also the ally :)

I don't think the strong suit of Quora (information based personal experiences) overlaps with what I'm trying to do with Wikiclaim (information from numerous reliable sources, editable by anyone), so no issues there either.

While I like how the software ties together the sourcing, I feel like the following is lacking (and I realize this is an alpha, but its things to consider)

1. Anyone who answers a question regarding whether the claim is "fully" proven by a source or whether the source is valid for the claim must or should (debatable) provide proof of their own interpretation by citing either specific instances in the source in question supporting it. For example, in the Net Neutrality article, Yatz states this: The adopted principles in this statement are at the top of page 3 in a response to does the source fully prove the claim. It would be nice if that was linked to the actual location in the document or identified through highlighting or some other mechanism to get full context for the response. There also should be uses for outside sources to be attached when evaluating these questions that are suppose to validate a source. This is because someone could submit a response to these source validity questions and cite an external source or internal quote of the source in question for their reasoning that doesn't support their answer.

2. Having some sort of expertise verification is extremely important and should be weighted either separately as in Experts: Yes Experts: No category and should be more important. It distinguishes this from random anon answers that have no training in how the source should be interpreted.

3. Having anonymous people do this seems rather dubious and the current platform seems like it would make it prone to Youtube comment syndrome. Maybe have some reputation system to gate keep like StackOverflow?

4. Sources usually require appropriate interpretation in order to be taken in the right context and be considered correct. Sometimes there's no single source or any source, philosophically, that fully proves a claim. I'm guessing that's what supporting documents are for?

5. Why is the original articles claim allowed to be edited?

This is really interesting. It reminds me a bit of peer-reviewing journalism via Web annotation: https://web.hypothes.is/

I have a couple comments to make as a former newspaper reporter:

* Many important stories use anonymous sources that are impossible to verify via crowd-sourcing or by linking to official documents. Much of what happens in the world is not publicly documented nor is it widely known (until a reporter publishes a story). That gap between events and any record of those events is one of the structural problems that allow "fake news" to occur.

* Timing matters. Reporters working for the dwindling number of publications that employ fact-checkers or editors have their facts checked before they publish. For that small group of outlets, this diminishes the number of falsehoods they are responsible for unleashing online. Fact-checking a story after its release is useful, but the cat's already out of the bag. The lie is already speeding around the world at the speed of viral outrage.

* As with any platform where better information leads people to make different choices, there are lots of incentives for publications not engage in crowd-sourced fact-checking. Why do people lie or distort the facts on their dating profiles and job applications? Why do companies put forward their most utopian face? They are trying to shape the world with information. And so are media outlets such as Fox News, to name just the most egregious.

* Fact-checking is a lot of work, and can descend into epistemological quibbles. Most people don't have time do engage in it, or even to understand the debate around a given fact. This is why, in the past, we outsourced this work to the editorial staff of the fourth estate. While this would bring more transparency to the process, I am not sure who would have time to take advantage of that transparency, anymore than the normal reader will closely follow the "talk" tab on a Wikipedia article.

I like the idea and genuinely hope that fact checking becomes more mainstream. Seriously, kudos to the author for building something. But what I'm not getting is why a news platform would use this, unless this is intending on becoming a news platform itself?

I think there is an underlying assumption that the general world cares about things like "primary sources" and "logical fallacies" enough to bother hovering over the text or viewing the content, and that there isn't an intentional manipulation of these things by media organizations to fit a narrative. Maybe if this was a browser plugin that came with <major browser> by default and automatically highlighted fact checked statements in exiting articles. That way folks wouldn't have to opt in, but as it is, whats the incentive for either the populace or the media to participate? Think of it this way- how did the fact checking in the examples get done? Someone spent 2 minutes on their favorite search engine. People willing to do that will do it if they have the time, and people not willing probably aren't interested in information that would contradict their opinions anyway - unless it comes from a source they already trust, like someone in their bubble. My own experience with this is that the folks who will care exist but are rare. Others, you can literally watch their eyes glaze over the moment you introduce a little cognitive dissonance.

How does this handle anonymous sources?

Not only do you have to account for what those sources say, but also, a reporter reporting what an anonymous source said can be 100% factual (Yes, the anonymous, valid source was indeed reported accurately), but maybe the anonymous source is incorrect. The reporting is accurate. The source is not.

Basically, a lot of grey area. And with that comes the issue of anything not "fact checked" by a service like this could be dismissed. What harm does that do?

Not a dismissal of a service like this. Rather, just open questions.

Only public primary sources are allowed as evidence on SourcedFact.

Anonymous sources certainly have value within journalism, but the goal of SourcedFact is to make facts that can be proven beyond a reasonable doubt with public primary sources stand out from everything else.

Okay. I think as long as that distinction is clear, sounds good!

The main fault I see is how do we augment your "facts" with other "facts".

For instance, I clicked on Net Neutrality. It's tough to truly appreciate the "net neutrality" dynamic of the past few years without understanding the backstory of Reed Hastings battling the ILECs (https://www.dslreports.com/shownews/Netflix-CEO-Comcast-Want...), buddying up to Obama (https://www.businessinsider.com/house-of-cards-obama-2013-12), getting the law changed in his favor (https://www.nytimes.com/2015/03/13/technology/fcc-releases-n...), and now rewarding Obama in kind: https://www.nytimes.com/2018/05/21/us/politics/barack-obama-....

It's almost as if it had nothing to with actual "net neutrality" at all since none of the horrible things have come to pass that were predicted.

Broadband isn't as good as it could or should be, but it isn't getting worse.

Interesting work, YazIAm. Conceptually this reminds me a little bit of the idea behind Kialo. Down the road, there might be some interesting synergy potential between fact attestation and debate.

I've had a few related thoughts, on the off chance you find them useful:

1. It would be interesting, variously for journalists and the reading public, if it was possible for journalists to generate, disclose, and research cryptographic sourcing identifiers that enable them to figure out when they share a source. This could, critically, help journalists identify sources with a record of feeding other journalists bad information.

It'd be nice if the same work could help the rest of us unravel citation chains based on a small number of unique sources, but I'm not sure there's a way to achieve that without it being fairly simple to re-identify sources via brute-force.

2. If we can drill enough to find some bedrock, I think it might be useful to have virtuous-cycle user tools, like browser extensions that warn users when visiting an article by writers/publications (and potentially even sources, per above) with repeated sourcing/attestation problems. I don't think of this as purely a journalistic thing. In the sciences it could apply to methodology, data-collection/statistical integrity. With some bedrock, tooling in place, and established trust/community practice in place here, it might also be possible to expand the scope a little and address things like headlines that aren't at all moored to evidence.

Would be nice to avoid having to open another tab to do fact checking - having some kind of a summary or something beyond "10 verifications" would at least let me know if opening the new tab is really worth it. Ideally the article and all verifications would live in one tab.

Also, instead of a dedicated site, a browser extension allowing you to select text then right-click to get facts would work better.


This reminds me a bit of an application I wanted to build many years ago: A platform for collections of arguments. For example, the question "are humans contributing to global warming?" Allow crowd-submitted answers. Collect information about facts cited (similar to how this works).

Mostly-rational people can be swayed by evidence and arguments, yet often disagree with each other because they doubt the truth of the other side's statements (fact checking), they believe the other side has missed important context in their argument, the other side has implicit assumptions that they disagree with (this is something that should be explored as a sub issue, then).

I'm very happy to see this. I'd love to see more of this kind of thing in the future.

Hi Joel. In case you didn't see my top level comment (it's buried quite low), I'm working on a similar project to document evidence for and against claims: https://www.wikiclaim.org

The tech is dumb (just MediaWiki on Heroku), but it's been sufficient to allow me to spend several hundred hours creating about 200 wiki pages so far.

But journalism is not only stating true facts, but choosing a limited set of facts to present.

You can pretty well do fake news by stating a list of true facts (and omitting a few other, fundamental, true facts).


Published news : person A killed person B by pushing a knife on his stomach, at a time when person B was unarmed.

Non-published statements : just before being killed, person B and a group of three friends armed with knifes attacked person A, which was unarmed at the time, and started cutting him. During the struggle, person A managed to get hold of B's knife and fought back, harming B. The friends of B were scared and ran away. Person A finally survived his injuries, but person B didn't.

I suppose everybody agrees that publishing the first statement alone is a clear example of fake news.

I have been thinking about something similar but stuck on a different project. Great work!

This is reminiscent of how some scientific papers are written. You might be able to pick up some tips from those processes as well. For example:

- Write each sentence on a different line.

This allows for better change tracking, commentary etc in a number of tools including github.

- Inline sources and compile into a references section.

I want to see that a claim is backed by more than one piece of supporting work whenever possible.

- Have a compiled / formatted version for holistic evaluation.

After you've done the hard work of writing factual statements the piece also needs to be readable. This is an orthogonal view into the same information however and should be presented as such.

This is a very good idea but I'm afraid it won't play along the human condition.

People construct reality with things they want to believe. Most people aren't interested and/or skilled enough to be reasonable and think logically.

How could someone like that orange ape become POTUS???? This is just one major indicator for the fall of reason.

The enlightenment has failed and instead of a mass of mature citizens who are able to govern the world via consensus like the democratic system needs it to be we are driven by emotions.

Keep on doing this and I hope it may still have some impact.

Little feedback : Nice. Please add a vertical splitter in the verify page so that the verification is not constrained to a small margin on the right, which make it not easily readable.

It would also be nice to have some color coding when reading articles, so you can know what's the majority opinion on the validity of the piece.

Hopefully once you have gathered enough data, you can get one of these NLP bots to check the fact in an even more neutral way.

How are you going to deal with underlying "sources" where the content disappears or is changed out from under the fact checkers?

Every source document submitted to the platform is automatically saved to the webarchive, and the webarchive snapshot is used :)

Definitely have a look at https://tlsnotary.org/ !

Just a couple quick feedback points:

I'd prefer if the currently selected fact text remained highlighted when the mouse is not hovered over it. That way you know what the active "Investigate >" link is referring to.

Also, for accessibility and convenience, it would be nice if you could tab through the instances of the fact text blocks.

Thanks for the feedback, the second (accessibility) should be a quick fix.

I definitely also want to address the first issue, the tricky part is coming up with a solution that also works when there are multiple assertions on one line, but I'll give it more thought.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact