"Another challenge was building an AI that knows the difference between a fact and an opinion so that we do not flag opinions as false, since only facts can be false."
In news media that is typically considered credible, opinions are very often presented as fact, often marked with "an unnamed official said" or "our anonymous source told us."
Much of the news regarding the outside world is gathered at the State Department Press Briefings, where the Press Secretary will him/herself admit that what's being communicated at the stand is the opinion of the US government, after it goes through an interagency research, vetting, and message coordination phases.
Very often these coordinated opinions are then presented as fact at CNN, the New York Times, the Wall Street Journal and others.
Of course, this relationship led Americans to think that Saddam Housein had something to do with 9/11 and justified the invasion of Iraq, led the media to report on Mass Surveillance with the opinion label "Bulk Collection", led to inaccurate coverage of Bahrain, Kosovo, Benghazi, Jessica Lynch, etc. President Obama's National Security Advisor publicly bragged about the ability of coordination within the interagency to create an echo chamber in the US press.
I don't think there is an algorithm that can tell the difference between fact an opinion, unless somehow that algorithm is an automated investigative reporter.
It seems insurmountable at this stage of technology to try to separate US government propaganda or propaganda from interest groups from facts.
So, you're telling me that this news story about a fake news story detector is in fact fake? Shocking. I wonder how the detector would do against the article?
Seriously though. I agree with all the above, and also as soon as we have something resembling a fake news story detector, it will be gamed. Like SEO games search engines. It's inevitable.
Somehow we need to teach people how to spot the bullshit and not propagate.
> Somehow we need to teach people how to spot the bullshit and not propagate
those two things are in contradiction!
It uses some Microsoft Project Oxford APIs to do all the image work. There's an "adult image" test API, and an OCR API.
I'd say that piggybacking this service would match the goal of the project, and the other stuff is for image macros/memes etc.
Actually pretty decent for a small project like this, but the title definitely makes it sound like it's looking at arbitrary text and evaluating it for truthfulness. Really it's just seeing if the source is reliable.
Not sure about that.
Does adult content make a source unreliable? Remember this?  It was flagged for adult content on fb, and there was a lot of controversy about how it shouldnt have been removed.
It also checks if there are tweets referenced in the article, and if so, checks to see it actually exists on twitter. What about a hypothetical story about someone saying something controversial on twitter, then deleting it? Is that news? Because that story would be marked as fake since the tweet no longer exists.
Then there is this story. The article "makes it sound like it's looking at arbitrary text and evaluating it for truthfulness". So would this be marked as fake news? Does the wapo domain get added to a blacklist?
“He who controls the past controls the future. He who controls the present controls the past.”
“Until they became conscious they will never rebel, and until after they have rebelled they cannot become conscious.”
“The Party seeks power entirely for its own sake. We are not interested in the good of others; we are interested solely in power, pure power.”
“Power is in tearing human minds to pieces and putting them together again in new shapes of your own choosing.”
But seriously though, this "interagency" you speak of sounds a whole lot like State Propaganda Machine.
"That's not the way the world really works anymore. We're an empire now, and when we act, we create our own reality. And while you're studying that reality -- judiciously, as you will -- we'll act again, creating other new realities, which you can study too, and that's how things will sort out. We're history's actors . . . and you, all of you, will be left to just study what we do."
 "later acknowledged to have been Karl Rove" http://www.hermes-press.com/suskind.htm
Actually, the interagency is a catchall term to mean communication between different parts of the executive branch, as they otherwise report up to different Secretaries and command structures.
There are a number of different manners and ways that interagency coordination and collaboration is done and it is not specific to public messaging.
'Sources' are an important part of journalism, and they are controlled.
If anyone from a respected news entity quotes such a source, then it's almost surely true.
Now - that source may be simply presenting an opinion - and not a fact - nevertheless, it's almost assuredly true.
A journo that 'makes up' sources would be fired and black-balled, and also, there is editorial oversight on that.
CNN, NYT, Fox - they don't go around inventing sources.
Now - applying 'spin' or 'narrative' or 'too much opinion' - yes, this is indeed a problem. And they can do it by selectively quoting Twitter or 'some source'.
But the 'source' itself is not a fabrication.
Can't tell if you're joking or not. If you are being sincere, I'd like to live in the world you describe, but I don't.
Let's imagine for a second that those in charge of editorial oversight have been corrupted, who is going to be a reliable fact checker at that point? There's a great deal of evidence of collusion between the mainstream media and powerful groups with an agenda to push (corporations, governments, lobbyists, etc...). Would you like me to show you or do you know this already?
I didn't mean to imply that it was - I meant to imply that the journos will credibly pass on what the anon sources say, and credibly pass in their status.
So - if NYT says 'a source within the white house said ABC' - then you can be almost assure that 'a source within the white house said that'.
I agree - it doesn't mean that the statement itself is factual - but it is only presented as 'what someone said' - not as a fact.
The 'bias' in reporting is not from the statements themselves - but the topics they chose to engage, the people they speak to, the selection of the quotes, the context etc..
Credible news outlets don't just lie and make stuff up. They do other things, but not that.
I do however view "selection" as an act omission. And I believe in the context of presenting information citizens have a common interest in, omission is a lie.
A key selection in the coverage of the election was to avoid disclosing the relative sizes of the crowds at the campaign rallies of the respective candidates. Accurate information may have spurred some of the 46% of people who didn't vote to go to the polls. I'm strictly independent, but I believe more people voting is always better.
I also believe a lot of bias comes in with the analysis of facts or statements, which can lead to omission of important perspectives. And I do believe often times, these perspectives are shaped by the needs of the network's advertisers.
It's basically impossible not to contextualize, and to provide information selectively.
That's the inherent and systematic problem with managing information.
They have no choice but to do it.
So there are a lot of 'editorial rules' and 'best practices' in the trade to try to ensure fairness. It doesn't always work, and it's not always applied.
They don't have to, the sources invent themselves.
I dont know the solution but it might involve links along with news items that are contentious or opinionated, based on input from people, and I can't even begin to imagine how to make it fair.
It's a hornets nest of opposing opinions, so whatever they do will likely be in gradual steps forward.