
What if journalists had story writing tools as powerful as those used by coders? - danso
http://pudo.org/blog/2014/12/03/newsclipse.html
======
jawns
When I was working as a web editor at a metro daily paper a few years ago, I
proposed something similar: an XML-like syntax that would allow for metadata
to be included in drafts of news stories, some (but not all) of which could be
made use of in online versions of the story (such as a link to a map when
you're referencing a location).

A lot of wire copy already includes metadata, but it's generally just in a
header that accompanies the story.

What I was envisioning was something more like what is being proposed for the
semantic web:

<name id="1394">John Smith</name> was elected president of the <organization
id="2315">New Castle County Council</organization> on <date
value="2014-12-10">Wednesday</date> at the <place lat="39.685881"
long="-75.613047">county headquarters</place>.<source id="23" name="Mila
Jones" title="New Castle County public relations officer"></source>

I also wanted to use the metadata to help copy editors trim wire stories:

<priority value="1">This amounts to de facto resegregation. <priority="4">(And
we all know how we segregation worked out the first time.)</priority> If the
school district still values integrated schools, it must act swiftly to
correct this effect.</priority>

It turns out, though, even when you create a UI that lets reporters and
editors easily plug in this metadata without having to understand XML, they
are not apt to fill it in, because they are just so overworked as it is.

Plus, in order for this to work on a larger scale, you'd have to get an
incredible amount of buy-in. You'd have to get reporters and editors to agree
that it's worth their time. You'd have to build software to support it. You'd
have to get all of the different media companies out there to agree on
standards.

It's just ... not what the media industry is (or should be) focused on right
now. They've got bigger things to think about, like how to find a viable
business model.

~~~
schnevets

      <priority value="1">This amounts to de facto resegregation. <priority="4">(And we all know how we segregation worked out the first time.)</priority> If the school district still values integrated schools, it must act swiftly to correct this effect.</priority>
    

Forget about editors, use this kind of mark-up for your readers. Imagine
changing an in-depth article into a truncated, 200-word summary with the click
of a button. Activating a different tag would include the reporter's
subjective commentary (or perhaps have multiple editorials based on the same
"scaffolding")?

~~~
kd5bjo
In print journalism, you're supposed to include information in descending
order of importance. As a reader, you're expected to stop reading once you get
to details you don't really care about, confident that you won't miss
something more important buried farther down.

~~~
dredmorbius
The so-called "inverted pyramid". I'm finding it applied less and less
frequently.

I've also noted that it's become pretty much standard practice for news
bureaus to write single-sentence paragraphs. Literally, every sentence of a
story is its own paragraph. I don't know when that became standard practice,
but sometime in the later 1990s or 2000s, particularly as stories moved
online.

~~~
mpclark
Sadly, hardly anyone writing now outside big traditional media orgs has done
any sort of formal journalism training.

~~~
dredmorbius
That's ... not _all_ bad.

I think there's too much insularity within much of the journo community. But
many of the newcomers are also subject to outside influences which call their
credibility strongly into question. Lack of uniform copyediting, for better or
worse, means a wide range of writing quality.

Though I'm seeing that even in long-standing brands -- NY Times, Forbes, and
elsewhere.

------
Alex3917
I've been messing around with something similar, specifically though focused
on tracking strengths and weaknesses in US infrastructure:

[http://www.alexkrupp.com/Citevault.html](http://www.alexkrupp.com/Citevault.html)

It's actually a lot easier than you'd think, because thanks to Zipf's law
pretty much every article in these evergreen topic areas is using the same set
of 1,000 or so facts. And these facts mostly come from the same set of
government or NGE reports, which are updated at most once a year, and often
only once every ten years.

The cool thing is that you can then use a javascript snippet to track which
facts are being used in which documents, automatically mark facts as outdated
when they change, etc.

~~~
mnarayan01
Bookmarked. Editing nit in the FAQ.

    
    
      C.f. comments on Peter Thiel's graph of the year...
    

Should be "Cf." \-- also I believe "cf." should be read as "compare", so
really it shouldn't even be that (see
[http://en.wikipedia.org/wiki/Cf.](http://en.wikipedia.org/wiki/Cf.))

~~~
indubitably
aaaaaaand no one cares.

~~~
pestaa
I didn't know about the English equivalent, so I did care.

------
Hansi
When I wrote my disertation I used Scrivener:
[http://www.literatureandlatte.com/scrivener.php](http://www.literatureandlatte.com/scrivener.php)
which is a wonderful tool for writing. An expanded open source application
similar to that with a good plugin system might be very useful.

~~~
egypturnash
My first thought upon seeing the article title was "How is this going to be
different from Scrivener", which is what a huge chunk of the pro writers I
know seem to use.

------
tezza
Most journalists are barely informed cut'n'paste merchants.

Journalists value the sensational over the factual, and work hard (true) to
tight deadlines.

So they _already_ do not care about the tech features promised, namely
indicating:

* there is not enough evidence to make a given point

* a certain person or company has not been investigated thoroughly enough

* a certain point is not relevant

[edit: data points]

Cut'n'paste obituary from Wikipedia:
[http://www.theregister.co.uk/2007/10/03/wikipedia_obituary_c...](http://www.theregister.co.uk/2007/10/03/wikipedia_obituary_cut_and_paste/)

"Hack": A self referential term journalists use for each other in the UK
[http://en.wikipedia.org/wiki/Hack_writer](http://en.wikipedia.org/wiki/Hack_writer)

~~~
hoobert
Hard to see how this apparent stereotyping (a) wouldn't apply to people
working in nearly every industry and (b) is relevant to this tool.

~~~
zo1
I guess it's relevant because the OP is saying that they wouldn't use it
because they think that facts are irrelevant.

Definitely, we'd have to look at the studies in this space, namely whether
journalists cite their facts, or write their pieces using citable
studies/facts, etc. So I also won't accept the OP's statement of it as fact.

Though, in my, supposedly biased, opinion I'd say journalists are quite adept
at twisting facts to suit their points, and also of omitting (i.e. cherry
picking) facts that support their views/points.

~~~
hoobert
Of course. I just hope you recognize comments like these do precisely the
thing they lament: make unsourced assertions designed to advance a point of
view.

"Journalism" is a broad tent. I suspect there's plenty of room for tools like
this one.

------
pavlov
It's true that coders have lots of great tools for working with textual
representations of programs... But IMHO programmers' tools are stuck in a
certain kind of local maximum due to the difficulty of moving beyond text.
We've done all we can to make textual programs easier to manipulate, but there
are fundamental difficulties that can't be solved this way.

I'm personally interested in this question: What if coders had design tools as
powerful as those used by architects and construction engineers?

I wrote a blog post about this recently:
[http://blog.neonto.com/?p=44](http://blog.neonto.com/?p=44)

~~~
TallGuyShort
I believe that's the idea behind this:
[http://www.mathworks.com/products/simulink/](http://www.mathworks.com/products/simulink/).
I know a software engineer in the aerospace field that loves this product and
describes it the way you do (except that it's more for embedded systems than
for user experiences)

------
declan
I worked full-time as a journalist for a bunch of different news organizations
before leaving to found the forthcoming
[http://recent.io/](http://recent.io/).

My suspicion is that neither I nor other journalists I know would use this
newsclip.se tool. In a single newsroom I've seen people using Word, TextEdit,
Google Docs, Notepad, BBEdit, Gmail, phone-based email clients, phone
dictation, and even emacs (me, a few times) to write news articles. While the
CMS is newsroom-wide, the writing and editing processes tend to be very
personal. Journalists also tend to be individualistic and dislike being forced
to use a standardized system without clear benefits.

Where something like newsclip.se _might_ be beneficial would be as a kind of
preprocessor/lint for the CMS. It could do a lot of what's being described in
the linked article but without replacing the entire journalistic stack.

------
cryoshon
Making a tool like this has occurred to me, but my idea had a slightly
different focus.

Rather than attempt to link evidence to statements, which, if we're being
honest, doesn't really bring us closer to the truth since many "sources" are
merely other people's words, it was much simpler: identify weasel words,
euphemisms, or use of the passive voice. Between these three features, I think
that most factual writing could be improved a colossal amount.

I would certainly appreciate an easier way to keep track of claims, citations,
evidence, and interplay between a story's moving parts, though. I think that
the article mentions a few tools which work toward that goal admirably. Right
now, to make a really bulletproof piece, you need to be extremely scrupulous
with self-identifying your claims and then providing written explication of
evidence or hyperlinked evidence.

Additionally, it'd be really useful to have a tool which kept track of
sentence structure and also allowed you to track logical rhetoric by keeping
track of "If this, then that" style statements.

This is probably asking too much, though. A final hurdle is that journalists
and writers tend to be old-school when it comes to technology, so it might be
a hard sell to the older segment of that market.

~~~
lstamour
Some of this (the identification of words, passive voice) exists already,
though results vary:
[https://www.goodreads.com/topic/show/1094904-proofreading-
so...](https://www.goodreads.com/topic/show/1094904-proofreading-software-i-
want-to-believe#comment_83082658) (I link to this more because it's a
comprehensive list of possible software even though the comparison was done by
a vendor.)

------
LukaAl
What do you mean for "writing tools as powerful as those used by coders"? In
my experience good coders doesn't need very powerful tool. Super-complex IDE
are usually a disturbance to good coders rather then an help and even the
uber-geeks that use Emacs don't really use in their jobs all the macros.

A lot of good software engineer I know prefer lightweight editor like the good
old Vim or SublimeText over IDE like Visual Studio, Xcode or Eclipse. That's
because when you write complex code your brain is slower than your finger and
the syntax and structure of the language you are using is something under your
skin, you don't need to think at it.

So, no, great coder doesn't use powerful editors, if they can chose they use
very simple one with the minimum level of features they need to help them
think faster (syntax highlighting, autocompletion, indention) and stop. All
other features of complex IDE are a disturbance rather than an help.

I don't know what a good journalist need but my guess is that they have the
same problem, their brain is slower than their finger and the grammar of the
language they are using should be well known. I don't know which tool they
use, but in my opinion a simple text editor with a minimal spellchecking
system (but not correction) well integrated with the publishing process is
more than enough.

From an Innovation point of view I'm usually suspect when someone claim to
have an innovative tool for an established industry. If the industry is
established the actors have already optimized their tools and if there could
be an improvement is very small. What usually happens is that something
changes in the industry and make the actual tool obsolete (e.g: a new process
or a new technology became available). But if you want to create something
really innovative, you have to find this change. Simply bringing something
existing in a field in a different field rarely works. And if it works is
because you have a really good understanding of both fields.

~~~
zo1
> _" So, no, great coder doesn't use powerful editors, if they can chose they
> use very simple one with the minimum level of features they need to help
> them think faster (syntax highlighting, autocompletion, indention) and stop.
> All other features of complex IDE are a disturbance rather than an help."_

Says you, and your anecdotal evidence/experience. Sounds to me that you have
more prejudice against "complex" IDEs than you do actual facts. Would
countering your points by telling you of the countless bad "coders" I've
worked with that use text-editors instead of IDEs convince you to change your
mind? (I'm going to stop using your term 'coders' here) Or the absolutely
great programmers that I've seen that use powerful IDEs to supplement and
augment their abilities?

Do you write in a compiled language that can provide rich intellisense, or
something more like Ruby/Python/bash that is a tad lacking in that department?

~~~
mkehrt
Unrelated to the actual point, but I'm curious. You seem to be loathe to use
the word "coder". Why is that? How is it different from programmer? Is there
some sort of cultural shibboleth I'm unaware of here?

~~~
zo1
I don't think it's a broader cultural thing, or anything. And personally, I'm
fine with the term. It's just there was something about the way the OP was
using the word "coder" that didn't quite sit well with me. Reading back at his
post, I can't quite put a finger on what it was that actually bothered me
about it.

------
Fede_V
My distinct impression is that the limiting factor in quality journalism is
gum-shoe reporting, not incredibly powerful CMS.

Buzzfeed has the most advanced CMS, but their reporting pales with respect to
the NYT. When they do bother to do proper reporting (McKay Coppins, etc) they
get excellent stories - the rest of the time they use filler, because it's
cheap to produce.

~~~
morgante
I actually think technology can improve the quality of journalism. Not
necessarily by making great stories better (though the NYT is definitely
proving that's possible with some of their interactives) but by making
reporting in general cheaper. If a great CMS can make it possible to produce
all the economically required filler in half a day, that leaves time to do
actual investigative reporting later.

Buzzfeed is definitely pushing this a little bit, though I do think they have
a bit too much baggage to get far enough with it.

(PS. If anyone wants to help tackle this problem, we're hiring @ Cafe:
[http://cafe.com/careers](http://cafe.com/careers))

~~~
Fede_V
I'm sure technology can improve things - for example, the NYT is doing a much
better job now of marketing its (excellent) cooking section.

My impression though, is that what makes great journalism is reporting. Almost
all the time, when I'm reading a great article, I do so using readability to
strip away all the crud and just end up with the text in a decent size and no
pictures/hyperlinks/movies.

~~~
morgante
> My impression though, is that what makes great journalism is reporting.

Yup. At the end of the day, I think in-depth textual reporting does rule. But
what tech can do is make that great reporting slightly cheaper (and thus more
plentiful/viable).

------
aethertap
This is a great idea. So much so that I have "Why didn't I think of that?"
syndrome.

Here are a couple of suggestions that I could use if you're looking for
feature requests. Most of these things exist in one place or another, but
having them integrated into a one-tool workflow would be awesome.

1\. Some kind of crowdsourced reputation system for sources (i.e. medical
journal sites have high reputation, naturalnews.com has low)

2\. Auto cross-referencing between articles based on content.

3\. TODO list management

4\. License-aware relevant image suggester (please!!!) This alone would be a
killer feature for me. Pick out topic words and search selected image sites,
then give me thumbnails to choose from.

~~~
patrics123
Yes please! Anybody needs such a thing for blogging anyway! > 4\. License-
aware relevant image suggester (please!!!)

~~~
jseliger
_Yes please! Anybody needs such a thing for blogging anyway!_

I wrote this in another comment, but a tool like this already exists:
[http://www.stevenberlinjohnson.com/movabletype/archives/0002...](http://www.stevenberlinjohnson.com/movabletype/archives/000230.html)
and has for about a decade. I drop all my blog posts in Devonthink Pro, and
I'm often surprised by the connections that emerge.

~~~
aethertap
Thank you for this. I don't use a Mac, but sometimes tools like this make me
wish I did.

~~~
mjklin
Similar tools for Windows:

[http://scan.sourceforge.net/](http://scan.sourceforge.net/)

[http://dtsearch.com/](http://dtsearch.com/)

------
phkahler
I've always wondered why writers don't use things like revision control and
decent diff tools. I'm not sure the existing tools are well suited to them
(yet).

~~~
danielweber
Microsoft Word has had revision control for at least 15 years, probably even
longer. It's not like the document creation world doesn't have exposure to
these ideas.

~~~
scholia
Microsoft Word has a shedload of features that journalists can't be bothered
learn and/or use, and that even the author appears not to know about, judging
by his reference to "replacing simple text processors like Microsoft Word".

------
morgante
This is a very cool prototype, and I especially like the idea of calling it a
"journalism IDE" (as opposed to a CMS).

At work ([http://cafe.com](http://cafe.com)), I've actually been working on
something similar with our CMS (Monsoon). We're trying to use technology to
make telling cohesive online narratives a lot easier.

Interestingly, one of the biggest hurdles so far has been in decomposing
stories. Media traditionally treats each story as a big blob of text (in most
cases, HTML), but we're trying to change that so that each story is actually
just an arrangement of smaller tidbits (we call them droplets). Switching to
that model helps us to encode a lot more semantic information, and also to
reflow stories effectively for context.

We're not yet to the point where we integrate/suggest droplets from other
stories automatically, but that's definitely the goal. Maybe we could
integrate something like Newsclip.se to encourage that.

(PS. If you want to help us get there, we're hiring:
[http://cafe.com/careers](http://cafe.com/careers))

~~~
div
I agree that it's a cool idea ! It may be nitpicky, but I had the opposite
reaction to calling it a journalism IDE.

It just seems as if the intended audience wouldn't have the faintest idea what
an IDE is.

~~~
morgante
So, I think it's a question of who you're marketing to. I'd never call this an
IDE to the journalists themselves, but calling it a "journalism IDE" is a
great way to recruit developers/technologists—which is actually a big
challenge (in my experience, most great developers don't want to touch
anything close to content, probably due to latent association with the
abomination that is WordPress).

------
pudo
Hi all, post author here! We're working on the code base and would love
everybody's input here:
[https://github.com/pudo/tmi](https://github.com/pudo/tmi)

~~~
barkingcat
Hi pudo,

What is the relationship btw the version of newsclipse that's running as a
demo online and the repo:

[https://github.com/Canvas-Hackathon-
Teams/Newsclipse](https://github.com/Canvas-Hackathon-Teams/Newsclipse)

It seems to be the same - is that where I get a copy to run locally?

~~~
pudo
I've just decided to move the code base to run against a SQL backend, rather
than mongo & begun to implement user permissions etc.

------
couchand
This somewhat plays into Rebecca Parsons' talk [0] at hack.summit() last week.
She emphasized that we've done a great job building tools for other technical
folks, but we've really dropped the ball on supporting non-technical fields.
Her perspective was specifically on the potential of DSLs, but the call to
developers to be more relevant is likewise compelling.

[0]:
[https://www.youtube.com/watch?v=3KO2YmU5RBY](https://www.youtube.com/watch?v=3KO2YmU5RBY)

------
coldtea
> _What if journalists had story writing tools as powerful as those used by
> coders?_

Not much would change, except more fluff.

What journalists need is a money making, sustainable outlet.

------
spypsy
It's funny, I was thinking about that while writing my dissertation.

After a year of coding on Visual Studio, during writing I would say something
like "As mentioned in previous chapters, the decision was based on..." and
then just click on "previous chapters" and press F12, expecting to be taken to
the original reference.

Unfortunately I found out Word does not offer such functionality just yet

~~~
JetSpiegel
Word can use "above" or "below" in cross-references, but I wouldn't count on
it.

------
dredmorbius
There's a standard called hNews, created by Jonathan Malek (Associated Press),
Stuart Myles (Associated Press), Martin Moore (Media Standards Trust), and
Todd B. Martin (Associated Press), which addresses this:

 _hNews is a microformat for news content. hNews extends hAtom, introducing a
number of fields that more completely describe a journalistic work. hNews also
introduces another data format, rel-principles, a format that describes the
journalistic principles upheld by the journalist or news organization that has
published the news item. hNews will be one of several open standards._

[http://microformats.org/wiki/hnews](http://microformats.org/wiki/hnews)

I learned of it (yesterday) re-reading Readability's Article Publishing
Guidelines:

[https://readability.com/developers/guidelines](https://readability.com/developers/guidelines)

hNews is an XHTML microformat -- the tags are entity names and classes added
to standard HTML entities.

Key among them are: hnews, hentry, source-org, dateline, geo, item-license,
principles.

Other microformats, including hCard, can be used to identify people,
companies, and organizations, similar to vCard properties. The elements will
be familiar to those who've worked with address, Active Directory, or LDAP
data: fn, n, nickname, org, email, tel, adr, and more.

[http://microformats.org/wiki/hcard](http://microformats.org/wiki/hcard)

An IDE that tied into these (or, possibly, other standards) could be useful.

You'd likely need some sort of natural-language processing in the copyediting
or publishing process to apply this uniformly. Field reporters may work on a
wide range of equipment and software (including pen-and-paper or simply
voiced-in reports). And expecting reporters to incorporate tags into their
copy is likely a stretch.

------
taylorbuley
The developer in me likes this very much. I mean, Taxonomies! Unfortunately
many non-journalists tend to romanticize and over-complicate the craft.
Journalists care mostly about the story telling, and it's not clear to me how
this translates into a better story. I believe this to be a (beautiful and
well-intentioned) example of overengineering a domain model that, at it's most
basic, involves just titles, deks and content such as text, images and video.
To me it does not clearly push journalism toward a more profitable and
sustainable future and thus, is a distraction from more challenging problems
at hand.

------
rrggrr
[http://www.literatureandlatte.com/scrivener.php](http://www.literatureandlatte.com/scrivener.php)

Scrivener comes closer than anything else I've seen to meeting the OP's goal.

------
davidgerard
This is vaguely reminiscent of some of the professional tools for
scriptwriters and novelists, which do in fact start to resemble IDEs: you can
shuffle around characters, outlines, plot fragments ...

~~~
hristov
So that is why tv and movies nowadays are so bad.

~~~
navait
Let's be completely honest: there was very little good television on until
1999. The only thing that's changed is the volume of tv being produced(which
naturally favors cheap reality shows). But more and more great things are
being produced. Films are cheaper to make than ever before.

Most shows and film will be mediocre. But you don't have to watch the dregs -
you only have to watch the best of the best! And by the measure of good things
being produced, media has never been better.

~~~
girvo
_> But you don't have to watch the dregs - you only have to watch the best of
the best_

This is my tactic. I have a television, but it's not actually connected to an
aerial or a cable box, just my Apple TV. I only watch television shows when
it's clear they're excellent and exactly to my tastes, and it's brilliant :)

------
Spooky23
We had some Github religious adherents who made a lot of noise about this. The
net in our case ended up being "You can eliminate complex stuff like Word and
just use markdown & Github!" (I almost spit out my coffee laughing)

I don't think that an IDE model makes sense because writing in human language
is more complex and nuanced than programming languages, which are more limited
in scope. At the end of the day, IDEs are shims to match human desires to the
language the machine expects.

------
jseliger
Great article, and great idea.

I'm not a journalist but I do a lot of writing, and I actually use a program
called Devonthink Pro in the manner described here by Steven Berlin Johnson:
[http://www.stevenberlinjohnson.com/2005/02/devonthink_cont.h...](http://www.stevenberlinjohnson.com/2005/02/devonthink_cont.html)
. It's surprisingly close to the tools Lindenberg is describing.

------
mvc
Interesting fact. The Django web framework was initially created by the dev
team at the "Lawrence Journal", a newspaper in Kansas.

------
heyts
This article, incidentally by one of the creators of the Django Framework,
still seems relevant, even 8 years later:

[http://www.holovaty.com/writing/fundamental-
change/](http://www.holovaty.com/writing/fundamental-change/)

~~~
pestaa
Trivia: Django was originally made for the publishing industry.

------
vonnik
Newsclip.se is a great idea.

As a former journalist, I think that [http://hypothes.is](http://hypothes.is)
will become another very important tools for reporters... It's an annotation
layer for the Web.

------
rtpg
reading this I feel like it would totally be possible to use CRM software like
Salesforce to manage investigating a story. Leads and the like, it's almost
like the workflow is the same.

------
iondream
I writing enough like coding to make this worthwhile?

------
milkers
I want IDE logic in every possible area, i.e. browsing experience.

------
chrismcb
We implemented something like this in Word 15 years ago

------
jtth
It's called Tinderbox and it's awesome.

------
dash2
Hmm...

* Your weekend newspaper would come out on Wednesday.

* There would be new editions many times daily... not with new stories, just corrections of the first edition, which was blatantly inaccurate and partly written in Moldovan.

* Every day, the newspaper would be in a different format which didn't fit the newspaper rack you just bought.

* Every week, the newspaper would get bigger, but contain no more content (just a new font). You would regularly be forced to buy a new newspaper rack.

* Also, once a week the paper boy would break into your house and steal your old papers. He would offer to sell them back for you in the new format, for a higher price.

* Also, the newspaper rack sellers would not let you store newspapers of which they disapproved.

* Rather than telling you about the world, the paper would track your behaviour and tell the world about you.

* Once a week, the front page would be 404 NEWS NOT FOUND.

* Reporters would be paid high six-figure salaries, but would be unable to relate or talk to anyone but other reporters.

* Many journalists would consider themselves brilliant, world-changing geniuses, with plans not just to report on government, but to replace it.

* At the same time, they would have secret deals with those governments to report people who read "subversive" news.

* et cetera...

~~~
lucian1900
Nitpick: I don't think Moldovan is a language. If you mean the language spoken
in the Republic of Moldova, that's just Romanian.

~~~
a-priori
Believe it or not, this topic has come up on Hacker News before. To paraphrase
my comment from then
([https://news.ycombinator.com/item?id=7324897](https://news.ycombinator.com/item?id=7324897)):
while Moldovans speak a language that's very similar to Romanian, in a 2004
census 60% self-identified their primary language as 'Moldovan', as compared
to 16.5% who say they speak 'Romanian'.

As linguists say, a language is a dialect with an army and a navy. Whether
'Moldovan' is its own language or simply a regional dialect of 'Romanian' is
entirely a question of politics.

~~~
devindotcom
I learned something interesting today from both you guys.

~~~
a-priori
Glad to help!

