
How I Judge the Quality of Documentation - craigkerstiens
http://ericholscher.com/blog/2014/feb/27/how-i-judge-documentation-quality/
======
btbuildem
Next time you come across a project documented solely in Lithuanian / Japanese
/ Swahili, I hope you will refer to
[http://ericholscher.com/blog/2014/feb/27/how-i-judge-
documen...](http://ericholscher.com/blog/2014/feb/27/how-i-judge-
documentation-quality/#language) for insight.

English is the de-facto language of the Internet (let's not discuss here how
damaging that is to English as a language in general). I strongly believe that
public-facing software projects should be documented in English first and
foremost, with translations to other languages added as resources allow.

I work for a small company in Quebec (that's the Canadian province that has a
Language Police) - despite most of the team being Francophone, all our
internal documentation and code docs are in English. Why would that be?
Because everyone realizes, without ever really having discussed it, that it is
the only efficient way to do it, if we want to collaborate with others, if we
want to grow, if we want others to contribute, having everything in French
would be creating unnecessary hurdles.

~~~
thoward37
I have to say that I completely disagree with this perspective. Yes, currently
our 'lingua franca' of business and technology is English, but I feel that is
a problem, not a solution.

It's a problem because it unfairly advantages native English speaking people
in business. It creates a barrier of entry to people who did not learn English
from an early age, along with their native language. It makes it much harder
for those people to be part of business, technology, and open source. It
degrades the quality of communication and work-output by forcing non-native
speakers to express themselves in English.

Does being a good engineer imply being good at spoken natural languages? I
have met some brilliant engineers in China who are barely functional in
English. If you were to interact with them purely in English, you would think
you're talking to a 5 year old, based on the language alone. Some of the
engineers I've met in China aren't even that capable with English, but are
wonderful developers writing interesting things.

Sometimes they created documentation in Chinese, sometimes in English. Guess
which one they were able to express their ideas in more clearly? Chinese.

When you write, you should write in your native language, and express your
idea to the best of your ability. Then, if you want to translate that to other
languages later, to make it more accessible to others, do that as a separate
task. Maybe hire a professional technical translator to do that work.

You gave this example of coming across a page written in Lithuanian or
Japanese, presuming of course that you're not a native speaker of those
languages. Fine, that's exactly how every Chinese native speaker feels when
they come across a page of English text, every day of their life as a
developer... but they figure it out, even with terrible language skills. You
know why? Not because "English is the language of business" but because they
just want to learn about the content, and aren't going to try to force you to
write it in Chinese so they can do that. If all those pages were in French
they would figure that out too.

With the attitude you expressed above, the main people who benefit from this
are native English speakers. Anyone else is still in the 2nd language boat. So
then of that group, you advantage people with a talent for natural languages,
or who are wealthy enough, as a child, to have access to education resources
to learn another language. As a former English teacher in China, I promise you
that this is a huge economic divide. The kids who can afford to learn English
succeed, those who can't, don't. That's fucked up, and not at all a reasonable
way to approach business. A good engineer should not be left with no
opportunities because they grew up poor, or have no real talent learning
foreign languages.

In business, such concerns about equality and fair access might not be very
important, because unfair advantages and competition are what much of business
is all about... but in open source, and in modern businesses that are learning
from the open source model, I think being concerned with inclusiveness is
paramount. So yes, you might have to struggle through reading something like
this:

[https://github.com/be5invis/youki/blob/master/README.yk](https://github.com/be5invis/youki/blob/master/README.yk)

... but it doesn't take much to punch "Youki
是一个文档生成语言，其具备完整的编程特性和完备的宏特性，使之可作任意复杂的文章。Youki 是完全开放源码的，依照 MIT 协议发布。" into
Google Translate (or use the auto-translate features of Chrome). Now we can
all get on with our lives and use be5invis's cool documentation generation
tool.

Would you prefer to exclude him and his work from our community by forcing
him, a Chinese square-peg through some English speaking round-hole?
Personally, I'm willing to do a little extra work to understand him in his own
language. I also appreciate deeply that he has to do that work constantly, in
order to read my code/docs in my native language.

~~~
vacri
Most people in the world do not speak one of the uber-languages (english,
chinese, spanish etc). These people are particularly poorly served by a drive
to document in their native language rather than a common language. If someone
only speaks Xhosa and never gets to exercise a documentation-common-language,
then they'll only be able to read and contribute to documentation in Xhosa,
which is a much smaller community than 'everyone'. And what if the leading
engineer in your field didn't document in that language? You'd be cut of from
what they had to say.

A common documentation language functions as a technical standard, like IPv4.
Sure, you can use your own standard, perhaps IPX, but you'll only be able to
communicate with a much smaller group of people.

It's hard to get documentation written in the first place, let alone good
documentation. Making it 'fair' by saying that documentation has to be
effectively splintered into every native language is a sisyphean task.

As always, write to your audience. If it's a general tech audience, that means
English. If it's Chinese engineers who don't speak English well, that means
Chinese. But those Chinese engineers are going to have an easier time
interpreting foreign documentation if it's all in one language rather than 50.

~~~
thoward37
I don't disagree. The problem of standards is that sometimes they help and
sometimes they hurt. They always seem like a great solution, but rarely are.

Wouldn't it be nice if we all spoke the same language?

So much less conflict, so much more efficient.

Wouldn't it be nice if we all had the same cultural background?

So much less conflict, so much more efficient.

Wouldn't it be nice if we all looked and acted exactly the same?

So much less conflict, so much more efficient.

Wouldn't the world be a nicer place without all this pesky diversity?

So much less conflict, so much more efficient.

In the end, there's not one correct answer. The point of documentation is
communication. Communication is a very malleable thing. It is negotiated on
demand and the form of it varies a lot. The real beauty of many of our
underlying systems is their ability to negotiate a protocol, not the pervasive
standard use of a single protocol.

A couple personal stories:

Once, I was in Switzerland, and went to a Chinese restaurant. The waitress
spoken Cantonese (1st), Swiss (2nd), and Mandarin (2nd). I spoke English
(1st), German (2nd), and Mandarin (2nd). We quickly figured out that we could
communicate most effectively in Mandarin, and so we did.

Later on in that trip, I went to an Italian restaurant. The owner spoke
Italian and some very limited Swiss, but no English. I couldn't speak Italian
or Swiss enough to communicate with him. Instead, we used gestures, body
language, and pictures. It worked out, and I got one of the best plates of
lasagna I've ever had.

The message here is that language is not standardized, and will never be, so
calling English a standard is just a failure of reason. It's a sub-par medium
for communication for the majority of the world.

You never know who is going to walk in the door of your restaurant to order
food. Do your best to communicate with them, without too many assumptions, and
you'll find that communication can be negotiated easily.

That said, in more practical terms, I actually encourage the use of less text,
but more pictures, examples, and tactile learning approaches in documentation.
When text is required, it should be "High Fidelity" and that means, written in
the native language of the person writing it. Then, you have the ability to
degrade fidelity later, by translating it to whatever language you've
negotiated for when the native language doesn't match.

~~~
smoe
I agree that language is not standardized and should never be.

I see English in programming as the somewhat lowest common denominator. At
least in the "western world".

From all the 4 languages I learned and some more I messed around with, English
was the easiest to get to a basic level. On Par with Spanish. This is highly
biased of course.

Although my mother tongue is German(Swiss), I usually avoid documentations,
tutorials, etc in German for multiple reasons.

\- The mix of english terminology, germanized terminology and normal German
makes it hard to read.

\- They are mostly just translated and not rewritten, which sometimes leads to
some strange formulations.

\- I'm forgiving/ignorant for grammatical errors in foreign languages

\- They tend to be outdated.

I agree that documentation should only use as few text as possible and as much
necessary. Editable and runnable examples are the best way in my opinion.
Also, no matter what language, documentation should be written using simple
words, short sentences and not try to win a pulitzer prize.

I don't mind documentation in English. What really pisses me off is, when I
have to work with software where the developers/designers clearly didn't
account for things like letters with stuff on it (éàüöøñ…), variable sentence
lengths, date formats, etc.

------
incision
Historically, I've been accessing documentation as an administrator rather
than a programmer.

There are a couple of documentation habits that have driven me half mad over
the years.

* Broken Usage Examples:

It's bad enough when an example demonstrating some now deprecated feature
hangs around in the introductory text that every new user will cut their teeth
on. It's an immediate vote of no confidence when typos or plainly busted
syntax makes it into the docs.

* Confusingly Abstract Examples:

Please no ./foobar -f foo -b bar -i foobar.foo -o barfoo.bar. Just paste an
actual usage example, explain it and be done with it. A cousin to this is the
use of overly complex "Look how clever I am!" examples.

* Explanation By Analogy / Google:

Saying things like "configuration files use ____ syntax" without summarizing
or at the very least linking to a definitive reference for "____ syntax".

* Insane Defaults:

This is really a design concern, but if we're already past that point and
destructive / poor performing default behavior is the chosen path at least
clearly document that fact.

\---

On a related note, when I'd just started to augment my sysadmin background
with code I saved time by writing scripts, small utilities and general
automation.

As I've learned more, my savings increasingly come from being able to jump
into the code to diagnose the odd behavior of poorly documented programs
instead of endless profiling, searching and experimentation.

~~~
noobface
From a systems engineering perspective:

Any marketing, at all, in documentation really complicates my research.

I was already interested in the product, and I'm trying to solve a problem
with it. I understand the use case. I don't need to know how this saved X
company $XXX,XXX dollars per second over the past billion years.

Obscuring the meat-and-potatoes technical specifications behind phrases like,
"cloud ready" and "virtualization optimized" just makes it harder to evaluate
the product.

------
massysett
"If your documentation is a directory full of files on GitHub, I close the
tab."

Without even stopping to see if there is an INDEX or README that gives the
files a hierarchy? Without considering whether those files are well-written?
Without remembering that plain text files are portable, easy to author in many
tools, and easy to view on many devices? You'd rather see a glitzy website
with no content?

"If your documentation is generated from source code, I am immediately
skeptical."

Even though keeping documentation with the code it documents makes it more
likely the two will be in sync? Even though some library authors use their
source code documentation tool to write entire files that are nothing but
documentation? Maybe you're viewing it wrong: my documentation is not
generated from source code. My documentation includes the source code and will
generate the library.

Since he seems bent on judging things in 30 seconds I guess it is inevitable
that he will elevate form over substance.

~~~
forsaken
Sure, keep the files as plain text in the repo, I'm all for that. Just don't
make it the main interface for viewing them. You are really arguing that a
GitHub file browser is the best UX for viewing documentation?

Sure, keep documentation about code in the source code. Where exactly in the
source code does your tutorial go?

You seem to be reading into a lot of things that aren't in the article based
on what you want to hear.

~~~
massysett
less(1)

vi(1)

emacs(1)

firefox(1)

google-chrome(1)

Microsoft Word or Internet Explorer, if that's your thing!

As for "where does your tutorial go," see for example
"Control.Pipes.Tutorial".

[http://hackage.haskell.org/package/pipes-4.0.0/docs/Pipes-
Tu...](http://hackage.haskell.org/package/pipes-4.0.0/docs/Pipes-
Tutorial.html)

If someone can't figure these things out, by all means, go use someone else's
library. This sort of documentation would not be acceptable for a consumer
product like TurboTax, but a programmer needs to know how to view a text file.

~~~
lowboy
I know how to view a text file, or even navigate a directory of them. But as
forsaken said, a GitHub file browser is often not the best UI for viewing
documentation, especially when you're already in a browser.

What's better, the docs for Backbone.Marionette[0], or Backbone[1]?

[0]:
[https://github.com/marionettejs/backbone.marionette/tree/mas...](https://github.com/marionettejs/backbone.marionette/tree/master/docs)
[1]: [http://backbonejs.org/](http://backbonejs.org/)

~~~
pseut
> What's better, the docs for Backbone.Marionette, or Backbone?

Offline or online viewing? Should I need a web connection to read the
documentation?

------
oskarth
I agree with most of these, but this one stuck out.

> I believe that translating documentation is a really important step towards
> helping people learn to program. Someone shouldn’t have to learn Programming
> and English at the same time.

Really? Knowing English is pretty much a prerequisite for programming. Should
all UNIX man pages exist in other languages too?

That said, of course tutorials in other languages are useful, but it's hardly
the job of the open source project.

EDIT: "Prerequisite for programming" in the literal sense is obviously wrong.
It's very useful though, and pretty much a must to participate in most big
open source projects (mailing lists, issue trackers, comments, documentation).
Is this really a controversial statement?

~~~
avmich
> Knowing English is pretty much a prerequisite for programming.

Emphatically no. You can be a brilliant programmer without knowing English. Of
course, that doesn't say that knowing English somehow hurts.

I've met many programmers smart enough to learn programming to expert levels
despite not knowing English and having to rely on whatever was available in
other languages - sometimes that was plenty, sometimes they had to reinvent
algorithms. At the same time I know other good programmers who were faced with
the problem of learning programming and English at the same time and succeeded
at that.

------
simonsarris
I'm glad that we do everyone one of these, except for language.

On the versioning we go one step further. Not only is the documentation
available at every version, such as:

[http://www.gojs.net/1.3.3/api/symbols/Diagram.html](http://www.gojs.net/1.3.3/api/symbols/Diagram.html)

The latest version is also available at its own eternally up-to-date permalink
(also the default), so if you want to bookmark the documentation, or samples,
or intro guide at _whatever the latest version might be_ , you can just
substitute _latest_ for the version number:

[http://www.gojs.net/latest/api/symbols/Diagram.html](http://www.gojs.net/latest/api/symbols/Diagram.html)

(The same goes for the current Beta, if there is one, at /beta/)

In addition, the top of the page in the documentation (or bottom of the page,
in the samples) always displays the currently-viewed version number.

We try hard to make sure nearly all classes are introduced in prose, and have
a separate set of technical introduction pages to accompany with several live
examples (One of the more wonderful things about writing a JavaScript
library!)

------
sergiotapia
Speaking of documentation... I'm starting work on my first Go library and I
want to make sure I'm not fighting against the ecosystem.

[https://github.com/sergiotapia/smitego](https://github.com/sergiotapia/smitego)

How do go programmers write their documentation? Do I just write comments on
func's and use a tool to generate it? What the standard?

Thanks

~~~
Lukasa
Most languages now have a tool for writing and hosting documentation to make
this as easy as possible for you. The author of the post runs
[http://readthedocs.org/](http://readthedocs.org/), which is primarily used by
Python developers, for instance.

For Go, the standard toolchain is the builtin go doc tool
([http://godoc.org/code.google.com/p/go.tools/cmd/godoc](http://godoc.org/code.google.com/p/go.tools/cmd/godoc)),
and the GoDoc website for hosting
([https://godoc.org/-/about](https://godoc.org/-/about)). Using that pair of
tools will be a good start.

The advantage of these standardised tools is that it gives most documentation
for projects in a given language a common 'look'. Go libraries have one kind
of appearance, Python ones another, and so on. This tends to make navigating
around much easier.

~~~
forsaken
However, Sphinx & Read the Docs is not just for Python :)

[http://ericholscher.com/blog/2014/feb/11/sphinx-isnt-just-
fo...](http://ericholscher.com/blog/2014/feb/11/sphinx-isnt-just-for-python/)

------
mooism2
> If you don’t provide a language in your URL, you are implicitly sending the
> message that the documentation will never be translated.

RIP Accept-Language.

~~~
forsaken
A good host will redirect to the proper version of the docs when presented
with a root URL, but how do you link someone to a specific language, if the
Accept-Language is the canonical source for this information?

~~~
couchand
Wouldn't the use case be just "link to the right spot in the docs" and the
user's client requests the appropriate language automagically? That way the
link is valid for everyone, and it shows each their preferred language.

~~~
ramchip
If I'm reading something in language A which deep links me to some
documentation, I think it's quite likely that I want to see it in language A,
not language B which happens to be my machine's native language.

For example recently I was linked to a Microsoft page via a comment, but it
rendered in Japanese, which was a pain to me because I was looking for a
specific citation in English. I had to switch language then dig through the
site to get back the text the comment author was talking about.

~~~
gnud
If sites actually respected accept-language, I think this UI problem would be
fixed by browsers in short order (change accept-language and reload)

------
vacri
It's hard to take documentation advice from a blog with a low-contrast text.

If I have to struggle a little to read your text, it's clear you're more
interested in looking cool than conveying information - a sin as bad as any in
the article's list.

Not everyone speaks English? Yes, well, not everyone has an excellent screen
that brings out the colours you've selected just so.

~~~
usea
Not to mention whatever web font that is [1] renders terribly in windows. If I
cannot read your article on criticizing text communication without copying and
pasting the contents into a text editor, I close the tab.

Screenshot: [http://i.imgur.com/qUWjWzb.png](http://i.imgur.com/qUWjWzb.png)

[1] `'goudy old style','minion pro','bell mt'` etc

~~~
Moru
Yes I had the same problem but at the end I found a useful button called "Page
source". That one makes the page readable again. It's just a normal textfile
then .-)

------
justinj
> When you look at a project’s code for the first time, what are the things
> you look for?

Tests.

~~~
clarry
Obvious bugs.

------
dlitz
> If your documentation is a directory full of files on GitHub, I close the
> tab.

I closed the tab after reading this.

~~~
rok3
Seriously, not every project warrants a full-blown site / custom web interface
for the documentation.

------
mystral
"If you included all of the things needed to document a project in source,
your code would be unreadable."

Donald Knuth would disagree.

------
chacham15
First of all, to begin with a famous quote:

Documentation "…when it is good, it is very, very good; and when it is bad, it
is better than nothing." \- Dick Brandon

Second of all, I couldnt disagree more with this statement for certain types
of projects: "If you included all of the things needed to document a project
in source, your code would be unreadable."

I believe in self documenting code with additional information for API
functions. A really good way of doing this is with wonderful tools like
Doxygen:
[http://www.stack.nl/~dimitri/doxygen/](http://www.stack.nl/~dimitri/doxygen/)

------
suprjami
Not reading Github pages is stupid and ignorant. You can use Github as a
publishing platform. It CAN be good enough for documentation for a small or
medium sized project. This can allow quick creation of fairly nice docs,
allowing the coder to get back to coding. Hell, all the kernel docs are text
files. You gonna tell me you'll never read them on principle?

Judge documentation by its quality of content, not its publishing platform.

~~~
dominicrodger
> If your documentation is a directory full of files on GitHub, I close the
> tab. With GitHub Pages, Read the Docs, and other places to host generated
> documentation for free, not making an effort is unforgivable.

He's not arguing against GitHub pages - he's arguing against just linking to a
file/folder in your GitHub repository and thinking you're done (e.g.
[https://github.com/<org>/<project>/docs](https://github.com/<org>/<project>/docs)).

~~~
Gigablah
GitHub renders markdown files. So the difference between clicking on
navigation links in a documentation website and clicking on markdown files in
a GitHub docs folder is... the CSS theme.

~~~
cobralibre
As I see it, the author of TFA is simply offering some heuristics for judging
whether or not he's going to have a hard or an easy time working with a
project's documentation. So in this view, presenting documentation using a
structure that is specifically intended for documentation is taken as an
indicator of care and attention to detail. I wouldn't say that he's arguing
that a directory full of files isn't functionally equivalent or isn't useful.

(Personally, out here in the real world, I'll put up with all sorts of
limitations — even a complete lack of documentation — if somebody else's
existing code solves a problem that I have.)

------
duwease
I would have rules for how I judge documentation, but I see it so rarely that
"any documentation at all" is the highest honorific.

------
technomancy
> If your documentation is a directory full of files on GitHub, I close the
> tab.

I don't get this at all. Why is a static site (where it's unclear which branch
it was generated from or when) better than a bunch of nicely-rendered markdown
files stored in a directory?

~~~
MrZongle2
No kidding.

I'll take a Markdown or plain-text README over a slick set of web pages if the
former is current and thorough. Heck, I'd settle for _either_ of those.

------
conductr
I use most code from OS projects and I've always preferred good use examples.
I see a lot of overly simple use examples which really give no direction for a
new user(Apple ios examples). Any format works but list all the methods of a
class. And list what arguments they accept and the acceptable types of each.
And finally list what is returned.

I document my code as a business logic statement. Eg given credit card,
process payment. I then write the validations in a way that reads simple and
thus is documentation. If it's a complex line that takes more that a few
seconds to parse mentally. I will write out what it does. That's pretty rare.
Non programmers never see my code so that works just fine.

------
Yahivin
I judge the quality of documentation by having interactive examples. Words are
great, and also important, but interactive demos that exist within the
documentation go a long way to assisting people in learning your project.

Here's a couple of examples: [http://distri.github.io/pixie-
canvas/docs/](http://distri.github.io/pixie-canvas/docs/)
[http://www.danielx.net/interactive/docs/](http://www.danielx.net/interactive/docs/)
<\- The meta one

------
susi22
ReadTheDocs [1] is by far the best platform to have documentation. The simple
reason: Everything gets into an Elasticsearch server (sponsored by Rackspace)
which means you get wonderful full text search (and faceting).

See:

[http://read-the-docs.readthedocs.org/en/latest/search.html?q...](http://read-
the-docs.readthedocs.org/en/latest/search.html?q=great)

[1] [https://readthedocs.org/](https://readthedocs.org/)

~~~
rogerbinns
Sadly they aren't useful if your documentation is more complex. For my project
I do use Sphinx for the doc, but there is a prebuild stage where some of the
API information is extracted from non-Python text files (C source). Another
part runs the example code and captures it as RST. There are various other
nuances like that.

readthedocs can't build that (make docs would work on my tree). I've contacted
them several times to ask for how to proceed and never get an answer.

You can see my Sphinx doc not at readthedocs but at
[http://rogerbinns.github.io/apsw](http://rogerbinns.github.io/apsw)

------
ben336
I think the author is conflating "quality" of documentation with "usability"
of documentation. They are probably correlated, and this might make a good
filter for ignoring BAD documentation, but a documentation site can be very
"usable" without containing quality content, and it can contain quality
content without being "usable"

------
jj00
I've been evaluating using a wiki for some of our documentation, with
DokuWiki. It uses plain-text markup files, but also allows for some
collaboration between users. I can see where it wouldn't be so hot on the
version front, but I'm hoping that we can come up with a procedure to keep
those text files in a repository.

------
schmichael
I really hope the golang community doesn't consider godoc.org to be the
beginning and end of their project documentation. It's a fantastic service and
probably Good Enough for small projects, but if larger projects don't include
prose like Effective Go it's going to really hurt adoption.

~~~
burntsushi
There is plenty of room for prose generated by godoc in a package's
documentation. (Which precedes the `package ...` line in a source file.)

------
adammil
I'm with the author until he says to encode extra information into the URL. If
the docs and the site are built well, you should find everything you need
while completely ignoring the URL structure.

~~~
antsar
Perhaps, but a well-designed URL structure is helpful when one sees a deep
link to the site and wants to know what they'll find.

    
    
      example.com/how-to-bar-the-foo-v1
    
      docs.example.com/en/1.0/foo#bar
    

The latter form clearly presents the possibility of jumping straight to a
different version's foo docs, to the docs homepage, etc. I'd consider this a
quality of good UX.

------
feralmoan
Worth it for the hint on MkDocs alone - thanks Eric Holscher!

------
chrisBob
Please don't make me open an MS Word file to read your documentation.

------
coldtea
How _I_ judge the quality of documentation:

Is it on a wiki? Leave project and never come back.

