
How I Judge the Quality of Documentation in 30 Seconds (2014) - Tomte
http://ericholscher.com/blog/2014/feb/27/how-i-judge-documentation-quality/
======
amingilani
Here's a summary:

1\. Get a website: don't use a readme on GitHub.

2\. Use Prose: don't generate from source, you need more.

3\. Give permalinks for citation purposes

4\. Your URL should acknowledge the documentation version and language, for
future-proofing

And now for my own opinion:

1\. No, not every project finds it rational to put time into creating a
website. Especially if they're small. Sometimes, the time is better spent
doing dev work than creating a pretty site to satisfy you.

2\. Rails' documentation is generated from source, so is Node's. A lot of
documentation is. They give accompanying guides, sure, but there's nothing
wrong with generating the majority of your documentation from source.

3 & 4\. These are nice to have, sure, but not requirements. If a project is
small, I'd rather the solo dev spend their free time solving tough problems.

Essentially: if you're judging a whole project in 30 seconds, I think you're
the problem and not the developer that spent all their time making a project
for you to use. So, go ahead, "close the tab" as many times as you like there.
But if you really want to help someone, open a pull request, or at least file
an issue in a curteous time.

~~~
hliyan
Disagree on the readme.md. At work, we recently moved _away_ from dedicated
project/component documentation pages and _toward_ readme.md.

It's the best way to ensure that the documentation is in sync with the
implementation: the developer is more likely to update the readme.md in the
same pull request he/she changes the implementation.

On Github, I seldom read the market-y website. I click the "Fork me on Github"
link and go to the documentation pages on the source itself. It's the most
reliable, in my view.

~~~
namanyayg
Likewise, I always consult the GitHub docs (usually README.md) and code rather
than see the website (which is sometimes out of sync) but usually projects
which do not have a dedicated website do not have good READMEs either. This is
my experience with full-stack JavaScript libraries/plugins.

~~~
Ayesh
Also helps when you have to work offline.

I work on projects while I travel, and I always force my package manager to
download the source (not dist) version of the package.

The readme files are now included in the project, and it's a lot easier to
just read them. Add an MD viewer to the IDE to make things easier.

------
ronilan
I’ve published Open Source code and documentation. It is a lot of work.

I think that _“How I judge someone’s else hours of effort in 30 seconds”_ is a
toxic attitude.

~~~
dwaltrip
It's often very useful to quickly evaluate a project. I don't think the author
of this article intended any disrespect or to diminish someone's hard work.
It's more about efficiently navigating a vast landscape of tools. Although,
perhaps this could have been made more clear.

Say some random project has 10 stars on github and no commits in the past 2
years. Good documentation and working examples could be the difference between
using it and not using it. If you only have a single day to spike the feature,
you need to efficiently explore possible solutions.

Obviously, this depends on many factors. How critical is the feature? What are
the alternatives? How long would implementing it in house take? And so on.

~~~
anothergoogler
I'm with parent, even my bad READMEs took a lot of time. Consumers of open
source are just so damn entitled. It's discouraging.

~~~
ubernostrum
OK. I use a very similar set of heuristics to what's described in this post.

Here's the documentation for the most recent thing I released:

[https://django-registration.readthedocs.io/en/3.0/](https://django-
registration.readthedocs.io/en/3.0/)

And the one before that:

[https://pwned-passwords-django.readthedocs.io/en/1.3.1/](https://pwned-
passwords-django.readthedocs.io/en/1.3.1/)

Here's the next of my packages that I'm working on, not because the code needs
work but because the documentation isn't up to my standards anymore (in fact,
I'm doing a rolling refresh of all my personal packages right now):

[https://webcolors.readthedocs.io/en/1.8.1/](https://webcolors.readthedocs.io/en/1.8.1/)

I don't expect everyone else to match my output in documentation. I do expect
people to write some type of prose documentation covering more than just
"here's an auto-generated API reference, good luck", or "here's a README with
a couple examples, good luck". And I absolutely treat quality of documentation
as a predictor for quality of code, because it tends to be a pretty strong
predictor.

Want to tell me how "entitled" I am?

~~~
JdeBP
Note that "anothergoogler" said that _consumers_ of open source were acting as
though entitled, not _producers_.

~~~
ubernostrum
Well, I'm both a producer and a consumer of open source code.

So is Eric, who wrote the "entitled" post being complained about.

------
MrTonyD
Interesting to see how someone experienced evaluates documentation. I've been
doing Curriculum Development for several years, and it's easy to identify bad
Curriculum. If I flip through a course and don't see a lot of diagrams
(20%-50%) then I know that somebody didn't know all the research showing that
diagrams aid recall and simplify the understanding of relationships. And if I
don't see frequent labs/practice, then somebody didn't know all the research
showing that the brain needs to have new concepts reinforced, and skipping
that step prevents new information from being well-absorbed. I could go on,
but those two are typical. And people wonder why Training isn't more
effective. I have seen bad training sink more than one high-tech company -
though nobody at those companies seemed to understand that. There is also an
art to properly decomposing complex topics so that they can be "intuitive". My
very best technical courses were often criticized as being "easy" and
"obvious" \- but only by those who hadn't tried to learn the same information
before.

~~~
ArchTypical
> With GitHub Pages, Read the Docs, and other places to host generated
> documentation for free

> If your documentation is generated from source code, I am immediately
> skeptical

Looks like another self-important troll. Those statements are just lolwhut.

I'm not going to learn (or force others to learn) a tool to read
documentation. Put a /docs folder in the appropriate project directories and
have the team that handles it, decide what to put in that (even if it's a URL
to somewhere else).

Disregarding existing documentation because you dislike the
organization/formatting is such an act of blind ignorance, I'm surprised he
thinks he knows anything about it. Documentation is not a solved problem
(although it's easier than testing).

> If you included all of the things needed to document a project in source,
> your code would be unreadable.

That's only because modern IDEs haven't implemented inline-documentation
methods yet. One day we'll get companion documents with every sourcefile that
will allow developers to read notes and link to associated content from
orthogonal comment files.

~~~
MrTonyD
While I don't agree with the article 100%, I too am skeptical when I see
documentation generated from source. A complex topic requires decomposing the
information into different types of information with different approaches to
explain it. Not everything is "here is the API". For example there may be an
important overarching conceptual model, and information which maps into it. I
don't know any way to express that in any of the source program documentation
tools (at least, I've never seen it done in the many hundreds of projects I've
seen.) The problems go well beyond formatting and organization. (Though, some
things are simple APIs where some trivial examples and some minimal text can
explain it sufficiently. That does describe a lot of existing libraries.)

~~~
kevin_thibedeau
Sphinx does this better than most since it is a general purpose documentation
tool first with API doc generation as an optional add on. After setting it up
you can easily add manually written documentation as needed. Most importantly,
you can easily cross-reference the API docs and the manual prose from each
other.

Doxygen and its ilk are overly focused on generated API documentation
extracted from meta-comments and you rarely see them used with well organized
manually written text.

------
jtms
I appreciate and agree with all of the author's points, but I think that a few
really well thought out and explained examples on a simple Github README is
worth far more to me from a practical standpoint. I generally will look at the
examples, try them out in a stand alone test, spend a few minutes seeing what
the quality of the source code is, and then make a decision to go further with
it or not. For a smaller project that only has a limited set of features I
think most of the author's points are maybe a bit overly picky.

------
bschwindHN
I'm very thankful for Rustdoc. Every library's documentation has the same,
easily searchable format. The syntax for writing documentation is easy to use
and standard across all Rust projects. You can easily generate HTML
documentation from your source code and standalone markdown files. crates.io
will generate your documentation and host it for you when you publish a
library. Your example code in your documentation can actually be compiled and
tested so it stays up to date. It's been a joy to use and a major reason I
like programming in Rust.

I don't know about localization support, but it addresses most of the other
issues outlined in the article.

[https://doc.rust-lang.org/beta/rustdoc/what-is-
rustdoc.html](https://doc.rust-lang.org/beta/rustdoc/what-is-rustdoc.html)

~~~
steveklabnik
Localization is basically non-existent :( it’s a thing we want to work on, but
aren’t sure how to accomplish.

~~~
Manishearth
I've discussed this a fair bit and we have a rough plan, but the idea is to
wait for other things to localize first.

~~~
steveklabnik
Ah great!

------
sverige
I know that writing good documentation is hard, and that a lot of devs don't
like to do it, but I completely agree with TFA.

If the project isn't worth the time to document properly, it's self-announcing
that it isn't worth using that code. Why would anyone use throwaway code
posted on GitHub? Even if it's been written by a "rock star," the risks are
too high.

The philosophy of OpenBSD is the correct one, I think: incorrect documentation
is a bug and should be fixed with the same energy that is directed toward bugs
in code.

If it's only going to be used by you, great! Don't be offended if someone else
looks at it and says, no thanks because of the lack of documentation.

------
hyperpallium
Code is the best documentation, it's what I seem to always end up consulting.
And always in sync.

Plus some guide to the overall architecture. Now... a pattern language was
meant to provide this, but failed. Perhaps, since a program is a theory, to
understand the structure of the code, you must understand the structure of the
problem/domain... and there's no shortcut. But even if so, a helpful guide
seems possible.

Autogenerated docs can help navigate the codebase, but tools can autonavigate
just as well. They could show the overall code structure, but don't. They
could be in sync, but aren't.

A minimal but complete (all-steps) example, partly to learn from. and build
on, and also to assure me that _it works_.

Idea: I keep kinda expecting the text alongside each file/dir on github to be
a comment on that file/dir (instead of its latest commit message). Could be
incredibly valuable for navigating a codebase and grokking its architecture.
(Though prone to desync.) I also like the format of a commit message for this:
<50 char summary line, optional further lines.

code: electronic documentation: edoc

~~~
kkarakk
pull the ladder up after you mentality, how is someone new to your
language/stack going to understand your code without a similar level of
knowledge? this type of thinking leads to abandoned codebases and full product
rebuilds in another language with better documentation. currently migrating a
product for this very reason.

~~~
hyperpallium
This was based entirely on my experience with other people's projects that I
knew nothing about.

------
benbristow
I think the language thing is a bit unfair.

If your service targets non-English speaking countries, 100% you need your
documentation translated.

Otherwise - why?

~~~
bArray
Agreed and also, maybe it will support other languages in the future - just
not at this moment.

------
uvtc
Given that GitLab/GitHub/etc (and, often, online package repositories)
automatically display markdown files as rendered html, a directory of markdown
files is a fine way to document your project. The docs can, at any point,
optionally be processed into html that goes into a separate website (lots of
folks have written tools for this. My own is
[Rippledoc]([https://gitlab.com/uvtc/rippledoc)](https://gitlab.com/uvtc/rippledoc\))).

Also, if the project is a library or framework with a public interface, you
need _both_ API docs generated from source, _and_ prose docs.

------
jquast
I noticed a lot of FOSS bug reports/emails for my projects are coming from
foreigners lately, and I spent a lot of time thinking about how to help.
Translation seemed insurmountable. I considered a lot of options.

My conclusion was to _always_ strive for brevity and clarity. Say it all, say
no more, be clear, and be concise. I now consider a foreign audience that may
take significantly longer to understand what I write, they may use translation
tools, they may be confused by any "flowery language".

------
kaycebasques
Eric runs the yearly Write The Docs conference in Portland every year. I
highly recommend attending if you are interested in documentation from a
cross-functional perspective.

~~~
maxxxxx
My company should start hiring from that conference. We really need people who
care about docs. Devs usually hate it.

~~~
Sgt_Apone
They actually have a fairly active Slack [1] with a job posts channel, as well
as a Job Board [2] on their main site.

[1] [http://slack.writethedocs.org/](http://slack.writethedocs.org/) [2]
[https://jobs.writethedocs.org/](https://jobs.writethedocs.org/)

------
gilrain
How I judge the quality of a judgment: if its maker boasts how little time
they spent making the judgment, it’s probably pretty uninformed.

------
kkarakk
>Someone shouldn’t have to learn Programming and English at the same time.

However the attitude seems to be very much "learn Chinese/local language or
gtfo" from the opposite end wrt localisation ie it is nonexistent. I think
devs are really giving up something valuable by not pushing for a global
standard language for development.

sure I shouldn't HAVE to learn English and Development at the same time BUT
who is developing in my language anyways? no language support, no compiler
support,no Unicode support and no readable font support. instead of
bootstrapping all of that(an impossible task as there is no requirement and no
funding from any sources,i've tried but the govt and general public ranges
from apathetic to stubborn on this issue), it was easier for me to learn
English. I could have theoretically learnt the language one level up on the
linguistic chain but I would eventually hit the same issues as I tried to use
the available tools.

~~~
jerf
It's a nice thought in some ways, but that requirement in particular is where
I figured the author must be talking about massive projects, because I'm sure
not taking my ~100 man-hour open source project documentation and getting it
translated into half-a-dozen languages. A project has to be pretty big
(multiple orders of magnitude larger than that) before it can justify having
documentation in anything but whatever language the probably-single developer
is most comfortable documenting in.

------
solipsism
I was excited about this, but it was mostly worthless. The equivalent of a
"how to write a great paper" guide explaining font size, where to put your
name, to include a table of contents... And that's it.

Nothing about _what_ to write and how to write it? This is the hard part.

------
mlthoughts2018
Internal development is very different than externally facing development. For
example, generated API docs are a horrible thing for purely internal code,
because you need a documentation build and deployment system that inevitably
breaks and nobody wants to maintain and gets out of sync and suddenly the
documentation is stale and there's some engineer on the brink of rage quitting
if someone pings them to manually rebuild the documentation pages one more
goddamn time.

Having it just in the repo as a set of Markdown docs or something is much
better, and "the website" is just URLs to those files in GitHub. Same risk of
becoming stale as anything else, but no overhead of documentation build and
deployment.

------
randallsquared
> If you don’t provide a language in your URL, you are implicitly sending the
> message that the documentation will never be translated.

Is the `accept-language` header basically dead, these days?

~~~
Tomte
Thank God, yes.

I mean, in theory it's great. In practice you always got cringeworthily bad
translated versions of the Debian web site.

(A few years ago Debian's German pages got much better)

~~~
jessaustin
Why would choosing language via URL be any different than choosing language
via header, with respect to the quality of the translation?

------
cube00
For APIs, I find having a good test suite means I can get started
understanding how to call the library quickly.

------
zygotic12
Judging books by covers. First time I've seen it in the wild!

