
Free software is suffering because coders don’t know how to write documentation - Aaronontheweb
https://thenextweb.com/dd/2017/06/02/free-software-is-suffering-because-coders-dont-know-how-to-write-documentation/?utm_source=social&utm_medium=feed&utm_campaign=profeed#.tnw_IZruvbiT
======
firethief
"Don't know how"? [A] flaw in human nature is that everybody wants to build
and nobody wants to do maintenance (Vonnegut). Most free software projects are
entirely built by volunteers contributing what they want to. I don't know how
much offering classes on "how" would help; it feels like leading a horse onto
a boat.

Also, the poll seems a suspect way to support the idea that more documentation
is justified; I agree with that idea, and TFA has other arguments supporting
the point, but I question the utility of the poll as an argument. Developers
would like better documentation of the libraries they are using, but they
would. In a Pareto-optimal scenario where exactly the right amount of work was
put into documentation, developers would still want better documentation. And
I would want a pony.

~~~
RealityVoid
I've seen this wierd issue in companies(not open-souce, just plain normal
companies with minimal OS involvement) where I'd send decent technical
writeups to people who sould understand them, with issues that affected them
but they didn't... friggin... read... them. I think maybe this problem is also
present in OS where, you write stuff but get a lot of questions back that
could be solved by people reading the friggin' documentation. Maybe this makes
people feel it's pointless.

------
epc
Technical writing is dead as a profession.

Too many companies expect technical writers to be as savvy, or more, as their
developers, but for lower pay, less respect, and zero political power.

This attitude bleeds through to open source/free software.

You want better documentation? Find a way to compensate people for
contributing to it, and that doesn't necessarily mean money. Separate your
documentation repository from your code base.

~~~
hkmurakami
Afaik RedHat pays their documentation engineers and QA engineers very well,
since they consider those roles to be essential to the product from the
customer's expectations. It's a perspective rarely shared by other companies
though.

~~~
praneshp
> "It's a perspective rarely shared by other companies though."

This is not the worst case scenario. Some places will sensationally hire
QA/Writers, hailing their importance, and lay them off at the first sign of
cost cutting.

------
austincheney
This isn't just a problem in open source. It is a huge problem in corporate
internal software too.

Most OSS projects that expect to receive (and actually get) the attention of
users or contributors tend to work harder on better documentation.

~~~
BaronSamedi
Even some commercial software has poor documentation. Still, open source
software has almost universally very poor documentation. I don't know if it
hurts free software but it certainly hurts those who use it. If a project is
reasonably popular, say a thousand users, and each user on average has to
spend an hour to figure out how to use it, then that is 1,000 hours wasted.
The developer, could instead have spent say 8 hours writing documentation to
save the users that one hour (* 1000).

Contrived example, I know, but the principle holds: time spent by the
developer writing good documentation saves users a lot of time. Most
developers, in my experience, do not like to write documentation so they
prioritize user's needs below their own discomfort.

Some try to take the lazy way out by going the "I'll let the users create the
docs on a Wiki" route. See, for example, the Freeswitch (VoIP server)
documentation for just how bad this can be.

A lot of developers do want to write good documentation they just don't know
how. In this case what we get are reference guides instead of user manuals.
Writing small user manuals for OSS projects is not that hard. The key is to
put yourself in the user's shoes. Your user wants to accomplish some task with
your software, so, in your manual give them step-by-step instructions for how
to accomplish it.

It's like writing an algorithm, except that the instructions will run on
wetware. For example, to install the software do A,B,C,etc. To compute the
standard deviation of your data do 1,2,3 & 4\. Spell everything out, just as
would do for the computer--assume nothing (ELI5). Don't just give a list of
functions.

Users want code that solves their problem with the least possible investment
in time and pain. I'm sure many people reading this have been through the OSS
pain cycle: find a library that does what you want, see the sparse
documentation, and then brace yourself for the pain of figuring out how in the
hell to actually use it.

~~~
vosper
> If a project is reasonably popular, say a thousand users, and each user on
> average has to spend an hour to figure out how to use it, then that is 1,000
> hours wasted. The developer, could instead have spent say 8 hours writing
> documentation to save the users that one hour (* 1000).

That's true, although it's worth remembering that the 1,000 hours are other
peoples time, not the developer's. It might be quite easy for a busy open-
source developer (often working in spare time, already) to think "I've got
other things I'd rather be doing".

Also, it's not just 8 hours one time - it's then ongoing maintenance. So
you're spending time in order to create even more work for yourself in the
future; documentation that doesn't exist doesn't have to be maintained
alongside code changes.

We need documentation bounties, or something.

~~~
GregBuchholz
>We need documentation bounties, or something.

...Are there write ups exploring what "or something" could possibly be? Like
futures contracts or options or potentially other risk mitigation practices
from the financial world? Something that would help with collaboration and
help with getting the timing correct? As in, the best time to have the
documentation is at the point, or right before some software becomes
"popular". But that seems like the time you are least likely to have enough
people contributing to bounties. Is there a way to encourage documentation
writing speculators? Is there a way to minimize the bad effects (if any) of
winner-take-all bounties in discouraging contributors? That is, for bounties,
how do you know if someone isn't already working on the documentation unknown
to you, and finishes it (and collects the bounty), one week before you finish
your version? And the fear of this prevents people from starting in the first
place. Is there a way to make "progress payments" on documentation bounties?

~~~
vosper
I don't know, but you raise some really interesting points, and we didn't even
talk about the quality of the documentation that might be produced. Bounties
might be the wrong way to do it, but I can imagine a Patreon-style approach
might work. Like "I'm the developer on this, and I'd rather be writing code.
But if people contribute $X to my Patreon I will dedicate 5 hours a month to
working on the documentation, instead".

------
franciscop
I was waiting until the dreaded Monday to launch this but I guess I'll do a
soft launch now since it's totally in-topic. I even updated it to add Github's
Survey results.

 _Let me write the documentation for you!_ I've created this:
[https://documentation.agency/](https://documentation.agency/) and you can
email me so my colleagues and I will write or improve your project's
documentation. Added a landing page (like
[https://picnicss.com/](https://picnicss.com/) ) and design if you want it. It
is _for-profit_ so far, but if everything goes well I'll start doing some
heavily reduced prices for some OSS projects.

[https://documentation.agency/](https://documentation.agency/)

~~~
CM30
Hey, that's actually a pretty neat idea. I mean, we already have agencies for
things like web development, SEO and design, so one for documentation seems
like it'd be useful for lots of people too.

Hope it does well.

~~~
franciscop
Hey thanks! I got 0 emails so far though, so I think I'll have to go talk with
some SMB to see if they are interested. If you happen to know anyone please
let me/them know!

------
hprotagonist
This is a problem in some really major codebases too.

I routinely curse whoever wrote the "docs" for openCV.

~~~
pc2g4d
Oh my gosh, yes, OpenCV is really hard to get into. Which is a shame because
it seems extremely capable.

~~~
ldiracdelta
OpenCV was initially written in Intel and having been an Intel employee I
found OpenCV's documentation to be totally consistent with internal
documentation quality.

------
thxaoe7oiu
I have to laugh at GitHub coming to this conclusion. About 80% of the time
when I follow a link to GitHub, there's a list of files and absolutely no
explanation of what the project is or why one would be interested in
downloading or contributing to it. Sometimes there's a link to something
called "markup" or "markdown" that sometimes has a short description that
vaguely describes the project. But that's often missing or cryptic, too.

~~~
szatkus
Yep, GitHub is usually backup for me. Most of my projects is abandoned after
few commits.

~~~
ldiracdelta
Bitbucket has unlimited, free, private repos. I initially hide my manure.

------
kwhitefoot
Software suffers from poor documentation principally because those who write
the code don't have the time or inclination to write documentation, especially
if they will have to do it in their own time. Most experienced developers can
write well enough but they know that no one will maintain the docs so why
bother. In my experience as an internal application developer the
documentation is created only when someone leaves or a major application needs
to be ported.

Internal software is probably the worst because there isn't a paying customer
in the usual sense of the word so there is no one to send the bill to.

Do technical writers still exist? I could write documentation, in fact I
intend to spend three days next week doing just that, but someone who has that
as their primary skill would do a better job; unfortunately the company no
longer employs such people.

------
Twisell
As a newbie I often felt repelled by FOSS project I want to contribute to,
because even if code is clear and I might have found a piece of code to
improve, a lot's of project don't even include the more minimalist instruction
on "how to contribute" / "how does my build system/doc system work".

If you are doing thing really standard, point to the standard doc of you build
system. But most of the time first think you have to do is to spend days
trying to guess how the fuck the project is organized.

If your excuse is that you don't have time to write doc, maybe don't expect a
lot of contributors, because guess what maybe they don't have time to guess
what you didn't writeh.

------
hutzlibu
I think there is much potential of new coders wanting to participate to
projects, but they just don't know how and where. They maybe used a project a
bit and stumbeld over bad documentation or not working examples ..

So in theory, helping with documentation and fixing that would be a good start
for them. But it takes confidence to tell someone, especially when you are a
newb, that your documentation is sh*t and that you could help with that.

So if projects are more open for that - and explicitly say that, that they are
open and thankful for that(also for beginners) - it could be a huge benefit
for everyone ...

------
biocomputation
At work we spend a ton of money keeping our documentation up to date,
searchable, and available on the web. Worth every penny, and that's just if we
count the productivity boosts for internal users of the docs.

------
darkerside
The frustrating thing about poor documentation is that the product actually
works! It's just that the creator didn't go the last 5% of the way to explain
to someone else how. The close-ness of the solution is what makes it
frustrating by nature (it's not frustrating when you encounter a 2% complete
solution to your problem).

My point is, the fact that poor documentation frustrates many people according
to a survey does not mean that software developers can't or don't write
documentation.

~~~
klodolph
> It's just that the creator didn't go the last 5% of the way to explain to
> someone else how.

We software developers have this borderline criminal tendency to undervalue
work that isn't software development, like writing documentation. I'd say that
documentation might be closer to 20% of the work of a project, and developers
are usually bad at it (by "bad" I mean "unskilled").

~~~
franciscop
A -quite old[1]- estimate puts it in 20-40% of the total time should be spent
writing documentation. If you find any newer reference please let me know, I'd
love to quote it in my project to write project's documentation [2].

[1] [http://www.techrepublic.com/blog/10-things/10-things-you-
can...](http://www.techrepublic.com/blog/10-things/10-things-you-can-do-to-
create-better-documentation/)

[2] [https://documentation.agency/](https://documentation.agency/)

------
a2decrow
The first thing that came to mind:

 _Eagleson 's Law: Any code of your own that you haven't looked at for six or
more months, might as well have been written by someone else. (Eagleson is an
optimist, the real number is more like three weeks.)_

Writing documentation isn't so much a "how" problem, but a "why" problem. Most
people writing code in their own time can't be bothered also writing
documentation for it.

Writing code is fun (mostly). Writing documentation is not.

~~~
finnh
Wow, I don't agree with that "law" at all. A couple times in my career I've
worked for 5+ years in a codebase that I started, and it's always easier to
read my old code than to read someone else's. Like, waay easier (unless the
other person's code is super duper clean).

~~~
CM30
Those two views aren't necessarily opposites. Your old code can be hard to
read, and other people's code can simply be even more difficult to understand
than that.

------
louithethrid
I dont have the time. Train a Neural Net to generate documentation from my
rambling and mumbling while editing the source. Do something else to find a
solution. I shall write code. With speaking names. With identifyable concepts
and algorithms. That must be enough.

Also the webs inability to allow the users to meaningfully interact with
documentation - by instantly saying -"This step didnt work" and getting then
feedback from so god forsaken irc-channel or mailinglist. There is no real
integration of the users ability to contribute to documentation. And wikkis
are the worst at this. The tagged syntax, the time wise seperation from the
point of usage of the documentation.. it all adds barriers, where there should
be none. Documentation should not be added online at some remote site..
documentation should be generated by the user, while he is tryialed & errored
in some forgotten config- while he defeats a obsticle - and it should mention
those who cleared the path through the djungle as heroes.

------
kafkaesq
And why should they? That's not what they test for in interviews.

------
kartan
> Documenting software is extremely difficult. People go to university to
> learn to become technical writers, spending thousands of dollars, and
> several years of their life. It’s not really reasonable to expect every
> developer to know how to do it, and do it well.

This is the answer. Software developers are bad doctors, and bad writers, and
bad musicians, ... Why do people expect that software developers should have
so many other professional skills? The time when your developer designed,
implemented and written content for your corporate web is long gone. But we
still expect them to be good technical writers.

In the companies I have worked for, we have technical writers that help to
write organized, consistent complete documentation. And it is a full time job.

~~~
ljk
> _Why do people expect that software developers should have so many other
> professional skills?_

to cut cost most likely

~~~
bioapparatus
though I share your cynicism, this could easily be reversed as "people value
software developers that can also do other stuff good too."

------
niftich
What do we mean by 'documentation'? Usage of the outwardmost-exported
interface (which, depending on the project, may be an UI or an API)?
Configuration and customization options, installation instructions? Design
rationale, deep-dive into internals, explanation of internal components?

'Documentation' covers a wide swath of prose, and therefore makes for an easy
target for criticism which is in fact very disparate, despite appearing
homogeneous.

That is not to say most projects have good documentation. Writing prose --
specifically, technical writing -- is a skill orthogonal to software
development and doesn't necessarily tend to attract the same kinds of
individuals.

------
squarefoot
... and (although to a lesser extent) user interfaces.

Writing software, its documentation and its way to interact with the user are
three totally different beasts requiring different skill sets: documentation
and UIs should be written with people in mind, not algorithms, so better not
leave that task to a programmer.

Also, writing documentation and user interfaces isn't seen as k3w1 among
coders so it's understandable that people good at writing docs and/or
designing UIs are rather doing it for a fee outside the FS/OSS world than
because they love it.

------
5ilv3r
I see a lot of documentation describing how it works, not what it does. I
think most README.md files would be better as code comments. Personally, the
most useful documentation contains a brief summery of purpose, usage, and then
a few examples. Dependencies should be called out if they are significant.

------
xtracerx
Is this really a problem? I very seldom run across projects worth using with
poor documentation. It's usually pretty good.

------
webkike
Open Source is not Free software, nor are most of the projects on GitHub
intended for anyone to use. Consider how well documented projects under the
GNU banner are. Im sure this is a problem for some projects, but free software
is not suffering because of it.

~~~
rosser
As well-documented as GNU software is — and it is! remarkably so — I think the
project loses even _more_ points for foisting the atrocity that is gnu-info on
everyone. That is some abysmally "You will do it the way I want you to, and
like it!" software.

When I say `man foo`, do not suggest I look elsewhere, please.

~~~
klodolph
So, they've written this amazing documentation system which generates info and
HTML from the same sources, hosted all the HTML online and made it available
as separate downloads, also made it accessible from the terminal using "info",
and on top of that they've made a separate, shorter version of the
documentation available as a man page and you say that they should lose points
for it?

Is this actually a problem for you? I'm genuinely curious.

"Man" is only really usable when the pages are a reasonable size. Once they
get too big, you end up spending most of your time grepping through it or
paging forwards and backwards looking for the section you want. But
apparently, gnu-info is an "atrocity". Why? Is it somehow morally wrong to
ever try to make something better than "man"?

~~~
rosser
An "atrocity" is admittedly hyperbole, but when the man pages are vague, at
best, and often just say "use info instead", that reads more "Do it the GNU
way, or you're wrong," than, "Weren't we thoughtful to have given you all
these alternatives?" Particularly when the former is not an uncommon attitude
among FSF projects.

I'll totally own that I'm reactive to that, but I really do think it's a
little thoughtless, and a violation of the Principle of Least Surprise, to
direct people elsewhere. I've been burned entirely too many times, trying to
put out a fire, checking the man page to confirm I remember some argument or
option correctly, and being told I'm looking in the wrong place. It's not that
it's "morally wrong" to do that; it's that it introduces friction in a place
where that might be — demonstrably has been, given my experience, which I know
I'm not alone in having had — particularly painful.

Sometimes, "grepping through the man page" is, in fact, the quickest route to
the _info_ you need...

~~~
klodolph
I mean, sure, it's nice to have the options documented in a man page. But most
of the GNU projects _have exactly this_ in addition to the full docs in info
pages, multi-page HTML, and single-page HTML available for online or offline
browsing.

I think historically, there were some bad man pages for GNU projects which
just pointed to info, but I haven't seen that happen recently.

~~~
rosser
Then that's improved markedly since the last time I bothered trying to `man
foo` in the terminal for a GNU tool. I just go straight to the web, because at
least there I can Ctrl-F properly.

I'm not saying "don't even try to improve things", and I think that's a pretty
un-generous read of my concern. I'm saying _don 't break people's
expectations_. Especially if those expectations might come into play in
exigent circumstances, and create friction in a place where the user already
has a problem that is _far_ more important to them than whether they're using
the output from most flexible documentation tool or not.

Like I said, it sounds like that's improved. If so, I'm glad. But I have
specifically been burned, and my time that was wasted was also someone else's
money. Given that, a bit of, "Hey, guys? Maybe think about how this choice
might affect people?" is, IMO, warranted.

~~~
klodolph
> Given that, a bit of, "Hey, guys? Maybe think about how this choice might
> affect people?" is, IMO, warranted.

That's exactly what I have a problem with... there's a leap between "this
sucks" to "therefore they must not care about how this affects other people"
that is not justified. If you want something that fits your expectations of
how Unix should work, we already have that, it's BSD and it's constructed with
a lot of care not to break users' expectations of how Unix works.

The GNU project has really never had that as a project goal. The whole "GNU's
Not Unix" acronym is not just a stab at copyrights, but it's a pretty good
summary of their goal to create something which is _not Unix_ but instead
better than Unix. I'm not saying that they're doing it the right way, or that
they actually are better, but that's their stated goal. Pretty much every GNU
tool has some incompatibility with the equivalent Unix tool, and they don't
care, because they're not trying to make a copy of Unix. Back in the day this
was a problem because people would write shell scripts which would assume GNU
userland. You still see it in a lot of packages which say that you have to use
GNU Make, GNU Awk, GNU Sed, GNU Bison, etc. instead of the equivalent Unix
program.

These pieces of advice have fallen by the wayside since GNU userspace has
become the norm, but if you really want the Unix experience instead of the GNU
experience, go use BSD.

~~~
rosser
It's probably all the BSD in my background that established these
expectations. But the people who buy my time use Linux, so I do too...

(I've also just spent my entire workday cleaning up the consequences of
someone else's think-o instead of working on my own tasks, some of which have
a hard deadline. I'm probably kvetching more than anything. Apologies for the
noise.)

