I would upvote this a million times if I could. I'm one of the kinds of people who wants to read the entire reference manual, front to back, before using a programming language, library, etc. I want to make sure I know exactly what it does, the proper way to use it, and what it doesn't do. This is both so I can make an informed choice about the technology, and it saves a huge amount of time in the long run.
But it seems like there's a big trend now to just "get things out there" and that good documentation isn't "cool" anymore, kind of like braces in syntax.
Examples of good docs: PHP, jQuery, MySQL. (The first two sites include user comments too, which make things even more useful.)
Examples of terrible docs: Python, CoffeeScript (the worst)
I can at least understand insufficient docs for pre-1.0 versions when the implementation is changing constantly, but when something has been around for more than a year, it's just inexcusable.
I don't want a "getting started" guide that gives a bunch of examples. I don't want to type in the console to find out what methods an object has.
I want a friggin' reference manual, that includes (as applicable) syntax rules, exact rules governing whitespace, orders of operations, all functions, all parameters, parameters passed to a callback (why are these forgotten so often?), default values, flag values, all possible return values, specific exceptions that can be thrown, what input parameters result in undefined behavior.
Really, it's just not that hard. It may be grunt work, but if you'd rather make your users waste a cumulative 25,000+ hours figuring things out, rather than you spending 100 hours of your own explaining things, I just can't have respect for your product, no matter how otherwise amazing it is.
Why is the Python documentation terrible in this respect? Python comes with a language reference[1], that has most of the components you ask for (one of the components is in a document that's called something other than the language reference, but that's because it's the stdlib, not the language itself... it's certainly the same style of document, not a tutorial!):
1. Syntax (sections 5, 6, 7 and I guess also 9 if you want the full grammar)
2. Whitespace (section 2.1)
3. Operation priority (section 5.15 has it explicitly, the rest of the sections also clarify)
5. All parameters? Documentation should already have these.
6. Callback parameters: I can't think of any callbacks in the stdlib offhand, but the ones that do exist are methods you implement, and all the ones I can think of are documented
7. Default values and their meanings are pretty much always documented
8. I don't know what flag values are, but if they're just parameters they fall under (5)
9. Return values and exceptions should always be documented
10. There are plenty of cases where some input parameters causing undefined behavior is documented; some maybe not (for example I don't think urlparse.urlsplit documents what happens when you give it a scheme it doesn't know about), but MOSTLY it's documented.
A while back I was trying to figure out something in Python. Python isn't hard, and I can read it (since its practically like reading Ruby) and get by writing it here and there when needed.
So... I'm trying to use some Python library (whatever your equivalent of a Gem is). I spent a few hours ripping out my hair trying to find good documentation for how to install and manage Python libraries. There didn't appear to be the equivalent of RubyGems or Bundler. There did seem to be 2-3 different ways of managing them (eggs?), but just getting those programs working on my local system wasn't liking me either. Googling for "Using Python libraries" didn't return much useful- nor did "Installing Python Libraries".
I'm still unclear what the standard method for managing these is. When I checkout a Ruby thing, I just type 'bundle install' and all is fine then.
Yes, the technical docs were fine- but the baseline "how do I get this damn stuff, working!?!?!" wasn't.
There are various methods for installing 3rd party python libraries from PyPI [1], including easy_install, pip, or just extracting the tarball from PyPI and running `python setup install`.
Edit: There's also the official documentation page called "Installing Python Modules" which covers the last of those three methods: http://docs.python.org/install/index.html
I think this might be an issue with what you're used to. We have things like rvm, gems and bundler: look at virtualenv, distutils/setuptools... The equivalent of a Gemfile is a requirements file, pip knows how to install these (pip install -r FILE).
If you ask me to do the same in Ruby, I'd probably be lost too.
You mention CoffeeScript being one of the worst, and I don't necessarily disagree, but how has it become so popular despite its poor documentation? Is it because CS is just /that/ good? Or is it that it has developed a great community early on that compensates for the lack of documentation? Either way, writing good documentation seems like an either path than doing those two things.
There's also a third reason: it's very easy to see what CoffeeScript compiles to - the JavaScript source is easy enough to read and comprehend - and that behaviour is well documented.
Most of the times I'm looking at CoffeeScript to just figure out what it compiles to!
I don't see why you say that Python docs are terrible while PHP docs are good. Can you provide details? Are you sure this isn't just a matter of liking certain conventions over others?
It's a recurring argument on HN whether Python docs are good. Someone claims they're terrible, and someone always asks, "Why? I think the docs are good!"
There's a critical gap between the expectations of Ruby/Java programmers of how to write Python and the way Python programmers behave. I think it really is cultural, to an extent. The Java programmers look for Java-style documentation, with classes and methods and lists of exceptional conditions. The Ruby programmers look for Ruby-style documentation, with lots of diversions into the "why" of things and the context, possibly with pictures of foxes.
A Python programmer would consider the Java docs needlessly detailed and the Ruby docs a bit fluffy and not to the point, but after all, it is just a matter of viewpoint, community goals, and culture. There's also the issue of installing packages in Python, which can be a bit of a bummer, but I've never had a problem downloading a Python package from some site and following the directions.
What do you reproach to python and coffeescript docs? I find them very good and very usable. I want to do something? I have the answer nearly immediately (well except for some shitty modules in python (I'm looking at you urllib(2)/popen)) and it's a real pleasure to use them while coding.
I agree 95%. However, I really do like to see a lot of simple examples. They can often make things much easier to understand once you see it. Similar in concept to "A picture is worth a thousand words".
You're in a small minority, what you want and don't want is irrelevant. It's 2012, nobody reads dry reference manuals cover to cover, nevermind doing so before firing up a shell window and an editor.
Reference manuals are still nice to have of course for clarifying edge cases and solving language lawyer disputes. However claiming that they are a superior learning and time-saving tool than the learn-by-example techniques is way out of touch with how most people actually learn.
People learn in a variety of different ways, and I'm also a developer who always prefers a reference manual over learn-by-example techniques. You may not be one of those developers, but that doesn't mean we don't exist. I imagine there are plenty of people who don't learn by the reference manual but would prefer to have it available - if it was available - as well.
People in the hardware or embedded system industry have to read it page by page, word by word because they do. People who actually design compilers have that too. People who actually work with specs like Mozilla have a big manual too. People who actually use a 3rd-party product and integrate them into their product have to read those manuals a hundred times including meeting with the vendor.
If you are only doing your own product using Python libraries, then certainly you don't.
When it comes to open source work, which I mostly do on my own, limited time and don't get paid for, I have a choice. I can spend time writing code or writing documentation. The former I enjoy very much, the latter I don't like at all. Guess which I am going to pick.
The times when I've actually ended up writing some docs, I don't think anyone has ever read them. And writing the docs is just the beginning, they have to be maintained too. Out of date docs are perhaps worse than no docs at all.
I don't read docs either, because they tend to be out of date. Formal specifications are an exception. But when it comes to open source, I just tend to read the source because it's never out of date and tells the whole story. What was obvious to writer of the doc isn't obvious to me and vice versa.
The first person who comes to me asking for documentation to my projects volunteers to write them, like it or not.
> I can spend time writing code or writing documentation.
You are selling yourself short. Your documentation the first thing people see. If it says "incomplete, confusing, and half-hearted", I'm going to hit the back button in about 15 seconds. I'm not going to spend 15 minutes reading your code to see if the first impression is wrong unless I think there's no viable alternative project.
Conversely, if your documentation is clear, thorough, and gives examples of usage, I'm likely to trust your project and dig deeper.
I remember reading a story about Pages for Pages on NeXTSTEP. The manual was done before the software, so if there was any design question about how the program should work, they turned to the manual. I think it was an article by Bruce Webster (he is an awesome writer).
That's a code authoring technique I picked up from Code Complete.
It's useful for keeping your code in conformance with documentation. The problem is when the original spec turns out to be impossible (or difficult/expensive) to implement, and needs to be changed.
However if you doc then build, you've got a change control process that should accommodate this.
But the thing is that he is not selling himself at all. He writes some code for him and then maybe decides to release it as open source in case someone else finds it useful.
People who release undocumented code have no sense of social responsibility, and they are wasting everyone else's time.
The only way you can figure out if these releases do what you want is to spend half an hour trawling through the source code. Over the years I must have wasted weeks of productive time doing just that, and I can't believe I'm the only person who has.
I don't expect full API documentation and set of unit tests for every open source project - I'd be happy if most projects came with a short overview of how the code works, how it is implemented, maybe a couple of examples, and a list of its limitations. I do that for most projects I write for myself, to make it easier to come back to in a year or two when I next need to work on it.
If your project isn't worth spending an hour writing some basic documentation, then it isn't worth releasing.
Conversely, I don't even use open source software that doesn't have a simple README on Github showing me plentiful examples on how to use the damn library.
Sorry, if you don't have documentation, even a little bit, you're just not worth my time. There are at least two other libraries out there with better documentation. The fact they might be worse software doesn't even matter because all I'm looking for is a solution.
Good for you. Don't use my open source stuff. Turns out I wrote it for me, whether you get anything out of it or not doesn't actually affect me. If you do get something out of it, great. If you contribute great. But unless you are paying me, don't tell me what I should be doing with my hobby time. Period, end of story, full stop. Take your demands to someone else.
Further, demands for great documentation are unreasonable. I can point to hundreds of examples of 'well documented' projects where I still dig into the source code and do simple experiments to learn what the heck it does. This is a combination of how I learn and how I use software. Your docs (for all values of you) are crappy and don't tell me how the software works - the code does that tho, so I actually can trust it. If you want to do truely great documentation, put some comments of expected use at the top of the function/class/whatever definition, some comments on tricky sections of code (not "this does the file read" i get that from the call to read(), but "this also triggers an event from the OS handled in foohandler()), and good clean loosely coupled components.
Yeah, see, here's the thing: for many projects of noteworthy complexity, it'll be easier to gain an understanding of how to make use of it through a clear set of docs than it will be by slogging through source code.
Good documentation provides context and use cases. Good comments in source code provide clarity as to what the code is doing. They are in no way equivalent.
One of the benefits of encapsulation in software is that it enables individuals to program against a documented interface without spending (although in many cases I'd say wasting) time to understand the intricacies of implementation. You lose that benefit if you're forced to dip into code to understand how to use that code.
You do have a point, though - as an open source dev, it's your time to spend as you see fit. But a quality open-source project is more than the sum of its code.
> You do have a point, though - as an open source dev, it's your time to spend as you see fit
I agree with both of you - I don't see software as "complete" unless it has documentation, it's part of the package to me. I happen to like writing it, but I hate some other aspects of programming - doesn't mean I skip those sections.
However, I have put things online without documentation before, because the software wasn't complete but maybe someone else would complete it, or find it useful, or learn something from it. Maybe they won't - but I lose nothing by putting it online, and the world stands to gain.
This is an understatement. Even if Github is just a place to put your toy library/project and it's not being promoted, there should be an awesome README showing off what it is meant to do (even if it doesn't do that yet, just point out that those features aren't yet available). When I find engineers on Github who've contributed to projects I enjoy using and I look through their repositories to find projects with interesting names but NO README, I get annoyed. I don't mind reading code, but sometimes I just want to see what this thing does before I go about reading 2 thousand lines of code.
The worst thing that could happen is that someone would want to start using your library while it's in an alpha state, and then complain that it's not perfect. But in the end, if people would really find something you're working on useful, you might get a lot of community support and motivation to finish it, so why not?
And as you state, when features are completed, there should be pages with ample examples on how to use them. Programmers can usually read a code example ten to a hundred times faster than they can read over the documentation for everything used in that example, and in well written code, the expected functionality (that is: documented behavior) is clear from an example alone. Programmers can also write a quick code example with a few inline comments faster and better than they can write good documentation.
GitHub READMEs are an excellent compromise. Doing more detailed docs are quite a lot of effort (that is better spent coding), especially to projects that are at an early stage.
Projects which are an early stage (like most of my projects) should mostly try to attract potential contributors, not just consumers/end-users so it's not unreasonable to require would-be users/contributors to walk the extra mile and actually read (at least parts of) the source. I do that even for projects that are well established with docs if I intend to depend on them.
If there actually were libs that are well documented and do the same thing, I wouldn't have started the projects I did but contribute to the existing projects instead. This may not be true for all kinds of projects.
There's a double standard here. You want developers to contribute to your project despite the lack of documentation but you require that libraries you contribute to are well documented.
> ...but you require that libraries you contribute to are well documented.
You misread me. I almost never read the docs because they suck more often than not. I start from example and test source code and almost always end up reading parts or most of the source code.
Even very early alpha code can find its way into production systems. I'm familiar with a case involving a three-letter household name company a few years back using early releases of cassandra, as well as nginx (not entirely beta at that point, but with much of the documentation still in Russian).
Engineering making it work was a long way from operations making it reliable and understood.
I've wasted hundreds of hours dealing with piss-poor or no documentation, reading/debugging code, searching sites and forums just to find the magical combo of steps that get a application to build correctly or an LED to blink.
I don't have unlimited time to figure out your code, API and lack of documents. I would rather deal with OSS or commercial software that respects me.
2. If you're designing tools for other people to use, documentation really, really, really matters. Even if it's just a mailing list and wiki initially.
When I'm evaluating tools, I look to the docs, and if I find them lacking, my interest dims very, very rapidly. I'm a systems admin, and don't do much coding (though programmers have a need for docs as well). My main concerns are uptime, reliability, predictability, and well-understood behavior. If a tool shows a wild cowboy shoot-from-the-hip, damn the torpedoes mentality, it's going to make my life (and my sleep quantity and quality) hell.
The sweet spot, I think, is a nice "getting started" guide along with an discoverable API design. The guide gives prospective users an idea what it's like to use the library, and the discoverableness means they'll be able to figure out the more complex stuff without precarious trial-and-error or obscure doc spelunking.
I think this is all well and good, so long as you're not looking to get people to use your code.
If you want people to use your code, you need some kind of documentation. Otherwise, a lot of people (including myself) aren't even going to give you more than 30 seconds worth of time.
Honestly, I'm not sure it does. I've seen plenty of software projects whose documentation consists of an automated compendium of every method in every class, none of which tells me how to actually use it.
I'd prioritise a simple getting started guide. It only has to be a page or so, but something that explains how to run the program, and achieve a few simple tasks. It's far easier to go from a simple case to a more complicated case than it is to go from nothing to even a simple case.
To pick on a specific project, Treetop http://treetop.rubyforge.org/ has a pretty detailed set of documentation (human-generated, not automated), but I found it quite hard to go from the abstract enumeration of its features to actual working code. So hard, in fact, that I wrote up an introduction to help others, and it's been a very popular page: http://po-ru.com/diary/getting-started-with-treetop/
And please, WTFV isn't enough. I don't have time to watch or search through a 40 minute video to find out how to set a few configuration options. Text is king when it comes to documentation. Videos are fine for tutorial purposes, but I need a solid reference that's well indexed and searchable.
There are actually many open-source projects that have excellent documentation and there are many that don't have documentation at all. I think there's actually a shortage of projects with simply adequate documentation.
Many open-source projects also allow you to contribute to the documentation, so I don't think you should criticize projects you're actually using but rather that you should help maintain the documentation (even if it's just bug reports).
If you have an open-source project with very few users, you'll often find that you can't get traction simply because when people can't figure out how to use your software, they'll go elsewhere. Want fame and (maybe) fortune? Make your project useable.
I fully endorse this. Coming from a sysadmins / devopps perspective, documentation is key to providing a reliable, dependable, available, scalable service. And dramatically improving the quality of my life.
How that documentation is written also matters. A lot.
Much proprietary documentation is also crap, for numerous reasons. Marketing having too much say is key (every reference to a Trademarked(r) Name(tm) Phrase(c) is both fully expanded and badged. Might keep the marketers and lawyers happy, but it's hell to read. Descriptions are vacuous to the point of idiocy ("More magic: select this option to enable more magic") -- tells me absolutely nothing not inherent in the control, and in particular, fails to tell me what the effect of enabling "more magic" is (feature name changed, but this construct is all too common in docs).
Usage notes and examples are mandatory.
As much as Free Software docs are pilloried, I still find that they tend to compare favorably with non-free docs.
Coming from the same perspective, I agree completely.
Software pushers all seem to give the same puzzled look when I ask to see their full docs before evaluating their product.
It's exactly as you describe, if the documentation isn't sufficient for me to learn everything I could possibly need and likely want to know about how things work - I can't confidently design, build, scale or support it.
Documentation in the "Enterprise" world seems almost intentionally bad, as if to force customers into professional services and support contracts for products which lack the proper design to be sold as full on SaaS.
Yes, this. I'm looking at you, freedesktop.org... PulseAudio documentation is poor. Considering it doesn't really work out of the box on half the systems out there that's pretty bad. fontconfig docs are some of the worst I've seen in a long time. In general, I feel dizzy when I find a link with "freedesktop.org" in it labeled "docs"...
There are plenty of other examples. Seriously, documentation is extremely important. Especially if you're relying on config files, document them properly.
i'm currently writing a tutorial/manual for a library i wrote several months ago. the library took maybe 1 month to write and now the tutorial has taken 3 months. creating examples, illustrations, demos, determining and writing the sections in a sensical, progressive order takes a long time (epecially when it's done in your free time) :(, but without it, the project is DOA and no one will use it.
also, writing a tutorial has helped me refactor and decouple parts of the lib a few times to simplify and hone the API to something much more elegant than it was in the beginning. sometimes i feel like Git could have used the same kind of process to create an much more refined, wart-free API also.
To be honest, good documentation is more or less my #1 criteria for choosing which library to use.
Not only is it much easier to work with something that is well documented but I also find as a general rule well documented projects seem to be maintained for a lot longer.
I think after you write a manual for your first few projects and clearly less than 1% of your users read it, it gets hard to motivate yourself to write another manual.
The advantage of documentation isn't that users read it before asking questions.
It's that you answer questions by pointing at the relevant section of the docs.
If that section doesn't exist, it's a good practice to see that it does (either write it yourself, have another contributor write it, or encourage the person asking the question to submit a doc).
I couldn't agree more. The lack of good documentation is a plague which is not emphasized quite enough. To me it is as (or even more) important than writing tests: debugging code is easy in comparison to figuring out what it is supposed to do in the first place.
Any pointers on how to make it easier to wtfm?
If anyone out there actually enjoys writing documentation I'd be willing to pay for some help and guidance on a rather large commercial open source project that is in need of some tlc regarding documentation
In the Python world, readthedocs.org has made writing docs easier - it takes care of rebuilding documentation each time you commit. It's implemented in Python, and most popular with the Python community, but it can be used for other languages as well.
API documentation can be generated from the source code with tools like doxygen. For everything else there's the standard toolchain consisting of a good editor and source control. Writing documentation is a chore but it's a very important aspect nevertheless.
The root of this issue is that writing documentation is not something that really good hackers like to do...these days [1].
And it's damn sure not something that good writers are interested in...unless they are paid. Paying good writers to write documentation is not a core competency of the FOSS community, nor part of its ethos.[2]
[1] These days being the age of the internet and languages implementing brogrammar. People like McCarthy and Knuth wrote their own documentation to a dead tree publication standard, not a rough draft of a Wiki standard.
[2] Erlang and Go aren't going to give the world another RPG or PG writing passionately about their wonders.
One thing I've done a few times is to help out just by writing docs for open source projects.
I know it's a drag. I know it's not that much fun. But it massively - MASSIVELY - improves the project's value, especially if it's low on the totem pole.
Lisp has a literary tradition stronger than other languages. See Richard P. Gabriel, or Paul Graham for that matter. Python and Cpp may have polemics, but none as useful as Let Over Lambda.
Reminds me of the old expression: "fast, cheap, good. pick two."
When we were largely buying shrink wrapped software they almost all came with decent sized manuals (quality varied, of course). He tagged the post with a bunch of open source projects, which has almost never been an area of good manuals. It's very uncommon for "scratching your own itch" to lead to comprehensive documentation for obvious reasons. Developers seldom write the extensive manuals, tech writers do.
I think his point was more that clearly the developers want people to use what they built. So taking that extra step and writing good documentation would only help ensure that people used their software. Not doing so actually hurts their chances. Not doing so and expecting SOMEONE ELSE to do it is just dumb.
I agree with the author, I have found myself in similar situations and it is frustrating. However, I also think programmers need to RTFSC (SC == Source Code)
And please, run your source through a tool like doxygen for me to get a better understanding of how your software is built and designed. I've been advising a startup that is facing issues with documentation not being available for the platform they chose to use. The platform looks good, and the code is fairly readable, but there is no documentation. To complicate things, the platform is under active development and things change weekly (if not daily). The solution? The startup is looking to have their application written in another platform. All we need is for you to explain how stuff works in a language that even a grilled cheese sandwich will understand.
There is a big need here waiting to be satisfied. Documentation is very important and a lot of projects are losing traction due not having the basics or it being done poorly. I'm actually working with a new platform by writing their documentation for them. Why would I do that? I have a talent for it, and it allows me to offer a service other full-stack engineers won't even think about doing. Writing documentation as a job is a great way to improve my engineering skills. If I can explain it to people, then I definitely know how something works.
In it, the author speculates that one of the reasons that kids have such a hard time following directions is that they never learn to read them, because nothing comes with directions anymore (you just "figure it out"). It's sort of the dark side of ubiquitous discoverable UI.
This is one of the reasons I consider my Safari subscription indispensable.
It provides immediate access to generally well-written and structured documentation and instruction covering a wide range of topics. The one downside would be the obvious lack of bleeding edge topics as those books have yet to be written.
Good documentation is really hard to do, but it's also one of the most important things for any product or project. If no one uses what you make then it truly is useless, and you will never get people to use something unless they can figure out how.
I've been trying to get better about just helping to fix the documentation when its bad- but that is dependent on me being able to figure it out in the first place with broken/poor docs.
I'm pretty sure that if there was a manual for the software or application you are using, you wouldn't read it anyways. The software should be intuitive in the first place.
But it seems like there's a big trend now to just "get things out there" and that good documentation isn't "cool" anymore, kind of like braces in syntax.
Examples of good docs: PHP, jQuery, MySQL. (The first two sites include user comments too, which make things even more useful.)
Examples of terrible docs: Python, CoffeeScript (the worst)
I can at least understand insufficient docs for pre-1.0 versions when the implementation is changing constantly, but when something has been around for more than a year, it's just inexcusable.
I don't want a "getting started" guide that gives a bunch of examples. I don't want to type in the console to find out what methods an object has.
I want a friggin' reference manual, that includes (as applicable) syntax rules, exact rules governing whitespace, orders of operations, all functions, all parameters, parameters passed to a callback (why are these forgotten so often?), default values, flag values, all possible return values, specific exceptions that can be thrown, what input parameters result in undefined behavior.
Really, it's just not that hard. It may be grunt work, but if you'd rather make your users waste a cumulative 25,000+ hours figuring things out, rather than you spending 100 hours of your own explaining things, I just can't have respect for your product, no matter how otherwise amazing it is.