

Literate Programming (and why Docco isn't enough) - steveklabnik
http://timeless.judofyr.net/literate-programming.html

======
tristan_juricek
I've messed around with my own literate programming system that was a more
"true" system (where you have a kind of macro-like approach), and I have to
agree: the docco approach is missing the key bonus: organizing your code into
chapters and paragraphs and then statements.

Think of it this way, if you're building an API, you should consider usage. So
you then generate examples. But not just one, maybe 3-5 of them to really do
it right.

Then, as you proceed, you implement and build tests, probably down towards the
end of "the chapter".

It's a different way of thinking about how your code is consumed. The
"documentation systems" of the past (e.g., javaDoc) really are reference
documentation on details of a method. But if you want to really grasp the
system, you probably lead with examples. Maybe a picture. And you probably
don't need to have to navigate to 7 different implementations of your
interface Foo to figure out what the design is.

But having those 7 different implementation files is nice when you know the
system later.

Where literate programming systems completely break down is the toolset. You
typically can't work in "example down" mode and "single source file" mode at
the same time - like, to debug, or make a small change. I've tried to build
"diff and patch" between the source and the literate documentation, and when
that's buggy, whew, that sucks.

------
evanrmurphy
One idea about design I take from pg is that it's difficult to "even talk
about good or bad design except with reference to some intended user." [1]

Now, I'm relatively new to literate programming, and perhaps the answer to my
question(s) will be obvious to someone more experienced with it. What I wonder
though every time I read about it is, who is the intended audience of all this
extensive documentation? More specifically, how much knowledge of the
programming language and problem space do we presume that the reader has?

There is something elegant about presuming the audience has expertise in both
areas. That way you only have to worry about documenting one thing, the
program, rather than three things: the program, the programming language and
the problem space. Readers lacking sufficient understanding of the lang or
problem space can refer to the more robust documentation about either one
individually to get themselves up to speed. But I'm not sure Docco users are
taking this approach.

\---

[1] <http://www.paulgraham.com/desres.html>

~~~
steveklabnik
Personally, I try to write at the same level that I always would for code,
with maybe a larger preamble at the beginning.

Then again, I'm still a student of the ways of the literate myself...

------
bguthrie
Although I enjoy jashkenas' Javascript documentation a great deal and find it
extremely helpful, I'm having a hard time getting excited by the resurrection
of interest in literate programming. I enjoy and appreciate the notion that
code should be designed by humans to read, but I'm reminded of (and agree
with) Bob Martin's admonition in Clean Code that energy spent polishing one's
code commentary is better spent improving the readability and flow of the code
itself.

Also, comments that come across elegant and attractive when rendered in HTML
look teeth-gratingly gratuitous when viewed in the context of the original
code. For example,

    
    
      // Attach all inheritable methods to the SomeObject prototype.
      _.extend(MyLib.SomeObject.prototype, MyLib.SomeMixin.prototype, { ... })
    

Yes, that's an accurate description of what it does. But if you can read
Javascript, you should know that; in code, it's visual clutter, and worse, it
adds overhead to maintain it when you move or update your code.

~~~
jashkenas
Redundant comments are as annoying as repeating yourself in writing.

Redundant comments are as annoying as repeating yourself in writing (and I'm
certainly as guilty of it as anyone). But the real point of the comments is to
explain the "why", just as the code explains the "how". In your example above,
to explain the reason that you've chosen to mix in the modules you're mixing
in...

~~~
bguthrie
I was trying to be subtle about it, but the example I used was drawn from a
fairly well-known (albeit, I should add, well-written) JS library. My point is
that this style of documentation encourages that behavior, perhaps
paradoxically because it's so attractive and big hunks of whitespace tend to
stand out.

------
jashkenas
Mr. Klabnik goes in depth into the argument that literate programming has to
be tangled and weaved into a different order, in order to be considered "true"
literate programming.

Respectfully, I have to disagree.

It may have been the case that a macro source-code rewriter was necessary
before the advent of the subroutine, but in this day and age you can factor
out functions as you see fit. Moving around source code with macros is an
unnecessary obfuscation between the code you read, and the code as it runs. At
any point where you're tempted to pull out a block of code and interpolate it
later, just make it a function with a descriptive name, and hey presto.

Edit: Steve wrote it.

~~~
jmillikin
> _Mr. Klabnik goes in depth into the argument that literate programming has
> to be tangled and weaved into a different order, in order to be considered
> "true" literate programming._

> _Respectfully, I have to disagree._

If you're not re-ordering the code, there's no difference between a "literate
program" and simple comments.

And I find code-reordering useful even in modern languages; most impose some
structure like:

    
    
      1. module-level documentation
      2. exports
      3. imports / includes
      4. implementation (functions, procedures, etc)
         4.1 return type (static languages only)
         4.2 function/procedure documentation
         4.3 code
    

With literate programming, you can add supporting text (like export entries,
module imports, or documentation) wherever would make the most sense for a
human reader, not to the compiler.

> _At any point where you're tempted to pull out a block of code and
> interpolate it later, just make it a function with a descriptive name, and
> hey presto._

What if you've got a few dozen local variables? Do you want to pass them all
as parameters, for every single logical block in your algorithm?

What sort of names do you give your extracted procedures? Literate programs
can have block names like "Check for common post-traversal error conditions",
which in a procedure name would likely be abbreviated to
"traversed_errcheck()".

~~~
jules
It seems that folding code to a natural language summary is what you really
want.

~~~
jmillikin
Not at all; I want to be able to print stuff like this in the generated
documentation:

\----------------------------------------------

3.2 HTTPS driver

The HTTPS driver is currently implemented using either GNU TLS or OpenSSL:

== driver includes

    
    
      #include <gnutls.h>
      #include <openssl.h>
    

== driver list

    
    
      , {"HTTPS (GNU TLS)", driver_https_gnutls}
      , {"HTTPS (Open SSL)", driver_https_openssl}
    

== drivers.c

    
    
      static void
      driver_https_gnutls(DriverState *s) {
        ||common driver local storage||
        gnutls_init();
        gnutls_ctx_init(&ctx);
        check_extra_algos = "FOO:BAR";
        some_feature_flag = 1;
        // etc
      }
    
      static void
      driver_https_openssl(DriverState *s) {
        ||common driver local storage||
        openssl_init();
        check_extra_algos = "BAZ";
        some_feature_flag = 0;
        // etc
      }
    

\----------------------------------------------

~~~
jules
I meant instead of pulling out a block of code and interpolating it later. For
example instead of:

    
    
        <set up>
        <do stuff>
        <tear down>
    

And then describing them in the rest of the document, a better solution to me
seems to be able to click on e.g. the <do stuff> part to expand it. This
allows the reader to see the high level structure and drill down to the stuff
he's interested in instead of spreading the code all over the place with
references.

~~~
steveklabnik
That might be an 80% solution, but it still means that you're tying your
presentation to the way that the computer needs to have things ordered. What
if all of the parts need explained, but I really want to explain the <do
stuff> before the <set up> and <tear down>?

------
jashkenas
A bit more Twitter conversation here, for the curious:

<http://bettween.com/jashkenas/mentalguy>

~~~
steveklabnik
Man, I really wish that this tool showed stuff in forwards order rather than
backwards order. And when I tried to flip it around, it showed a bunch of your
previous conversations first instead...

------
pandakar
org-mode in emacs with 'org-babel tangle' works well. But Docco looks great on
the web.

