
Literate Programming: Retrospect and Prospects - fogus
http://lambda-the-ultimate.org/node/3663
======
psranga
One thing really stood out for me from the book "Programmers at Work".

Many of the programmers repeatedly emphasized the importance of data
structures to both understanding programs as well to solving the problem. And
after noticing how I work, I find that to be true.

English documentation of the core data structures is more important than
verbose description of the steps in the algorithm. This is the key thing that
is different among progarms. By and large, we use the same algorithms again
and again. And I can trivially find English documentation of those to
understand rapidly. I don't need it in the code also.

And by documentation I mean something that explains at a programming language
level with the identifiers etc. For git, it would be insufficient to say
"trees are collections of blobs". I would need to know the struct names and an
explanation of each of the members. And whether the "collection" is a
array/list/hash etc.

Having looked into literate programming, it seems to emphasize documenting the
algorithms more than it does the data structures. Hence it misses the "big
picture" which is what is really needed.

I'm not convinced that LP is useful.

~~~
jsyedidia
Look at the gb_graph module of Knuth's "Stanford GraphBase" (the Stanford
GraphBase is available for download at his web-site). His explanation of his
graph data structures, written in CWEB, are the clearest documentation of a
complex data structure that I have ever seen.

------
jsyedidia
I use literate programming, and Knuth's CWEB in particular, and I find it fits
for me perfectly. Honestly, I think the biggest problem is that most
programmers have never heard of it or think it's something like JavaDoc. I
find it astonishing that so few people are willing to try something developed
and strongly advocated by the man who single-handedly programmed TeX,
MetaFont, and the Stanford GraphBase, and wrote the most respected texts in
computer science.

------
chipsy
I recently started using Leo for a combination of outlining, literate
programming, and general project management.

I don't go as far as Knuth and try to formulate a complete narrative of my
program, but Leo's outlining features let me clone and regroup blocks of code
on a task-by-task basis and document: "In these blocks, I'm implementing
feature x," or "fixing bug y." It's very useful for getting the big picture
and making sense of problems in the codebase. When using only a code editor,
you waste a lot of time jumping through the project and searching through
files, and I don't have that problem anymore.

------
jeremyw
A fun reminder that where most programmers (I suspect) are of the iterative
sort -- especially where the language is concise and testable -- Knuth is an
inveterate planner. If you've trained yourself to strategize mathematically
and execute long streams of pseudocode in your head, a word processor is your
friend. :)

That said, I've long wanted more expressive commentary in the trickier parts
of the codebase. i.e. some kind of comment xml that IDEs make into a rich text
area, for the diagrams and maths and whatnot.

~~~
stcredzero
Why not do the entire codebase as some sort of nested structure, designed only
to be presented inside an IDE? So not only comments, but types, namespaces,
every bit of programming metadata there is.

Many programmers feel insecure abandoming their text editor -- because this is
something they're accustomed to and which they understand conceptually. Some
sort of nested structure expressed in a markup language like XML would be just
as understandable conceptually. Many of the incompatibilities between
languages would disappear if there was a widely used standard for such a code
representation.

~~~
jeremyw
Flat files continue to have a simplicity, portability and fungibility that is
difficult (if not folly! :) to walk away from.

Instead, there are opportunities to enhance things in ways the are still
perfectly useful in emacs/vi.

~~~
stcredzero
As a user of Smalltalk for almost a decade, let me assure you that walking
away from source files is _not_ folly! In fact, I get something which is just
as nimble and reliable, and yet even more powerful. I can write snippets of
code that act like queries into my codebase. With a short script I can rewrite
broad swaths of my code with syntactic precision, and with a little work be
assured that I have done a perfect refactoring. I can undo/redo all changes. I
can even do dangerous things that even bring the VM to a screeching halt, and
_not lose any code_. All of this is enabled by giving up source files and
going to a "change log" which is like a transactional log of my changes.

You can think of this as something like going from ext2 to ext3, then onto
ZFS.

------
jerf
One other point I think is critical to understanding its lack of lift is that
it solves some problems most modern languages don't have with code
organization. Earlier languages were much more restrictive about how you could
organize code, and giving the programmer the ability to rearrange code in a
more flexible manner could be a genuine advantage. Nowadays our problems lie
more in unwise use of our extensive ability to organize and reorganize code,
rather than our inability to do so at all, and adding in "conventional" LP
only gives us another dimension to screw up on.

I do not believe this is the entire story, but it's an important part of it.

------
spitfire
Where can I get this penelope system? It sounds interesting and I am an Ada
programmer. Anything that can make software more reliable is a good thing,
imho.

Incidentally I've used cweb before and can second their comments about it.
It's a fantastic idea, that needs a bit of a refresh. Particularly getting
away from simple commenting of how things are done to the why/data
structure/higher level arena.

------
amichail
I wonder how well Google Wave would work as a real-time collaborative literate
programming tool.

