
More shell, less egg (2011) - michaelsbradley
http://leancrew.com/all-this/2011/12/more-shell-less-egg/
======
Animats
"Literate programming" is to some extent an artifact of Stanford's
intellectual property rules for faculty at the time. If a faculty member wrote
a program, that was the property of the university, a work for hire. But
Stanford did not insist on owning the rights to published books and papers
authored by faculty employees.

"Literate programming" was thus a way to monetize software if you were on the
Stanford faculty. (This was before professors started doing startups.)

------
blt
It's hard to figure out McIlroy's motivation for responding this way. It's
like Bentley asked Knuth to demonstrate long division, and McIlroy then wrote
an article criticizing Knuth for not using a calculator.

Yes, Knuth could have used a calculator, but the calculator still needs a
division algorithm, and `sort` probably needs a complex data structure or
algorithm. (At least if it's to be fast and work on large files.)

The only thing I can imagine, is that he wanted to advertise UNIX pipes (I
would too!) and saw an opportunity to get attention by criticizing someone
famous.

------
lisper
This is not exactly a fair comparison. The shell version works by using a rich
library. The Pascal version presumably worked ab initio. If the source code
for all the programs invoked by the shell scripts were included, Knuth's
solution might look considerably better by comparison.

~~~
vessenes
I think that's the full and whole point of the article: The unix command line
tools can be combined by a mediocre systems engineer to do the work of Knuth
in less time and with easily assessable correctness.

This is a form of wonderful magic, available only because of the discipline
and vision of those who wrote the original tools.

~~~
reagency
OK, now tell whose technique is more likely to lead to TeX or Google Brain:
Knuth's Literate Programming, or McIlroy's "everything worth doing is already
a Unix built-in"?

~~~
gnaritas
McIlroy's, because he'll finish; that's kind of the point, good engineers ship
and reuse as much as possible. Look at all the work of DJB[1], nearly
everything he does is just a bunch of well tied together shell scripts.

[1]
[https://en.wikipedia.org/wiki/Daniel_J._Bernstein](https://en.wikipedia.org/wiki/Daniel_J._Bernstein)

------
fenomas
How to eviscerate a programming technique:

1\. Think of a task that can be done in six lines of shell

2\. Ask person pushing a new programming technique to apply their idea to said
task

3\. Reply that it would have been better to just use six lines of shell

Call me cynical but it strikes me that Bentley's (edit: McIlroy's, see
replies) reply could easily have been written without seeing Knuth's code. I
don't know anything whatsoever about Literate Programming, but did Knuth mean
it to be applied to such simple tasks?

~~~
jewel
I think you're misunderstanding what happened, there are three people in the
story. Bentley asked Knuth to write a literate program to solve the problem.
He then asked McIlroy to critique the solution. McIlroy was the one that
provided the shell script code.

It's impossible to know if Bentley intentionally picked the problem that would
be easy to solve in shell script, but I don't think it's likely.

A lot of text processing problems are solved really well by the standard set
of unix tools, I write similar scripts probably about once a month to extract
counts out of log files.

~~~
fenomas
Oops, you're right - I joined them into one person. With that said, I didn't
really mean to suggest the exercise was set up to snipe at Knuth, just that
the criticism (McIlroy's) seems off the mark.

I mean, one presumes that when Knuth wants to count the words in a text file
he uses shell scripts, right? It would follow that he's only applying Literate
Programming to a toy problem for purposes of illustration. As such, saying
he's wrong to use LP for this task seems to miss the point (and doesn't say
much about LP).

~~~
KC8ZKF
I agree with you as long as we are talking about this _blog post_ and not
about Knuth's article or McIlroy's critique, which I don't have access to.

Knuth republished both, so they must have some things to say about Literate
Programming.

------
mwsherman
The key bit of why we do things like this: “useful for testing the value of
the answers and for smoking out follow-on questions”.

That statement is not about Knuth’s presumably beautiful work. It’s that the
least deterministic (and most important) part of programming is discerning the
value of doing the thing at all.

------
sthatipamala
It's pretty rare in my day-to-day work to deal with flat, string-delimited
text files.

This advice would be more actionable for me if there were more shell tools
that operated on data structures (e.g. objects represented as JSON).

~~~
conradev
Something like this?
[https://stedolan.github.io/jq/](https://stedolan.github.io/jq/)

~~~
sthatipamala
Yeah, I see JQ as the grep/sed of JSON. If there were a curl-analogue that
constructed HTTP requests from its output, it would take me a lot of the way
there.

But more generally, I'm dealing all the time with a hodgepodge of json, csv,
yaml, POM files, git logs, whatever. We need more tools that understand the
semantics of data.

------
im3w1l
Knuth’s code, and the criticism of it:
[http://onesixtythree.com/literate/literate2.pdf](http://onesixtythree.com/literate/literate2.pdf)

