
The Birth of Standard Error (2013) - bcaa7f3a8bbc
https://www2.dmst.aueb.gr/dds/blog/20131211/index.html
======
notacoward
This might have been a case of parallel invention. When I was using Michigan
Terminal System in 1981, and it wasn't new then, I remember SERCOM/ _MSINK_
being distinct from SCARDS/ _SOURCE_ and SPRINT/ _SINK_. Yes, that's cards as
in punch cards. My impression is that similar concepts and terminology have
also existed in the IBM world since $forever. I doubt that the MTS folks and
the Bell Labs folks were aware of each others' identical inventions, but if
there was a relationship I'd bet on the MTS folks having been first.

~~~
cat199
I'd posit, though without experience of contemporary systems of the era, that
the article's more correct title would be 'the birth of standard error in
Unix'..

------
tpaschalis
I'm dropping a link [1] for the same post, but on the author's actual webpage;
this mirror seems to have stopped updating circa 2016.

He's got some pretty neat stuff in there!

[1]
[https://www.spinellis.gr/blog/20131211/](https://www.spinellis.gr/blog/20131211/)

------
OliverJones
This is about Unix stderr, not about Gauss's Central Limit Theorem and related
statistical topics.

~~~
stdbrouw
For those interested in the origins of the standard error in statistics, "The
Most Dangerous Equation" by Howard Wainer is a good, short read:
[https://www.researchgate.net/publication/255612702_The_Most_...](https://www.researchgate.net/publication/255612702_The_Most_Dangerous_Equation)

~~~
divan
Thanks for posting this, wonderful read. I didn’t really get the example with
chromosomes (I would expect some references proving that X chromosome really
does contain genes linked to the intelligence), but it was an enjoyable piece
of writing.

~~~
divan
Interestingly enough, I can’t find mention of de Moivre equation anywhere
except of this article. Does anyone know its more commonly used name?

~~~
notfashion
[https://math.stackexchange.com/questions/253486/name-of-
de-m...](https://math.stackexchange.com/questions/253486/name-of-de-moivres-
equation)

As far as I can remember, Wainer doesn't really assert that the equation is
known as "de Moivre's equation", that's just what he calls it in the context
of his article.

~~~
divan
Thank you!

------
throwaway_bad
As someone who is prone to overengineering, I wonder if they also tried adding
more standard streams after that (e.g., debug or verbose, different levels you
would find in logging libraries today).

I think just out/err has been proven by history but that couldn't have been
obvious to the original designers?

~~~
Crinus
It'd be useful if there was a standardized way for applications to expose
additional streams that worked exactly like stdin/stdout/stderr but with
custom names.

At a first glance, named pipes (both the Unix and the Windows approach) sound
like that, but they are global names - i can't have an application exposing a
"commands" output stream that from the shell is piped to awk which is then
piped to a "monitor" input stream of another application - and then have
arbitrary number of instances of those.

You can ducktape a solution using randomly created unique global names and
some shell script code, but what i think would be nice is something like:

    
    
        /* app1 */
        FILE* commands = stream("commands", "w"); /* write-only stream */
        /* app2 */
        FILE* terminal = stream("terminal", "r"); /* read-only stream */
    

that could be used via something like (:foo would be used wherever a file
descriptor could be used to specify stream foo)

    
    
        # redirect commands stream from app1 to terminal stream of app2
        app1 :commands>&:terminal app2
    
        # redirect commands stream from app1 to stdout
        # (stdin of the right side which is empty, hence stdout)
        app1 :commands>&0
    
        # pipe commands to awk's stdin then grep then app2's terminal stream
        # regular app1 stdout is not affected and written wherever stdout is
        app1 :commands| awk blah | grep bleh 1>&:terminal | app2
    

the shell syntax would also need to be extended a bit to allow for parallel
pipelines (e.g. app1 could also export a "comments" stream that could be piped
to a separate file or to a projector application).

~~~
DSpinellis
See the Directed Graph Shell (dgsh), which does allow multiple streams across
commands.
[https://www.spinellis.gr/sw/dgsh/](https://www.spinellis.gr/sw/dgsh/)

~~~
Crinus
Neat but i think this would be more useful if it was a standalone utility that
did the launching itself instead of replacing your existing shell.

------
tyingq
I remember something similar where something would go wrong with lpr, and
printing a PostScript document would instead print the (really long) textual
PostScript source code.

~~~
siberianbear
lol I remember those days. It's amazing how much PostScript is required to
render a few pages of text. I remember one print job I sent that used about
200 pieces of paper after it printed the actual PostScript source instead of
parsing and rendering it for some reason.

~~~
yiyus
PostScript, as a concatenative programming language, can be used to write very
efficient code. By efficient I mean terse: doing much with very few code.
However, it's usually autogenerated, so most ps code is long and ugly (but,
obviously, good enough for it's purpose).

------
paulsutter
Be nice if they said the year of this invention, but it seems to have been in
the 70s. I tried finding earlier references, two candidates:

\- IBM JCL has DD statements for SYSERR, SYSIN, and SYSOUT, but I can't find
the date that SYSERR was introduced.

\- Any old Fortran IV programmers know that I/O unit 0 is STDERR, unit 5 is
STDIN, and unit 6 is STDOUT. And Fortran IV is from 1966 (aka "Fortran 66").

However, I found a 1970 manual for Fortran IV, and at that time, unit 0 was
illegal (see table 123-3 on page 13-4), so unit 0 must have been added later,
adding token support for the claim made in this article.

[http://www.bitsavers.org/www.computer.museum.uq.edu.au/pdf/D...](http://www.bitsavers.org/www.computer.museum.uq.edu.au/pdf/DEC-10-LFLMA-
B-D%20FORTRAN%20IV%20\(F40\)%20Programmer%27s%20Reference%20Manual.pdf)

~~~
cjsplat
In the early days, stdin and stdout was a convention of the shell, not the
operating system. Eg Programmers Workbench circa 1977: see execute() in
[https://minnie.tuhs.org/cgi-
bin/utree.pl?file=PWB1/sys/sourc...](https://minnie.tuhs.org/cgi-
bin/utree.pl?file=PWB1/sys/source/s2/sh.c)

In Version 7 (1979), it moved to the kernel - see setregs() in
[https://minnie.tuhs.org/cgi-
bin/utree.pl?file=V7/usr/sys/sys...](https://minnie.tuhs.org/cgi-
bin/utree.pl?file=V7/usr/sys/sys/sys1.c)

The V7 kernel model makes it generic - on exec, Unix will close() any file
that has the close_on_exec flag set.

I believe that is the critical concept - stderr is supported by an OS level
feature and inherits across various processes through pipes, forks and execs.

To tie back to the initial case in this thread, this is very useful for the
Unix print system because it was/is a maze of squirrelly processes.

~~~
paulsutter
Right and the big question is whether unix was first to have stderr, or if it
was predated on other OS’s.

------
AlexCoventry
Oh, _that_ standard error. I assumed it was about the statistical concept.
Might be helpful to amend the title so it specifies that.

------
teddyh
Ah, the beauty of dogfooding, a.k.a. bootstrapping, as Douglas Engelbart
called it¹.

1\.
[https://www.youtube.com/watch?v=agdPQuFr0yg](https://www.youtube.com/watch?v=agdPQuFr0yg)

------
jgalt212
Somewhat related, and an excellent view:

The Great 202 Jailbreak - Computerphile

[https://www.youtube.com/watch?v=CVxeuwlvf8w](https://www.youtube.com/watch?v=CVxeuwlvf8w)

------
faehnrich
I once saw a video of someone describing this, can't for the life of me find
it again.

