
Robert Fano has died - tjr
http://www.eecs.mit.edu/news-events/media/robert-fano-computing-pioneer-and-founder-csail-dies-98
======
hcs
I've never been able to find an electronic copy of his 1949 report "The
Transmission of Information", it's missing from DSpace@MIT, so here's a scan:

[https://archive.org/details/fano-tr65.7z](https://archive.org/details/fano-
tr65.7z)

This was an original conception of information theory, somewhat beat to the
press by Shannon, though the two had talked about it (and indeed Shannon cites
this in "A Mathematical Theory of Communication").

I have a copy of TR 149, Transmission of Information II, as well, but no
scanner anymore.

~~~
jakub_h
"I have a copy of TR 149, Transmission of Information II, as well, but no
scanner anymore."

How delightfully ironic! ;)

------
cvwright
Among other things, he was known for Fano's Inequality, which gives a limit
for how well you can predict the value of a random variable based on knowledge
of a related variable. Apparently this is quite useful in signal processing
for communications over a noisy channel, where the signal that you receive is
not quite the same as what was sent.

[https://en.wikipedia.org/wiki/Fano%27s_inequality](https://en.wikipedia.org/wiki/Fano%27s_inequality)

~~~
hyperbovine
Very useful in statistics too: if you can show that your estimator attains the
Fano lower bound rate of convergence then it's optimal--nothing else can
possibly beat it (in the minimax sense anyways).

~~~
thearn4
Though I'm not a stats researcher, that sounds very similar to the Cramer Rao
bound (a lower limit on the variance of any unbiased estimator, something akin
to the Cauchy-Schwartz inequality only for point estimation).

~~~
gajjanag
That is good intuition, but note that one of the general drawbacks of Cramer
Rao is that it applies for the unbiased case only (technically no, there is a
version for the biased case, I have never seen it used though), and that too
only well under certain regularity conditions. Indeed, there is a term for
estimators where equality is met in CR, they are called "efficient"
estimators.

There is significant research effort these days on going beyond unbiasedness
and asymptotics, which in many cases has been beaten to death and is
"classical", though there are still plenty of questions.

Here is a simple illustration of unbiasedness/going beyond it. Consider n iid
(independent and identically distributed) coin flips of coin of P(heads) = p,
and based on the observations, we want to estimate p, call the estimate p'.
Optimum estimate in the sense of minimax - min (over estimators) max (over p
in [0, 1]) E[(p-p')^2] is not the simple maximum likelihood which is the
sample average. Sample average yields a score of (1/4) _(1 /n), while optimum
is (1/4)_(1/(sqrt(n)+1)^2), which is slightly lower. It is natural to wonder
why people care, typically the motivation is in high dim statistics where
there is dependence on the dimension as well. In such a case fine grained
characterization of performance sometimes affects the scaling in terms of the
target loss amount and dimension, i.e the asymptotics now has at least 2
parameters, sometimes more.

My personal unhappiness with these problem formulations is their lack of
robustness to transformations - suppose I now want to estimate p^3 instead of
p, the optimum min-max will change in most cases, and definitely does here.
Plugging in the sample average and then cubing is far more intuitive, and
often works fine even though it is not optimal. There are more extreme
examples of this, but that will take the thread too far afield.

If one is interested in the topic of biased estimators, I suggest reading
about James-Stein estimators and more generally shrinkage estimators.

~~~
thearn4
Good read, I've been meaning to make more time for reading up on the state of
biased-but-low-variance estimation.

------
ChuckMcM
I did not realize he had developed CTSS. Folks in the AI lab developed ITS
which was the "Incompatible Timesharing System" that had a user interface
which was a REPL to the debugger (DDT at the time). They gave out 'tourist'
accounts for anyone who wanted one and I used mine and met several people that
way.

It sounds like he was way ahead of his peers in understanding the privacy
aspects of future computers.

------
dorianm

        In those days [batch processing] programmers never even
        documented their programs, because it was assumed that
        nobody else would ever use them. Now, however, time-sharing
        had made exchanging software trivial: you just stored one
        copy in the public repository and therby effectively gave it 
        to the world. Immediately people began to document their
        programs and to think of them as being usable by others.
        They started to build on each other's work.
    

Robert Fano – Scientist

[http://www.inspiringquotes.us/quotes/pCID_em9jKLpk](http://www.inspiringquotes.us/quotes/pCID_em9jKLpk)

------
j2kun
Also notable as the son of Gino Fano:
[https://en.wikipedia.org/wiki/Gino_Fano](https://en.wikipedia.org/wiki/Gino_Fano)

~~~
Raphmedia
You people might enjoy
[http://www.genealogy.ams.org/](http://www.genealogy.ams.org/) then.

~~~
j2kun
Yes, I'm in that tree.

~~~
NobleSir
just looked myself up through that link, and apparently, me too!

------
jpmattia
In addition to the other contributions listed, we should add the
Fano/Adler/Chu E&M series, which was one of the touchstones for an earlier gen
of students. RIP

------
jmspring
Another passing in my college lineage. Studied under Glem Langdon (passed a
few years back), Huffman (a few years prior to that), I recall Shannon
passing.

Glen Langdon and David Huffman I knew from UC Santa Cruz. Shannon and Fano, I
only heard stories about. Jorma Rissanen is still alive at the age of 98.

Pillars in compression and information theory.

~~~
jmspring
And of course Huffman's breakthrough came while taking a class from Robert
Fano -
[https://en.m.wikipedia.org/wiki/Huffman_coding](https://en.m.wikipedia.org/wiki/Huffman_coding)

One of my favorite memories of David was how he got grumpy about being best
known for that term paper. He had done much more.

(Sorry for side track memory)

------
cia48621793
Rest in peace, the father of mainframes.

~~~
cpr
Perhaps better, the father of timesharing.

------
chronic81
I just wanted to chime in and hopefully prevent selection bias (for posting)
and say:

Who?

~~~
tjr
While I do imagine that Robert Fano's name is familiar to many who have
studied computer science, the original article title (which I submitted) was a
bit more descriptive; the HN moderators changed it.

