

Measuring the Entropy of the MOS 6502 CPU - daniel02216
http://www.pagetable.com/?p=401

======
ced
There's a bit of a conceptual problem with trying to _measure_ entropy. That's
because entropy is based on probabilities, and probabilities are _subjective_.
What are "the" odds that my brother will pass his SAT exam? My brother can
make the most confident prediction (lowest entropy[1]). But the reader of this
comment can do no better [2] than take the generic SAT success rate (higher
entropy). And some peasant in Tuvalu might as well say 1/2 (maximum entropy).

So entropy is subjective. It's a measure of how much _new information_ you
expect to get from a message. The more you know already, the lower the
entropy.

EDIT: An awesome quote from Jaynes.

"Probability assignments are not assertions of any physical property; they are
a description of a _state of knowledge_. It is therefore illogical to speak of
"verifying" the probabilities by performing experiments on the physical world.
That would be like trying to verify a boy's love for his dog by performing
experiments on the dog"

[1] Not necessarily true, see Wiki or David MacKay's wonderful book for
details. <http://www.inference.phy.cam.ac.uk/mackay/itila/>

[2] Also not strictly true! You might argue that his passing the SAT should
correlate with my passing it, and estimate what my odds would be/were from my
comment history.

------
phaedrus
I wondered how it would compare if you measure this from the gate level rather
than the transistor level. To check this, I looked online for a Verilog file
describing a 6502 core. This site has one:

<http://www.birdcomputer.ca/Projects/Prj6502/bc6502_page.html>

The zipped size of that file is 38 kilobytes, which is actually larger than
the approximately 7 kilobytes guessed in the article for transistor level
information. That is probably due to the verilog code being more verbose; you
would think zipping it would make up the balance but apparently not. However,
even though 38K >> 7K, I think the fact that the two are at least in the same
ballpark is encouraging.

