Measuring Information

How much information have you got?

 

Quantity surveyors in information space:

Jane Austen's Emma

Compare these two versions:

File statistics:

-rwxr--r--    1 afb21     68683 Feb  7 11:41 emma.txt
-rwxr--r--    1 afb21    118039 Feb  7 11:41 emma-by-numbers.txt

Compressed files:

-rwxr--r--    1 afb21     25505 Feb  7 11:41 emma.txt.gz
-rwxr--r--    1 afb21       678 Feb  7 11:41 emma-by-numbers.txt.gz
  

How much information is in graphics?

 

Compare these:

Square A:

File statistics:

-rwxr--r--    1 afb21       433 Jan 29 17:30 by-tiny.gif

Square B:

File statistics:

-rwxr--r--    1 afb21      5413 Jan 29 17:33 by-text.gif
-rwxr--r--    1 afb21       433 Jan 29 17:30 by-tiny.gif

Square C:

File statistics:

-rwxr--r--    1 afb21     36573 Jan 29 17:37 by-noisy.gif
-rwxr--r--    1 afb21      5413 Jan 29 17:33 by-text.gif
-rwxr--r--    1 afb21       433 Jan 29 17:30 by-tiny.gif
  

How much information is in photographs?

 

Compare these:

 

File statistics:

-rwxr--r--    1 afb21    104931 Jan 29 17:52 Rachel-hires.jpg
-rwxr--r--    1 afb21     30850 Jan 29 17:52 Rachel-lores.jpg

Where did the information go?

  

What do these measures have in common?

 

Claude Shannon (again)

1948:

A Mathematical Theory of Communication.
Bell System Technical Journal, Vol. 27 (July and October 1948), pp. 379-423

All information can be described in terms of communication (or storage) requirements. These are absolute limits - apply to the world, and our bodies, as well as in computers. (But accurate measures, fixed limits, and exact comparison make them more obvious there).