Information theory

Summary

 * Q: How do we formalize notions of information and surprise?
 * Ideas:
 * Entropy, conditional entropy, mutual information
 * Source coding and simple bounds
 * Channel coding
 * Exercises:
 * Problems solvable by information theory
 * Conceptual problems

Logistics

 * Time: ~ 1.5 hours
 * Teacher: ?
 * Relations:
 * Requires basic probability
 * Pairs with unit on using words