## TL;DR

Information theory (Claude Shannon, 1948) quantifies information, establishing the theoretical limits of data compression and reliable communication. Entropy H = -Σ p(x) log p(x) measures uncertainty. Mutual information measures shared information between variables.

## Core Explanation

Shannon's source coding theorem: data can be compressed to entropy bits per symbol (lossless limit). Channel capacity: maximum rate of reliable communication over a noisy channel. Applications: data compression (Huffman coding, LZ77/ZIP), error-correcting codes (Reed-Solomon, LDPC, Turbo), cryptography (entropy of keys). Shannon invented the bit as a unit of information.

## Further Reading

- [undefined](undefined)