Shannon’s Theorem

The content of this chapter is rather different in nature to what appears in the rest of the book, since Shannon’s theorem is really a theorem from information theory and not coding theory. However, we include it here because it tells us that reliable communication can be achieved using a noisy channel and sets a limit for what is feasible in terms of the proportion of data we can send whilst being almost sure to be able to recover the original message from the distorted signal. Essentially, this chapter is a very brief introduction to information theory, the mathematics this entails is probabilistic in nature, whereas later it will be more algebraic and to some extent geometric. It is not essential to the rest of the text and can be treated as optional.