An Information-Theoretic Approach to Algorithms

A quite general result, called the Leaf Entropy Theorem, is proved. This theorem gives the relation between the entropy of the leave and the branching entropies of the nodes in a rooted tree with probabilities. It is shown that this theorem can be used to bound the average number of essential tests performed by a testing algorithm for a discrete random variable. The approach is illustrated by two examples, namely, the sorting of a list and the polling of the stations in a multiple-access communications system.