Computer Solution of the Discrete Maximum Entropy Problem

Consider an experiment with a finite number of outcomes. Suppose, for example, we are given the following data on the yield strength of a Bofors steel [8], [9]: Suppose also that we know the means of certain random variables relating to our experiment; in the above example we might calculate the mean and variance of the observed outcomes: ,/ = 35.6, o2 = 4.19. On the basis of this information, how should we choose a probability distribution P to best estimate the unknown probability measure P* underlying our experiment? There is, of course, no set solution to this problem. The maximum entropy method, however, is particularly appealing and has received considerable attention in recent years. Introduced by Shannon [7] in connection with communication theory, entropy was given an information theoretic interpretation first formulated by Jaynes [4] and further developed by Tribus [8]. According to Jaynes, P should be chosen to maximize the entropy E: