Consider an experiment with a finite number of outcomes. Suppose, for example, we are given the following data on the yield strength of a Bofors steel [8], [9]: Suppose also that we know the means of certain random variables relating to our experiment; in the above example we might calculate the mean and variance of the observed outcomes: ,/ = 35.6, o2 = 4.19. On the basis of this information, how should we choose a probability distribution P to best estimate the unknown probability measure P* underlying our experiment? There is, of course, no set solution to this problem. The maximum entropy method, however, is particularly appealing and has received considerable attention in recent years. Introduced by Shannon [7] in connection with communication theory, entropy was given an information theoretic interpretation first formulated by Jaynes [4] and further developed by Tribus [8]. According to Jaynes, P should be chosen to maximize the entropy E:
[1]
Claude E. Shannon,et al.
A Mathematical Theory of Communications
,
1948
.
[2]
C. E. SHANNON,et al.
A mathematical theory of communication
,
1948,
MOCO.
[3]
W. Weibull.
A Statistical Distribution Function of Wide Applicability
,
1951
.
[4]
E. Jaynes.
Information Theory and Statistical Mechanics
,
1957
.
[5]
Aleksandr Yakovlevich Khinchin,et al.
Mathematical foundations of information theory
,
1959
.
[6]
G. Zoutendijk,et al.
Methods of feasible directions : a study in linear and non-linear programming
,
1960
.
[7]
O. L. Mangasarian,et al.
Linear and Convex Programming
,
1966
.
[8]
M. Tribus.
Rational descriptions, decisions, and designs
,
1969
.
[9]
L. Campbell.
Equivalence of Gauss's Principle and Minimum Discrimination Information Estimation of Probabilities
,
1970
.