Classic Maximum Entropy

This paper presents a fully Bayesian derivation of maximum entropy image reconstruction. The argument repeatedly goes from the particular to the general, in that if there are general theories then they must apply to special cases. Two such special cases, formalised as the “Cox axioms “, lead to the well-known fact that Bayesian probability theory is the only consistent language of inference. Further cases, formalised as the axioms of maximum entropy, show that the prior probability distribution for any positive, additive distribution must be monotonic in the entropy. Finally, a quantified special case shows that this monotonic function must be the exponential, leaving only a single dimensional scaling factor to be determined a posteriori. Many types of distribution, including probability distributions themselves, are positive and additive, so the entropy exponential is very general.

[1]  B. Frieden.  Restoring with maximum likelihood and maximum entropy. , 1972, Journal of the Optical Society of America.

[2]  S. F. Gull,et al.  The Maximum Entropy Method , 1979 .

[3]  E. T. Jaynes,et al.  Where do we Stand on Maximum Entropy , 1979 .

[4]  C. V. Schooneveld.  Image Formation from Coherence Functions in Astronomy , 1979 .

[5]  Rodney W. Johnson,et al.  Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy , 1980, IEEE Trans. Inf. Theory.

[6]  E. T. Jaynes,et al.  Papers on probability, statistics and statistical physics , 1983 .

[7]  Y. Tikochinsky,et al.  Consistent inference of probabilities for reproducible experiments , 1984 .

[8]  J. Justice.  Maximum entropy and bayesian methods in applied statistics , 1986 .

[9]  E. T. Jaynes,et al.  Maximum Entropy and Bayesian Methods in Applied Statistics: Monkeys, Kangaroos, and N , 1986 .

[10]  R. Levine,et al.  Geometry in classical statistical thermodynamics , 1986 .

[11]  G. Klir.  Where do we stand on measures of uncertainty, ambiguity, fuzziness, and the like? , 1987 .

[12]  C. Ray Smith,et al.  Maximum-entropy and Bayesian methods in science and engineering , 1988 .

[13]  J. Skilling.  The Axioms of Maximum Entropy , 1988 .

[14]  C. Ray Smith,et al.  From Rationality and Consistency to Bayesian Probability , 1989 .

[15]  Carlos C. Rodriguez,et al.  The Metrics Induced by the Kullback Number , 1989 .

[16]  Stephen F. Gull,et al.  Developments in Maximum Entropy Data Analysis , 1989 .

[17]  R. T. Cox.  Probability, frequency and reasonable expectation , 1990 .