Partially-Finite Programming in L1 and the Existence of Maximum Entropy Estimates

Best entropy estimation is a technique that has been widely applied in many areas of science. It consists of estimating an unknown density from some of its moments by maximizing some measure of the entropy of the estimate. This problem can be modelled as a partially-finite convex program, with an integrable function as the variable. A complete duality and existence theory is developed for this problem and for an associated extended problem which allows singular, measure-theoretic solutions. This theory explains the appearance of singular components observed in the literature when the Burg entropy is used. It also provides a unified treatment of existence conditions when the Burg, Boltzmann-Shannon, or some other entropy is used as the objective. Some examples are discussed.

[1]  Charles A. Micchelli,et al.  ConstrainedLp approximation , 1985 .

[2]  C. Ray Smith,et al.  Maximum-entropy and Bayesian methods in science and engineering , 1988 .

[3]  S. F. Gull,et al.  The Entropy of an Image , 1985 .

[4]  C. R. Smith,et al.  Maximum-Entropy and Bayesian Methods in Inverse Problems , 1985 .

[5]  J. P. Burg,et al.  Maximum entropy spectral analysis. , 1967 .

[6]  Michael M. Fitelson,et al.  Notes on maximum-entropy processing (Corresp.) , 1973, IEEE Trans. Inf. Theory.

[7]  S. Kay.  Spectral estimation , 1987 .

[8]  Adrian S. Lewis,et al.  Moment-Matching and Best Entropy Estimation , 1994 .

[9]  Didier Dacunha-Castelle,et al.  Maximum d'entropie et problème des moments , 1990 .

[10]  R. Rockafellar,et al.  Duality and stability in extremum problems involving convex functions. , 1967 .

[11]  G. J. Janacek,et al.  Maximum Entropy and Bayesian Spectral Analysis and Estimation Problems , 1987 .

[12]  J. McClellan,et al.  Multidimensional MEM spectral estimation , 1982 .

[13]  Klaus Glashoff,et al.  Linear Optimization and Approximation , 1983 .

[14]  Marc Teboulle,et al.  A dual approach to multidimensional L p spectral estimation problems , 1988 .

[15]  L. D. Irvine,et al.  Constrained interpolation and smoothing , 1986 .

[16]  Fabrice Gamboa,et al.  Méthode du maximum d'entropie sur la moyenne et applications , 1989 .

[17]  Jonathan M. Borwein,et al.  A simple constraint qualification in infinite dimensional programming , 1986, Math. Program..

[18]  E. Anderson.  Linear Programming In Infinite Dimensional Spaces , 1970 .

[19]  John W. Woods,et al.  Two-dimensional Markov spectral estimation , 1976, IEEE Trans. Inf. Theory.

[20]  Jonathan M. Borwein,et al.  Convergence of Best Entropy Estimates , 1991, SIAM J. Optim..

[21]  I. Zucker,et al.  Extended Watson integrals for the cubic lattices. , 1977, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Edward K. Wong,et al.  A survey on the maximum entropy method and parameter spectral estimation , 1990 .

[23]  J. Navaza,et al.  The use of non‐local constraints in maximum‐entropy electron density reconstruction , 1986 .

[24]  W. J. Studden,et al.  Tchebycheff Systems: With Applications in Analysis and Statistics. , 1967 .

[25]  J. Shore,et al.  Which is the better entropy expression for speech processing: -S log S or log S? , 1984 .

[26]  L. Mead,et al.  Maximum entropy in the problem of moments , 1984 .

[27]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[28]  Claude Lemaréchal,et al.  Dual Methods in Entropy Maximization. Application to Some Problems in Crystallography , 1992, SIAM J. Optim..