Maximum Entropy and the Nearly Black Object

SUMMARY Maximum entropy (ME) inversion is a non-linear inversion technique for inverse problems where the object to be recovered is known to be positive. It has been applied in areas ranging from radio astronomy to various forms of spectroscopy, sometimes with dramatic success. In some cases, ME has attained an order of magnitude finer resolution and/or an order of magnitude smaller noise level than that obtainable by standard linear methods. The dramatic successes all seem to occur in cases where the object to be recovered is 'nearly black': essentially zero in the vast majority of samples. We show that near-blackness is required, both for signal-to-noise enhancements and for superresolution. However, other methods-in particular, minimum /1-norm reconstruction-may exploit near-blackness to an even greater extent.

[1]  I. Good.  Maximum Entropy for Hypothesis Formulation, Especially for Multidimensional Contingency Tables , 1963 .

[2]  L. Brown.  Admissible Estimators, Recurrent Diffusions, and Insoluble Boundary Value Problems , 1971 .

[3]  B. Roy Frieden,et al.  Restoring with Maximum Entropy, II: Superresolution of Photographs of Diffraction-Blurred Impulses* , 1972 .

[4]  W. Pelz.  Topics on the estimation of small probabilities. , 1977 .

[5]  R. Tapia,et al.  Nonparametric Probability Density Estimation , 1978 .

[6]  S. Gull,et al.  Image reconstruction from incomplete and noisy data , 1978, Nature.

[7]  N. Sloane,et al.  Hadamard transform optics , 1979 .

[8]  E. T. Jaynes,et al.  Where do we Stand on Maximum Entropy , 1979 .

[9]  Mark A. Lukas,et al.  The application and numerical solution of integral equations , 1980 .

[10]  G. Wahba.  Ill Posed Problems: Numerical and Statistical Methods for Mildly, Moderately and Severely Ill Posed Problems with Noisy Data. , 1980 .

[11]  Rodney W. Johnson,et al.  Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy , 1980, IEEE Trans. Inf. Theory.

[12]  I. Good,et al.  Density Estimation and Bump-Hunting by the Penalized Likelihood Method Exemplified by Scattering and Meteorite Data , 1980 .

[13]  I. J. Good,et al.  Recent Advances in Bump Hunting , 1981 .

[14]  Grace Wahba,et al.  Constrained Regularization for Ill Posed Linear Operator Equations, with Applications in Meteorology and Medicine. , 1982 .

[15]  Mario Bertero,et al.  Resolution in Diffraction-limited Imaging, a Singular Value Analysis: III. The Effect of Sampling an , 1982 .

[16]  Mario Bertero,et al.  Resolution in Diffraction-limited Imaging, a Singular Value Analysis: I. The Case of Coherent Illumi , 1982 .

[17]  P. Bickel.  MINIMAX ESTIMATION OF THE MEAN OF A NORMAL DISTRIBUTION SUBJECT TO DOING WELL AT A POINT , 1983 .

[18]  Y. Tikochinsky,et al.  Alternative approach to maximum-entropy inference , 1984 .

[19]  D. Titterington,et al.  The maximum entropy method for data analysis , 1984, Nature.

[20]  P. Jansson.  Deconvolution : with applications in spectroscopy , 1984 .

[21]  John Skilling,et al.  Data analysis: The maximum entropy method , 1984, Nature.

[22]  J. Skilling,et al.  Maximum entropy image reconstruction: general algorithm , 1984 .

[23]  Y. Tikochinsky,et al.  Consistent inference of probabilities for reproducible experiments , 1984 .

[24]  J. Skilling,et al.  Maximum entropy signal processing in practical NMR spectroscopy , 1984, Nature.

[25]  Richard Barakat,et al.  Algorithms for reconstruction of partially known, band-limited Fourier-transform pairs from noisy data , 1985 .

[26]  D. Titterington.  Common structure of smoothing techniques in statistics , 1985 .

[27]  B. Frieden.  Dice, entropy, and likelihood , 1985, Proceedings of the IEEE.

[28]  Mario Bertero,et al.  Linear inverse problems with discrete data. I. General formulation and singular system analysis , 1985 .

[29]  R FriedenB,et al.  サイコロ,エントロピー,優度 , 1985 .

[30]  J. Sanz.  Mathematical Considerations for the Problem of Fourier Transform Phase Retrieval from Magnitude , 1985 .

[31]  A. R. Davies,et al.  Optimisation in the regularisation ill-posed problems , 1986, The Journal of the Australian Mathematical Society. Series B. Applied Mathematics.

[32]  R. Narayan,et al.  Maximum Entropy Image Restoration in Astronomy , 1986 .

[33]  F. O’Sullivan.  A Statistical Perspective on Ill-posed Inverse Problems , 1986 .

[34]  R. Freeman.  A handbook of nuclear magnetic resonance , 1987 .

[35]  I. Koch,et al.  A direct surface smoothing procedure for Fourier image reconstruction in radiophysics , 1987 .

[36]  E. T. Jaynes,et al.  Bayesian Spectrum and Chirp Analysis , 1987 .

[37]  D. Donoho.  One-sided inference about functionals of a density , 1988 .

[38]  I. Daubechies.  Orthonormal bases of compactly supported wavelets , 1988 .

[39]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[40]  J. Skilling.  The Axioms of Maximum Entropy , 1988 .

[41]  R. Newman,et al.  Maximization of entropy and minimization of area as criteria for NMR signal processing , 1988 .

[42]  J. Skilling.  Classic Maximum Entropy , 1989 .

[43]  D. Donoho,et al.  Uncertainty principles and signal recovery , 1989 .

[44]  Stephen F. Gull,et al.  Developments in Maximum Entropy Data Analysis , 1989 .

[45]  B. Ripley,et al.  Using spatial models as priors in astronomical image analysis , 1989 .

[46]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  I. Good,et al.  C328. Diagnosis of heart attack from two enzyme measurements by means of bivariate probability density estimation: statistical details , 1989 .

[48]  S. P. Luttrell,et al.  A Bayesian derivation of an iterative autofocus/super-resolution algorithm , 1990 .

[49]  Didier Dacunha-Castelle,et al.  Maximum d'entropie et problème des moments , 1990 .

[50]  R. T. Cox.  Probability, frequency and reasonable expectation , 1990 .

[51]  E. Gassiat.  Problèmes des moments et concentration de mesure , 1990 .

[52]  A S Stern,et al.  Does the maximum entropy method improve sensitivity? , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[53]  P. Green.  On Use of the EM Algorithm for Penalized Likelihood Estimation , 1990 .

[54]  G. Klir,et al.  Generalized information theory , 1991 .

[55]  B. D. Ripley,et al.  The use of spatial models as image priors , 1991 .

[56]  Xinhua Zhuang,et al.  Maximum entropy image reconstruction , 1991, IEEE Trans. Signal Process..

[57]  M. Charter.  Quantifying Drug Absorption , 1991 .

[58]  B. Logan,et al.  Signal recovery and the large sieve , 1992 .

[59]  Heinz W. Engl,et al.  Convergence rates for maximum entropy regularization , 1993 .