Iterative image reconstruction algorithms based on cross-entropy minimization

The related problems of minimizing the functionals F(x)=alphaKL(y,Px)+(1-alpha)KL(p,x) and G(x)=alphaKL(Px,y)+(1-alpha)KL(x,p), respectively, over the set of vectors x=/>0 are considered. KL(a, b) is the cross-entropy (or Kullback-Leibler) distance between two nonnegative vectors a and b. Iterative algorithms for minimizing both functionals using the method of alternating projections are derived. A simultaneous version of the multiplicative algebraic reconstruction technique (MART) algorithm, called SMART, is introduced, and its convergence is proved.

[1]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[2]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[3]  L. Bregman.  Proof of the convergence of Sheleikhovskii's method for a problem with transportation constraints , 1967 .

[4]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[5]  G. Herman,et al.  Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography. , 1970, Journal of theoretical biology.

[6]  R. Lacoss.  DATA ADAPTIVE SPECTRAL ANALYSIS METHODS , 1971 .

[7]  B. Frieden.  Restoring with maximum likelihood and maximum entropy. , 1972, Journal of the Optical Society of America.

[8]  J. Darroch,et al.  Generalized Iterative Scaling for Log-Linear Models , 1972 .

[9]  J. Burg.  THE RELATIONSHIP BETWEEN MAXIMUM ENTROPY SPECTRA AND MAXIMUM LIKELIHOOD SPECTRA , 1972 .

[10]  P Schmidlin,et al.  Iterative Separation of Sections in Tomographic Scintigrams , 1972, Nuklearmedizin.

[11]  A. Lent,et al.  Iterative reconstruction algorithms. , 1976, Computers in biology and medicine.

[12]  Albert Macovski,et al.  A Maximum Likelihood Approach to Emission Image Reconstruction from Projections , 1976, IEEE Transactions on Nuclear Science.

[13]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[14]  L. Norton-Wayne,et al.  Image Analysis and Evaluation , 1978 .

[15]  S. Gull,et al.  Image reconstruction from incomplete and noisy data , 1978, Nature.

[16]  R. Krupp.  Properties of krulthof's Projection method , 1979, The Bell System Technical Journal.

[17]  Rodney W. Johnson,et al.  Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy , 1980, IEEE Trans. Inf. Theory.

[18]  J. Shore.  Minimum cross-entropy spectral analysis , 1981 .

[19]  E. Jaynes.  On the rationale of maximum-entropy methods , 1982, Proceedings of the IEEE.

[20]  L. Shepp,et al.  Maximum Likelihood Reconstruction for Emission Tomography , 1983, IEEE Transactions on Medical Imaging.

[21]  Y. Censor.  Finite series-expansion reconstruction methods , 1983, Proceedings of the IEEE.

[22]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  K. Lange,et al.  EM reconstruction algorithms for emission and transmission tomography. , 1984, Journal of computer assisted tomography.

[24]  Thomas M. Cover,et al.  An algorithm for maximizing expected log investment return , 1984, IEEE Trans. Inf. Theory.

[25]  Michael I. Miller,et al.  The Use of Sieves to Stabilize Images Produced with the EM Algorithm for Emission Tomography , 1985, IEEE Transactions on Nuclear Science.

[26]  L. Shepp,et al.  A Statistical Model for Positron Emission Tomography , 1985 .

[27]  Gabor T. Herman.  Application of Maximum Entropy and Bayesian Optimization Methods to Image Reconstruction from Projections , 1985 .

[28]  D. Titterington.  On the Iterative Image Space Reconstruction Algorthm for ECT , 1987, IEEE Transactions on Medical Imaging.

[29]  E. Levitan,et al.  A Maximum a Posteriori Probability Expectation Maximization Algorithm for Image Reconstruction in Emission Tomography , 1987, IEEE Transactions on Medical Imaging.

[30]  K. Lange,et al.  A Theoretical Study of Some Maximum Likelihood Algorithms for Emission and Transmission Tomography , 1987, IEEE Transactions on Medical Imaging.

[31]  Y. Censor,et al.  On Block-Iterative Entropy Maximization , 1987 .

[32]  Z. Liang,et al.  Bayesian image processing of data from constrained source distributions—II. valued, uncorrelated and correlated constraints , 1987 .

[33]  Gabor T. Herman,et al.  Special-Purpose Algorithms for Linearly Constrained Entropy Maximization , 1987 .

[34]  Yair Censor,et al.  Parallel application of block-iterative methods in medical imaging and radiation therapy , 1988, Math. Program..

[35]  I. Csiszár.  A geometric interpretation of Darroch and Ratcliff's generalized iterative scaling , 1989 .

[36]  L. Jones.  Approximation-theoretic derivation of logarithmic entropy principles for inverse problems and unique extension of the maximum entropy method to incorporate prior knowledge , 1989 .

[37]  R. MacKinnon.  Minimum cross-entropy noise reduction in images , 1989 .

[38]  T. Hebert,et al.  A generalized EM algorithm for 3-D Bayesian reconstruction from Poisson data using Gibbs priors. , 1989, IEEE transactions on medical imaging.

[39]  Charles L. Byrne,et al.  Axiomatic approach to certain inverse problems , 1990, Optics & Photonics.

[40]  J. D. Wilson,et al.  A smoothed EM approach to indirect estimation problems, with particular reference to stereology and emission tomography , 1990 .

[41]  Charles L. Byrne,et al.  General entropy criteria for inverse problems, with applications to data compression, pattern classification, and cluster analysis , 1990, IEEE Trans. Inf. Theory.

[42]  P. Green.  Bayesian reconstructions from emission tomography data using a modified EM algorithm. , 1990, IEEE transactions on medical imaging.

[43]  S. Holte,et al.  Iterative image reconstruction for positron emission tomography: a study of convergence and quantitation problems , 1990 .

[44]  Yair Censor,et al.  The primal-dual algorithm as a constraint-set-manipulation device , 1991, Math. Program..

[45]  R Leahy,et al.  Applications of Markov random fields in medical imaging. , 1991, Progress in clinical and biological research.

[46]  Y. Censor,et al.  Parallel computing with block-iterative image reconstruction algorithms , 1991 .

[47]  Z Liang,et al.  A spatial interaction model for statistical image processing. , 1991, Progress in clinical and biological research.

[48]  L. Shepp,et al.  New Insights into Emission Tomography Via Linear Programming , 1992 .