Convergent incremental optimization transfer algorithms: application to tomography

No convergent ordered subsets (OS) type image reconstruction algorithms for transmission tomography have been proposed to date. In contrast, in emission tomography, there are two known families of convergent OS algorithms: methods that use relaxation parameters , and methods based on the incremental expectation-maximization (EM) approach . This paper generalizes the incremental EM approach by introducing a general framework, "incremental optimization transfer". The proposed algorithms accelerate convergence speeds and ensure global convergence without requiring relaxation parameters. The general optimization transfer framework allows the use of a very broad family of surrogate functions, enabling the development of new algorithms . This paper provides the first convergent OS-type algorithm for (nonconcave) penalized-likelihood (PL) transmission image reconstruction by using separable paraboloidal surrogates (SPS) which yield closed-form maximization steps. We found it is very effective to achieve fast convergence rates by starting with an OS algorithm with a large number of subsets and switching to the new "transmission incremental optimization transfer (TRIOT)" algorithm. Results show that TRIOT is faster in increasing the PL objective than nonincremental ordinary SPS and even OS-SPS yet is convergent.

[1]  Hakan Erdogan,et al.  Ordered subsets algorithms for transmission tomography. , 1999, Physics in medicine and biology.

[2]  Alfred O. Hero,et al.  Ieee Transactions on Image Processing: to Appear Penalized Maximum-likelihood Image Reconstruction Using Space-alternating Generalized Em Algorithms , 2022 .

[3]  Alfred O. Hero,et al.  A Convergent Incremental Gradient Method with a Constant Step Size , 2007, SIAM J. Optim..

[4]  Alvaro R. De Pierro,et al.  Fast EM-like methods for maximum "a posteriori" estimates in emission tomography , 2001, IEEE Transactions on Medical Imaging.

[5]  Quanzheng Li,et al.  A fast fully 4D incremental gradient reconstruction algorithm for list mode PET data , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[6]  R. Leahy,et al.  High-resolution 3D Bayesian image reconstruction using the microPET small-animal scanner. , 1998, Physics in medicine and biology.

[7]  G. Herman,et al.  Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography. , 1970, Journal of theoretical biology.

[8]  P. Khurd,et al.  A globally convergent regularized ordered-subset EM algorithm for list-mode reconstruction , 2003, IEEE Transactions on Nuclear Science.

[9]  J. Fessler,et al.  Convergent algorithms for statistical image reconstruction in emission tomography , 2004 .

[10]  Frank Natterer,et al.  Mathematical methods in image reconstruction , 2001, SIAM monographs on mathematical modeling and computation.

[11]  Jeffrey A Fessler,et al.  Penalized-likelihood image reconstruction for digital holography. , 2004, Journal of the Optical Society of America. A, Optics, image science, and vision.

[12]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[13]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[14]  K. Lange.  Convergence of EM image reconstruction algorithms with Gibbs smoothing. , 1990, IEEE transactions on medical imaging.

[15]  Jeffrey A. Fessler,et al.  Emission image reconstruction for randoms-precorrected PET allowing negative sinogram values , 2004, IEEE Transactions on Medical Imaging.

[16]  Anand Rangarajan,et al.  Convergence Proofs for the COSEM-ML and COSEM-MAP Algorithms , 2003 .

[17]  Alvaro R. De Pierro,et al.  A row-action alternative to the EM algorithm for maximizing likelihood in emission tomography , 1996, IEEE Trans. Medical Imaging.

[18]  Katta G. Murty,et al.  Nonlinear Programming Theory and Algorithms , 2007, Technometrics.

[19]  Freek J. Beekman,et al.  Accelerated iterative transmission CT reconstruction using an ordered subsets convex algorithm , 1998, IEEE Transactions on Medical Imaging.

[20]  J. Fessler.  Statistical Image Reconstruction Methods for Transmission Tomography , 2000 .

[21]  Jeffrey A. Fessler,et al.  A paraboloidal surrogates algorithm for convergent penalized-likelihood emission image reconstruction , 1998, 1998 IEEE Nuclear Science Symposium Conference Record. 1998 IEEE Nuclear Science Symposium and Medical Imaging Conference (Cat. No.98CH36255).

[22]  Y. Censor,et al.  Strong underrelaxation in Kaczmarz's method for inconsistent systems , 1983 .

[23]  Krzysztof C. Kiwiel,et al.  Convergence of Approximate and Incremental Subgradient Methods for Convex Optimization , 2003, SIAM J. Optim..

[24]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[25]  William Byrne,et al.  Convergence of EM variants , 1999 .

[26]  Richard M. Leahy,et al.  Resolution and noise properties of MAP reconstruction for fully 3-D PET , 2000, IEEE Transactions on Medical Imaging.

[27]  Jeffrey A Fessler,et al.  Relaxed ordered-subset algorithm for penalized-likelihood image restoration. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[28]  A. Gunawardana,et al.  The information geometry of em variants for speech and image processing , 2001 .

[29]  H. Malcolm Hudson,et al.  Accelerated image reconstruction using ordered subsets of projection data , 1994, IEEE Trans. Medical Imaging.

[30]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[31]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[32]  E. M. L. Beale,et al.  Nonlinear Programming: A Unified Approach. , 1970 .

[33]  Anand Rangarajan,et al.  An accelerated convergent ordered subsets algorithm for emission tomography , 2004, Physics in medicine and biology.

[34]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[35]  Jeffrey A. Fessler,et al.  Spatial resolution properties of penalized-likelihood image reconstruction: space-invariant tomographs , 1996, IEEE Trans. Image Process..

[36]  Jorge Nocedal,et al.  Large Scale Unconstrained Optimization , 1997 .

[37]  Kenneth Lange,et al.  [Optimization Transfer Using Surrogate Objective Functions]: Rejoinder , 2000 .

[38]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[39]  L. Shepp,et al.  Maximum Likelihood Reconstruction for Emission Tomography , 1983, IEEE Transactions on Medical Imaging.

[40]  F. Beekma,et al.  Ordered subset reconstruction for x-ray CT. , 2001, Physics in medicine and biology.

[41]  J.A. Fessler.  Analytical approach to regularization design for isotropic spatial resolution , 2003, 2003 IEEE Nuclear Science Symposium. Conference Record (IEEE Cat. No.03CH37515).

[42]  Alfred O. Hero,et al.  Space-alternating generalized expectation-maximization algorithm , 1994, IEEE Trans. Signal Process..

[43]  Jeffrey A. Fessler,et al.  Globally convergent image reconstruction for emission tomography using relaxed ordered subsets algorithms , 2003, IEEE Transactions on Medical Imaging.

[44]  Anand Rangarajan,et al.  Provably convergent OSEM-like reconstruction algorithm for emission tomography , 2002, SPIE Medical Imaging.

[45]  Richard M. Leahy,et al.  A theoretical study of the contrast recovery and variance of MAP reconstructions with applications to the selection of smoothing parameters , 1998, 1998 IEEE Nuclear Science Symposium Conference Record. 1998 IEEE Nuclear Science Symposium and Medical Imaging Conference (Cat. No.98CH36255).

[46]  Á. R. De Pierro,et al.  Fast EM-like methods for maximum "a posteriori" estimates in emission tomography. , 2001, IEEE transactions on medical imaging.

[47]  Dimitri P. Bertsekas,et al.  A New Class of Incremental Gradient Methods for Least Squares Problems , 1997, SIAM J. Optim..

[48]  I. Duff,et al.  The state of the art in numerical analysis , 1997 .

[49]  Charles L. Byrne,et al.  Accelerating the EMML algorithm and related iterative algorithms by rescaled block-iterative methods , 1998, IEEE Trans. Image Process..

[50]  Richard M. Leahy,et al.  A theoretical study of the contrast recovery and variance of MAP reconstructions from PET data , 1999, IEEE Transactions on Medical Imaging.

[51]  Anand Rangarajan,et al.  A new convergent MAP reconstruction algorithm for emission tomography using ordered subsets and separable surrogates , 2002, Proceedings IEEE International Symposium on Biomedical Imaging.

[52]  A J Reader,et al.  Statistical list-mode image reconstruction for the high resolution research tomograph. , 2004, Physics in medicine and biology.

[53]  Quanzheng Li,et al.  Fast hybrid algorithms for PET image reconstruction , 2005, IEEE Nuclear Science Symposium Conference Record, 2005.

[54]  S Grootoonk,et al.  Performance Evaluation of the Positron Scanner ECAT EXACT , 1992, Journal of computer assisted tomography.

[55]  Robert R. Meyer,et al.  Sufficient Conditions for the Convergence of Monotonic Mathematical Programming Algorithms , 1976, J. Comput. Syst. Sci..

[56]  D. Hunter,et al.  Optimization Transfer Using Surrogate Objective Functions , 2000 .

[57]  William J. Byrne,et al.  Comments on "Efficient training algorithms for HMMs using incremental estimation" , 2000, IEEE Trans. Speech Audio Process..

[58]  Jeffrey A. Fessler,et al.  Fast Monotonic Algorithms for Transmission Tomography , 1999, IEEE Trans. Medical Imaging.

[59]  Richard M. Leahy,et al.  Statistical approaches in quantitative positron emission tomography , 2000, Stat. Comput..

[60]  A. Ostrowski.  Solution of equations in Euclidean and Banach spaces , 1973 .

[61]  Jeffrey A. Fessler,et al.  Statistical image reconstruction for polyenergetic X-ray computed tomography , 2002, IEEE Transactions on Medical Imaging.

[62]  Y. Censor.  Row-Action Methods for Huge and Sparse Systems and Their Applications , 1981 .

[63]  K. Lange,et al.  EM reconstruction algorithms for emission and transmission tomography. , 1984, Journal of computer assisted tomography.

[64]  Jeffrey A. Fessler,et al.  Compensation for nonuniform resolution using penalized-likelihood reconstruction in space-variant imaging systems , 2004, IEEE Transactions on Medical Imaging.

[65]  Yair Censor,et al.  Component averaging: An efficient iterative parallel algorithm for large and sparse unstructured problems , 2001, Parallel Comput..

[66]  Jeffrey A. Fessler,et al.  Image recovery using partitioned-separable paraboloidal surrogate coordinate ascent algorithms , 2002, IEEE Trans. Image Process..

[67]  Jeffrey A. Fessler,et al.  Ieee Transactions on Image Processing: to Appear Globally Convergent Algorithms for Maximum a Posteriori Transmission Tomography , 2022 .

[68]  Gabor T. Herman,et al.  Algebraic reconstruction techniques can be made computationally efficient [positron emission tomography application] , 1993, IEEE Trans. Medical Imaging.

[69]  Jeffrey A. Fessler,et al.  Regularization for uniform spatial resolution properties in penalized-likelihood image reconstruction , 2000, IEEE Transactions on Medical Imaging.

[70]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[71]  F J Beekman,et al.  Evaluation of the ordered subset convex algorithm for cone-beam CT. , 2005, Physics in medicine and biology.

[72]  Hakan Erdogan,et al.  Monotonic algorithms for transmission tomography , 1999, IEEE Transactions on Medical Imaging.

[73]  Ken D. Sauer,et al.  A local update strategy for iterative reconstruction from projections , 1993, IEEE Trans. Signal Process..

[74]  D. Bertsekas,et al.  Convergen e Rate of In remental Subgradient Algorithms , 2000 .