A Majorize-Minimize Memory Gradient method for complex-valued inverse problems

Complex-valued data are encountered in many application areas of signal and image processing. In the context of the optimization of functions of real variables, subspace algorithms have recently attracted much interest, owing to their efficiency for solving large-size problems while simultaneously offering theoretical convergence guarantees. The goal of this paper is to show how some of these methods can be successfully extended to the complex case. More precisely, we investigate the properties of the proposed complex-valued Majorize-Minimize Memory Gradient (3MG) algorithm. Important practical applications of these results arise in inverse problems. Here, we focus on image reconstruction in Parallel Magnetic Resonance Imaging (PMRI). The linear operator involved in the observation model then includes a subsampling operator over the k-space (2D Fourier domain) the choice of which is analyzed through our numerical results. In addition, sensitivity matrices associated with the multiple channel coils come into play. Comparisons with existing optimization methods confirm the better performance of the proposed algorithm. HighlightsAn extension of the Majorize-Minimize Memory Gradient algorithm for minimizing functions of complex variables is proposed.The convergence of the algorithm is proved under weak assumptions.A novel application of the algorithm to parallel Magnetic Resonance Imaging reconstruction is proposed.Through simulations on real data, the algorithm is shown to outperform recent optimization strategies in terms of convergence speed.The algorithm can handle various subsampling schemes, both convex and nonconvex penalization functions and different possibly redundant frame representations.

[1]  Hugues Talbot,et al.  A Majorize-Minimize Subspace Approach for ℓ2-ℓ0 Image Regularization , 2011, SIAM J. Imaging Sci..

[2]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[3]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[4]  J. CandesE.,et al.  Robust uncertainty principles , 2006 .

[5]  Philippe Ciuciu,et al.  3D wavelet-based regularization for parallel MRI reconstruction: Impact on subject and group-level statistical sensitivity in fMRI , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[6]  Nelly Pustelnik,et al.  Relaxing Tight Frame Condition in Parallel Proximal Methods for Signal Restoration , 2011, IEEE Transactions on Signal Processing.

[7]  Pierre Weiss,et al.  Travelling salesman-based variable density sampling , 2013 .

[8]  L. Shao,et al.  From Heuristic Optimization to Dictionary Learning: A Review and Comprehensive Comparison of Image Denoising Algorithms , 2014, IEEE Transactions on Cybernetics.

[9]  Ling Shao,et al.  Nonlocal Hierarchical Dictionary Learning Using Wavelets for Image Denoising , 2013, IEEE Transactions on Image Processing.

[10]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[11]  Yves Goussard,et al.  Extended forms of Geman & Yang algorithm: application to MRI reconstruction , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[12]  Jérôme Idier,et al.  Regularized Doppler radar imaging for target identification in atmospheric clutter , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[13]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[14]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[15]  Émilie Chouzenoux,et al.  A Majorize–Minimize Strategy for Subspace Optimization Applied to Image Restoration , 2011, IEEE Transactions on Image Processing.

[16]  Pierre Weiss,et al.  Variable density compressed sensing in MRI. Theoretical vs heuristic sampling strategies , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[17]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[18]  Jérôme Idier,et al.  A half-quadratic block-coordinate descent method for spectral estimation , 2002, Signal Process..

[19]  Laurent Jacques,et al.  A panorama on multiscale geometric representations, intertwining spatial, directional and frequency selectivity , 2011, Signal Process..

[20]  Lieven De Lathauwer,et al.  Unconstrained Optimization of Real Functions in Complex Variables , 2012, SIAM J. Optim..

[21]  Hiêp Quang Luong,et al.  Augmented Lagrangian based reconstruction of non-uniformly sub-Nyquist sampled MRI data , 2011, Signal Process..

[22]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[23]  Jeffrey A. Fessler,et al.  Parallel MR Image Reconstruction Using Augmented Lagrangian Methods , 2011, IEEE Transactions on Medical Imaging.

[24]  Wei Lin,et al.  Fast MR Image Reconstruction for Partially Parallel Imaging With Arbitrary $k$ -Space Trajectories , 2011, IEEE Transactions on Medical Imaging.

[25]  Tülay Adali,et al.  Complex-Valued Signal Processing: The Proper Way to Deal With Impropriety , 2011, IEEE Transactions on Signal Processing.

[26]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[27]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[28]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[29]  Jean-Philippe Thiran,et al.  Spread Spectrum Magnetic Resonance Imaging , 2012, IEEE Transactions on Medical Imaging.

[30]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[31]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[32]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[33]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[34]  Robert F. H. Fischer,et al.  Precoding and Signal Shaping for Digital Transmission , 2002 .

[35]  Pierre Weiss,et al.  HYR2PICS: Hybrid regularized reconstruction for combined parallel imaging and compressive sensing in MRI , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[36]  Radu Ioan Bot,et al.  Convergence Analysis for a Primal-Dual Monotone + Skew Splitting Algorithm with Applications to Total Variation Minimization , 2012, Journal of Mathematical Imaging and Vision.

[37]  Pierre Weiss,et al.  From variable density sampling to continuous sampling using Markov chains , 2013, 1307.6960.

[38]  Yao Wang,et al.  High-Speed Compressed Sensing Reconstruction in Dynamic Parallel MRI Using Augmented Lagrangian and Parallel Processing , 2012, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[39]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[40]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[41]  BolteJérôme,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems , 2010 .

[42]  Yinyu Ye,et al.  On stress matrices of (d + 1)-lateration frameworks in general position , 2013, Math. Program..

[43]  Seung-Jean Kim,et al.  A fast method for designing time-optimal gradient waveforms for arbitrary k-space trajectories , 2008, IEEE Transactions on Medical Imaging.

[44]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[45]  Klaas Paul Pruessmann,et al.  A Fast Wavelet-Based Reconstruction Method for Magnetic Resonance Imaging , 2011, IEEE Transactions on Medical Imaging.

[46]  Jean-François Giovannelli,et al.  Regularized estimation of mixed spectra using a circular Gibbs-Markov model , 2001, IEEE Trans. Signal Process..

[47]  Amel Benazza-Benyahia,et al.  A wavelet-based regularized reconstruction algorithm for SENSE parallel MRI with applications to neuroimaging , 2011, Medical Image Anal..