Bayesian Inference with Error Variable Splitting and Sparsity Enforcing Priors for Linear Inverse Problems

Regularization and Bayesian inference based methods have been successfully applied for linear inverse problems. In these methods, often simple Gaussian or Poisson models for the forward model errors have been considered. In this work, we use variable splitting for the errors to model different sources of errors and their possible non-stationarity or impulsive nature using Student-t or other heavy tailed distributions. Also, as a prior model, a sparsity enforcing hierarchical model of Infinite Gaussian Mixture model is introduced. With these prior models, we obtain a complete Bayesian inference framework which can efficiently be implemented for any linear inverse problem. Interestingly, many recent regularization-based algorithms such as Alternating Direction Method of Multipliers (ADMM) as well as more classical Bayesian based methods such as Sparse Bayesian Learning (SBL) are obtained as particular cases. One advantage of the Bayesian approach is the possibility to estimate, jointly with the reconstruction, the hyper-parameters such as the regularization parameter, thus the capability of proposing unsupervised methods. Examples of implementation of the proposed method in different signal and image processing such as deconvolution in mass spectrometry, estimation of periodic components estimation in biological signals and computed tomography are mentioned and referenced.

[1]  Ali Mohammad-Djafari,et al.  Bayesian approach with prior models which enforce sparsity in signal and image processing , 2012, EURASIP J. Adv. Signal Process..

[2]  Ali Mohammad-Djafari,et al.  Computed tomography reconstruction based on a hierarchical model and variational Bayesian method , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Jiansong Deng,et al.  Variational Mesh Denoising Using Total Variation and Piecewise Constant Function Space , 2015, IEEE Transactions on Visualization and Computer Graphics.

[4]  Ali Mohammad-Djafari,et al.  Super-Resolution: A Short Review, A New Method Based on Hidden Markov Modeling of HR Image and Future Challenges , 2009, Comput. J..

[5]  A. Mohammad-Djafari A full Bayesian approach for inverse problems , 2001, physics/0111123.

[6]  Ali Mohammad-Djafari,et al.  A 3D Bayesian Computed Tomography Reconstruction Algorithm with Gauss-Markov-Potts Prior Model and its Application to Real Data , 2017, Fundam. Informaticae.

[7]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[8]  A. Mohammad-Djafari,et al.  Bayesian 3D X-ray computed tomography image reconstruction with a scaled Gaussian mixture prior model , 2015 .

[9]  Jeffrey A. Fessler,et al.  A Splitting-Based Iterative Algorithm for Accelerated Statistical X-Ray CT Reconstruction , 2012, IEEE Transactions on Medical Imaging.

[10]  Shuhang Chen,et al.  Simultaneous Reconstruction and Segmentation of Dynamic PET via Low-Rank and Sparse Matrix Decomposition , 2015, IEEE Transactions on Biomedical Engineering.

[11]  A. Basarab,et al.  Fast Single Image Super-resolution using a New Analytical Solution for l2-l2 Problems. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[12]  Vaclav Smidl,et al.  The Variational Bayes Method in Signal Processing (Signals and Communication Technology) , 2005 .

[13]  A. Mohammad-Djafari Bayesian inference with hierarchical prior models for inverse problems in imaging systems , 2013, 2013 8th International Workshop on Systems, Signal Processing and their Applications (WoSSPA).

[14]  Ali Mohammad-Djafari,et al.  Direct nonparametric estimation of the period and the shape of a periodic component in short duration signals , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[15]  Susanto Rahardja,et al.  Alternating Direction Method for Balanced Image Restoration , 2012, IEEE Transactions on Image Processing.

[16]  Jeffrey A. Fessler,et al.  Parallel MR Image Reconstruction Using Augmented Lagrangian Methods , 2011, IEEE Transactions on Medical Imaging.

[18]  Peyman Milanfar,et al.  Robust Multichannel Blind Deconvolution via Fast Alternating Minimization , 2012, IEEE Transactions on Image Processing.

[19]  Li Wang,et al.  Unsupervised sparsity enforcing iterative algorithms for 3D image reconstruction in X-ray computed tomography , 2017 .

[20]  Ali Mohammad-Djafari,et al.  On the estimation of hyperparameters in Bayesian approach of solving inverse problems , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[21]  Lizhi Cheng,et al.  Alternative minimisation algorithm for non-local total variational image deblurring , 2010 .

[22]  Matthew J. Beal Variational algorithms for approximate Bayesian inference , 2003 .

[23]  Camille Chapdelaine,et al.  A joint segmentation and reconstruction algorithm for 3D Bayesian Computed Tomography using Gaus-Markov-Potts Prior Model , 2017, ICASSP 2017.

[24]  Ali Mohammad-Djafari,et al.  Precise periodic components estimation for chronobiological signals through Bayesian Inference with sparsity enforcing prior , 2016, EURASIP J. Bioinform. Syst. Biol..

[25]  Wei Zhang,et al.  A Fast Adaptive Parameter Estimation for Total Variation Image Restoration , 2014, IEEE Transactions on Image Processing.

[26]  Shinichi Nakajima,et al.  Bayesian Group-Sparse Modeling and Variational Inference , 2014, IEEE Transactions on Signal Processing.

[27]  Jeffrey A. Fessler,et al.  Augmented Lagrangian with Variable Splitting for Faster Non-Cartesian ${\rm L}_{1}$-SPIRiT MR Image Reconstruction , 2014, IEEE Transactions on Medical Imaging.

[28]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[29]  Guobao Wang,et al.  Statistical Image Reconstruction for Muon Tomography Using a Gaussian Scale Mixture Model , 2009, IEEE Transactions on Nuclear Science.

[30]  M. Nikolova An Algorithm for Total Variation Minimization and Applications , 2004 .

[31]  Jeffrey A. Fessler,et al.  Accelerated Regularized Estimation of MR Coil Sensitivities Using Augmented Lagrangian Methods , 2013, IEEE Transactions on Medical Imaging.

[32]  Yu Liu,et al.  Bregman Iteration Based Efficient Algorithm for MR Image Reconstruction From Undersampled K-Space Data , 2013, IEEE Signal Processing Letters.

[33]  Jeffrey A. Fessler,et al.  Fast Parallel MR Image Reconstruction via B1-Based, Adaptive Restart, Iterative Soft Thresholding Algorithms (BARISTA) , 2015, IEEE Transactions on Medical Imaging.

[34]  Jean-Yves Tourneret,et al.  Fast Single Image Super-Resolution Using a New Analytical Solution for $\ell _{2}$ – $\ell _{2}$ Problems , 2016, IEEE Transactions on Image Processing.

[35]  Gene H. Golub,et al.  A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration , 1999, SIAM J. Sci. Comput..

[36]  Michel Barlaud,et al.  Deterministic edge-preserving regularization in computed imaging , 1997, IEEE Trans. Image Process..

[37]  Jeffrey A. Fessler,et al.  Alternating Direction Method of Multiplier for Tomography With Nonlocal Regularizers , 2014, IEEE Transactions on Medical Imaging.

[38]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[39]  Camille Chapdelaine,et al.  Joint Reconstruction and Segmentation of Real 3D Data in Computed Tomography thanks to a Gauss-Markov-Potts Prior Model, , 2017 .

[40]  Ali Mohammad-Djafari,et al.  X-ray Computed Tomography using a Sparsity Enforcing Prior Model Based on Haar Transformation in a Bayesian Framework , 2017, Fundam. Informaticae.

[41]  Antonio J. Plaza,et al.  Total Variation Spatial Regularization for Sparse Hyperspectral Unmixing , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[42]  A. Mohammad-Djafari,et al.  A variational Bayesian approximation approach via a sparsity enforcing prior in acoustic imaging , 2014, 2014 13th Workshop on Information Optics (WIO).

[43]  Ben Adcock,et al.  Efficient Compressed Sensing SENSE pMRI Reconstruction With Joint Sparsity Promotion , 2016, IEEE Transactions on Medical Imaging.

[44]  A. Mohammad-Djafari,et al.  Optical imaging in a variational Bayesian framework , 2014 .