Efficient generalized Golub–Kahan based methods for dynamic inverse problems

We consider efficient methods for computing solutions to and estimating uncertainties in dynamic inverse problems, where the parameters of interest may change during the measurement procedure. Compared to static inverse problems, incorporating prior information in both space and time in a Bayesian framework can become computationally intensive, in part, due to the large number of unknown parameters. In these problems, explicit computation of the square root and/or inverse of the prior covariance matrix is not possible. In this work, we develop efficient, iterative, matrix-free methods based on the generalized Golub-Kahan bidiagonalization that allow automatic regularization parameter and variance estimation. We demonstrate that these methods can be more flexible than standard methods and develop efficient implementations that can exploit structure in the prior, as well as possible structure in the forward model. Numerical examples from photoacoustic tomography, deblurring, and passive seismic tomography demonstrate the range of applicability and effectiveness of the described approaches. Specifically, in passive seismic tomography, we demonstrate our approach on both synthetic and real data. To demonstrate the scalability of our algorithm, we solve a dynamic inverse problem with approximately $43,000$ measurements and $7.8$ million unknowns in under $40$ seconds on a standard desktop.

[1]  A K Louis,et al.  Efficient algorithms for the regularization of dynamic inverse problems: I. Theory , 2002 .

[2]  Barbara Vantaggi,et al.  Bayes meets Krylov: preconditioning CGLS for underdetermined systems , 2015, 1503.06844.

[3]  Marko Vauhkonen,et al.  Image reconstruction in time-varying electrical impedance tomography based on the extended Kalman filter , 2001 .

[4]  Bernadette N. Hahn Efficient algorithms for linear dynamic inverse problems with known motion , 2014 .

[5]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[6]  Misha Elena Kilmer,et al.  A Framework for Regularization via Operator Approximation , 2015, SIAM J. Sci. Comput..

[7]  Per Christian Hansen,et al.  Regularization methods for large-scale problems , 1993 .

[8]  M. Fuentes Testing for separability of spatial–temporal covariance functions , 2006 .

[9]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[10]  C. Vogel Computational Methods for Inverse Problems , 1987 .

[11]  Georg Stadler,et al.  Extreme-scale UQ for Bayesian inverse problems governed by PDEs , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[12]  J. Nagy,et al.  KRONECKER PRODUCT AND SVD APPROXIMATIONS IN IMAGE RESTORATION , 1998 .

[13]  Kun Wang,et al.  Impact of nonstationary optical illumination on image reconstruction in optoacoustic tomography. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[14]  Rosemary A. Renaut,et al.  Hybrid and iteratively reweighted regularization by unbiased predictive risk and weighted GCV for projected systems 5 December 2016 , 2016 .

[15]  K. Luxbacher,et al.  Passive seismic tomography for three-dimensional time-lapse imaging of mining-induced rock mass changes , 2012 .

[16]  Julianne Chung,et al.  Motion Estimation and Correction in Photoacoustic Tomographic Reconstruction , 2016, SIAM J. Imaging Sci..

[17]  D. Calvetti,et al.  Priorconditioners for linear systems , 2005 .

[18]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[19]  Daniela Calvetti,et al.  Preconditioned iterative methods for linear discrete ill-posed problems from a Bayesian inversion perspective , 2007 .

[20]  P. Hansen Discrete Inverse Problems: Insight and Algorithms , 2010 .

[21]  Mark A. Anastasio,et al.  A Constrained Variable Projection Reconstruction Method for Photoacoustic Computed Tomography Without Accurate Knowledge of Transducer Responses , 2015, IEEE Transactions on Medical Imaging.

[22]  O. Scherzer Handbook of mathematical methods in imaging , 2011 .

[23]  Alexander Katsevich,et al.  Local Tomography and the Motion Estimation Problem , 2011, SIAM J. Imaging Sci..

[24]  Kamiar Rahnama Rad,et al.  Fast Kalman Filtering and Forward–Backward Smoothing via a Low-Rank Perturbative Approach , 2014 .

[25]  Dominique Orban,et al.  Iterative Methods for Symmetric Quasi-Definite Linear Systems. Part I: Theory , 2013 .

[26]  Tiangang Cui,et al.  Optimal Low-rank Approximations of Bayesian Linear Inverse Problems , 2014, SIAM J. Sci. Comput..

[27]  Carsten H. Wolters,et al.  Efficient algorithms for the regularization of dynamic inverse problems: II. Applications , 2002 .

[28]  Bart G. van Bloemen Waanders,et al.  Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations , 2011, SIAM J. Sci. Comput..

[29]  Lihong V Wang,et al.  Fast spatiotemporal image reconstruction based on low-rank matrix estimation for dynamic photoacoustic computed tomography , 2014, Journal of biomedical optics.

[30]  Alan J. Laub,et al.  Matrix analysis - for scientists and engineers , 2004 .

[31]  E. Westman,et al.  Imaging of temporal stress redistribution due to triggered seismicity at a deep nickel mine , 2016 .

[32]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[33]  Gene H. Golub,et al.  Calculating the singular values and pseudo-inverse of a matrix , 2007, Milestones in Matrix Computation.

[34]  J. Nagy,et al.  A weighted-GCV method for Lanczos-hybrid regularization. , 2007 .

[35]  A. Saibaba,et al.  Fast computation of uncertainty quantification measures in the geostatistical approach to solve inverse problems , 2014, 1404.1263.

[36]  Rosemary A. Renaut,et al.  Computational Statistics and Data Analysis , 2022 .

[37]  Peter K. Kitanidis,et al.  Efficient methods for large‐scale linear inversion using a geostatistical approach , 2012 .

[38]  Wuqiang Yang,et al.  Dynamic imaging in electrical capacitance tomography and electromagnetic induction tomography using a Kalman filter , 2007 .

[39]  P. Guttorp,et al.  Geostatistical Space-Time Models, Stationarity, Separability, and Full Symmetry , 2007 .

[40]  Mark A. Anastasio,et al.  Simultaneous reconstruction of speed-of-sound and optical absorption properties in photoacoustic tomography via a time-domain iterative algorithm , 2008, SPIE BiOS.

[41]  Eric Darve,et al.  A Kalman filter powered by H2 ‐matrices for quasi‐continuous data assimilation problems , 2014, ArXiv.

[42]  Bernadette N. Hahn,et al.  Dynamic linear inverse problems with moderate movements of the object: Ill-posedness and regularization , 2015 .

[43]  M. Genton Separable approximations of space‐time covariance matrices , 2007 .

[44]  Rosemary Knight,et al.  Application of an extended Kalman filter approach to inversion of time‐lapse electrical resistivity imaging data for monitoring recharge , 2011 .

[45]  Mark A. Anastasio,et al.  Photoacoustic and Thermoacoustic Tomography: Image Formation Principles , 2015, Handbook of Mathematical Methods in Imaging.

[46]  K. Luxbacher,et al.  Three-dimensional time-lapse velocity tomography of an underground longwall panel , 2008 .

[47]  B. AfeArd CALCULATING THE SINGULAR VALUES AND PSEUDOINVERSE OF A MATRIX , 2022 .

[48]  Dominique Orban,et al.  Iterative Solution of Symmetric Quasi-Definite Linear Systems , 2017 .

[49]  Arvind K. Saibaba,et al.  Generalized Hybrid Iterative Methods for Large-Scale Bayesian Inverse Problems , 2016, SIAM J. Sci. Comput..

[50]  Michael K. Ng,et al.  Kronecker Product Approximations forImage Restoration with Reflexive Boundary Conditions , 2003, SIAM J. Matrix Anal. Appl..

[51]  Maria Rosaria Russo,et al.  On Krylov projection methods and Tikhonov regularization , 2015 .

[52]  Peter K. Kitanidis,et al.  Fast Kalman filter using hierarchical matrices and a low-rank perturbative approach , 2014, 1405.2276.

[53]  Liam Paninski,et al.  Fast Kalman filtering on quasilinear dendritic trees , 2010, Journal of Computational Neuroscience.

[54]  Per Christian Hansen,et al.  REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems , 1994, Numerical Algorithms.

[55]  A Rashid,et al.  A dynamic oppositional biogeography-based optimization approach for time-varying electrical impedance tomography , 2016, Physiological measurement.

[56]  Gene H. Golub,et al.  Linear algebra for large scale and real-time applications , 1993 .

[57]  C. Loan,et al.  Approximation with Kronecker Products , 1992 .

[58]  Mario Arioli,et al.  Generalized Golub-Kahan Bidiagonalization and Stopping Criteria , 2013, SIAM J. Matrix Anal. Appl..

[59]  M. Toksöz,et al.  Passive seismic tomography using induced seismicity at a petroleum field in Oman , 2009 .

[60]  Steven J. Benbow,et al.  Solving Generalized Least-Squares Problems with LSQR , 1999, SIAM J. Matrix Anal. Appl..

[61]  Eric de Sturler,et al.  Recycling Krylov Subspaces for Sequences of Linear Systems , 2006, SIAM J. Sci. Comput..

[62]  James Martin,et al.  A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems Part I: The Linearized Case, with Application to Global Seismic Inversion , 2013, SIAM J. Sci. Comput..

[63]  T. Johansen,et al.  State estimation and inverse problems in electrical impedance tomography: observability, convergence and regularization , 2015 .

[64]  P.A. Karjalainen,et al.  A Kalman filter approach to track fast impedance changes in electrical impedance tomography , 1998, IEEE Transactions on Biomedical Engineering.

[65]  Phaedon C. Kyriakidis,et al.  Geostatistical Space–Time Models: A Review , 1999 .

[66]  Julianne Chung,et al.  A Hybrid LSMR Algorithm for Large-Scale Tikhonov Regularization , 2015, SIAM J. Sci. Comput..

[67]  Eric Darve,et al.  Application of Hierarchical Matrices to Linear Inverse Problems in Geostatistics , 2012 .

[68]  Misha Elena Kilmer,et al.  Choosing Regularization Parameters in Iterative Methods for Ill-Posed Problems , 2000, SIAM J. Matrix Anal. Appl..