Numerical solution of inverse problems by weak adversarial networks

We consider a weak adversarial network approach to numerically solve a class of inverse problems, including electrical impedance tomography and dynamic electrical impedance tomography problems. We leverage the weak formulation of PDE in the given inverse problem, and parameterize the solution and the test function as deep neural networks. The weak formulation and the boundary conditions induce a minimax problem of a saddle function of the network parameters. As the parameters are alternatively updated, the network gradually approximates the solution of the inverse problem. We provide theoretical justifications on the convergence of the proposed algorithm. Our method is completely mesh-free without any spatial discretization, and is particularly suitable for problems with high dimensionality and low regularity on solutions. Numerical experiments on a variety of test inverse problems demonstrate the promising accuracy and efficiency of our approach.

[1]  Faming Liang,et al.  Statistical and Computational Inverse Problems , 2006, Technometrics.

[2]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[3]  Reinhard Heckel,et al.  Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators , 2020, ICLR.

[4]  Sambuddha Ghosal,et al.  Generative Models for Solving Nonlinear Partial Differential Equations , 2019 .

[5]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[6]  Lexing Ying,et al.  Meta-learning Pseudo-differential Operators with Deep Neural Networks , 2019, J. Comput. Phys..

[7]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[8]  Elisa Francini Recovering a complex coefficient in a planar domain from the Dirichlet-to-Neumann map , 2000 .

[9]  Tobias Kluth,et al.  Regularization by Architecture: A Deep Prior Approach for Inverse Problems , 2019, Journal of Mathematical Imaging and Vision.

[10]  Dong Liu,et al.  Dominant-Current Deep Learning Scheme for Electrical Impedance Tomography , 2019, IEEE Transactions on Biomedical Engineering.

[11]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[12]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[13]  Guy Chavent,et al.  Nonlinear Least Squares for Inverse Problems: Theoretical Foundations and Step-by-Step Guide for Applications , 2009 .

[14]  Philippe Labazuy,et al.  Contribution of 3D inversion of Electrical Resistivity Tomography data applied to volcanic structures , 2016 .

[15]  R. Lopez,et al.  Artificial Neural Networks for the Solution of Inverse Problems , 2006 .

[16]  Zhiping Mao,et al.  DeepXDE: A Deep Learning Library for Solving Differential Equations , 2019, AAAI Spring Symposium: MLPS.

[17]  Sébastien Martin,et al.  A Post-Processing Method for Three-Dimensional Electrical Impedance Tomography , 2017, Scientific Reports.

[18]  R. Ottoboni,et al.  Active monitoring apparatus for underground pollutant detection based on electrical impedance tomography , 2002, IMTC/2002. Proceedings of the 19th IEEE Instrumentation and Measurement Technology Conference (IEEE Cat. No.00CH37276).

[19]  Edward B Curtis,et al.  The Dirichlet to Neumann map for a resistor network , 1991 .

[20]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[21]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[22]  George Em Karniadakis,et al.  Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements , 2019, European Journal of Applied Mathematics.

[23]  David Isaacson,et al.  Reconstructions of chest phantoms by the D-bar method for electrical impedance tomography , 2004, IEEE Transactions on Medical Imaging.

[24]  Sebastien Martin,et al.  Nonlinear Electrical Impedance Tomography Reconstruction Using Artificial Neural Networks and Particle Swarm Optimization , 2016, IEEE Transactions on Magnetics.

[25]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[26]  Hichem Sahli,et al.  Thermal infrared identification of buried landmines , 2005, SPIE Defense + Commercial Sensing.

[27]  Jonas Adler,et al.  Solving ill-posed inverse problems using iterative deep neural networks , 2017, ArXiv.

[28]  Hadi Meidani,et al.  A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations , 2018, Probabilistic Engineering Mechanics.

[29]  Alexander J. Smola,et al.  Proximal Stochastic Methods for Nonsmooth Nonconvex Finite-Sum Optimization , 2016, NIPS.

[30]  Junshan Lin,et al.  Inverse scattering problems with multi-frequencies , 2015 .

[31]  Andreas Hauptmann,et al.  Model-Based Learning for Accelerated, Limited-View 3-D Photoacoustic Tomography , 2017, IEEE Transactions on Medical Imaging.

[32]  E Weinan,et al.  Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations , 2017, Communications in Mathematics and Statistics.

[33]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[34]  E Weinan,et al.  The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems , 2017, Communications in Mathematics and Statistics.

[35]  Clifford H. Thurber,et al.  Parameter estimation and inverse problems , 2005 .

[36]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[37]  Li Jun Jiang,et al.  Two-Step Enhanced Deep Learning Approach for Electromagnetic Inverse Scattering Problems , 2019, IEEE Antennas and Wireless Propagation Letters.

[38]  A. Calderón,et al.  On an inverse boundary value problem , 2006 .

[39]  Gang Bao,et al.  Weak Adversarial Networks for High-dimensional Partial Differential Equations , 2019, J. Comput. Phys..

[40]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[41]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[42]  E Weinan,et al.  Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-order Backward Stochastic Differential Equations , 2017, J. Nonlinear Sci..

[43]  Jong Chul Ye,et al.  A deep convolutional neural network using directional wavelets for low‐dose X‐ray CT reconstruction , 2016, Medical physics.

[44]  Andreas Hauptmann,et al.  Beltrami-net: domain-independent deep D-bar learning for absolute imaging with electrical impedance tomography (a-EIT) , 2018, Physiological measurement.

[45]  Demetri Terzopoulos,et al.  Regularization of Inverse Visual Problems Involving Discontinuities , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Lexing Ying,et al.  Solving Electrical Impedance Tomography with Deep Learning , 2019, J. Comput. Phys..

[47]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[48]  George Em Karniadakis,et al.  DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators , 2019, ArXiv.

[49]  Tomasz Rymarczyk,et al.  Comparison of Selected Machine Learning Algorithms for Industrial Electrical Tomography , 2019, Sensors.

[50]  David Colton,et al.  Looking Back on Inverse Scattering Theory , 2018, SIAM Rev..

[51]  Masahiro Takei,et al.  Image Reconstruction Based on Convolutional Neural Network for Electrical Resistance Tomography , 2019, IEEE Sensors Journal.

[52]  A. V. D. Vaart,et al.  BAYESIAN INVERSE PROBLEMS WITH GAUSSIAN PRIORS , 2011, 1103.2692.

[53]  Andy Adler,et al.  A neural network image reconstruction technique for electrical impedance tomography , 1994, IEEE Trans. Medical Imaging.

[54]  Stephan Antholzer,et al.  NETT: solving inverse problems with deep neural networks , 2018, Inverse Problems.

[55]  David Mera,et al.  Towards a Fast and Accurate EIT Inverse Problem Solver: A Machine Learning Approach , 2018, Electronics.

[56]  Lexing Ying,et al.  SwitchNet: a neural network model for forward and inverse scattering problems , 2018, SIAM J. Sci. Comput..

[57]  Lexing Ying,et al.  Solving for high-dimensional committor functions using artificial neural networks , 2018, Research in the Mathematical Sciences.

[58]  Jian Li,et al.  A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimization , 2018, NeurIPS.

[59]  C. Croke,et al.  Geometric Methods in Inverse Problems and PDE Control , 2004 .

[60]  Jun Xiao,et al.  Deep Learning Image Reconstruction Simulation for Electromagnetic Tomography , 2018, IEEE Sensors Journal.

[61]  R. Snieder Inverse Problems in Geophysics , 2001 .

[62]  Sai Ho Ling,et al.  Review on Electrical Impedance Tomography: Artificial Intelligence Methods and its Applications , 2019, Algorithms.

[63]  Michael M. Zavlanos,et al.  VarNet: Variational Neural Networks for the Solution of Partial Differential Equations , 2019, L4DC.

[64]  Haizhao Yang,et al.  Int-Deep: A Deep Learning Initialized Iterative Method for Nonlinear Problems , 2020, J. Comput. Phys..

[65]  Steven L. Brunton,et al.  Data-Driven Identification of Parametric Partial Differential Equations , 2018, SIAM J. Appl. Dyn. Syst..

[66]  M. Kelter,et al.  Advances in Spectral Electrical Impedance Tomography (EIT) for Near-Surface Geophysical Exploration , 2016 .

[67]  Yaroslav Kurylev,et al.  Boundary control, wave field continuation and inverse problems for the wave equation , 1991 .

[68]  P. Hua,et al.  Measuring lung resistivity using electrical impedance tomography , 1992, IEEE Transactions on Biomedical Engineering.

[69]  E Weinan,et al.  Overcoming the curse of dimensionality: Solving high-dimensional partial differential equations using deep learning , 2017, ArXiv.

[70]  George Em Karniadakis,et al.  A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems , 2019, J. Comput. Phys..

[71]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[72]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[73]  Boris Rubinsky,et al.  Electrical impedance tomography for imaging tissue electroporation , 2004, IEEE Transactions on Biomedical Engineering.

[74]  S. J. Hamilton,et al.  Deep D-Bar: Real-Time Electrical Impedance Tomography Imaging With Deep Neural Networks , 2017, IEEE Transactions on Medical Imaging.

[75]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[76]  Leah Bar,et al.  Unsupervised Deep Learning Algorithm for PDE-based Forward and Inverse Problems , 2019, ArXiv.

[77]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[78]  Hyeontae Jo,et al.  Deep Neural Network Approach to Forward-Inverse Problems , 2019, Networks Heterog. Media.

[79]  Tim Dockhorn,et al.  A Discussion on Solving Partial Differential Equations using Neural Networks , 2019, ArXiv.

[80]  Saeed Ghadimi,et al.  Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization , 2013, Mathematical Programming.

[81]  Masahiro Yamamoto,et al.  Stability, reconstruction formula and regularization for an inverse source hyperbolic problem by a control method , 1995 .

[82]  Luis Tenorio,et al.  Prior information and uncertainty in inverse problems , 2001 .

[83]  G. Backus,et al.  Numerical Applications of a Formalism for Geophysical Inverse Problems , 1967 .

[84]  Naif Alajlan,et al.  Artificial Neural Network Methods for the Solution of Second Order Boundary Value Problems , 2019, Computers, Materials & Continua.

[85]  Michal Prauzek,et al.  Image reconstruction in electrical impedance tomography using neural network , 2014, 2014 Cairo International Biomedical Engineering Conference (CIBEC).

[86]  David Isaacson,et al.  Electrical Impedance Tomography , 1999, SIAM Rev..

[87]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[88]  Jingzhi Li,et al.  Electrical Impedance Tomography with Restricted Dirichlet-to-Neumann Map Data , 2018, 1803.11193.

[89]  Stability , 1973 .

[90]  Stephan Antholzer,et al.  Deep learning for photoacoustic tomography from sparse data , 2017, Inverse problems in science and engineering.

[91]  Lorenzo Bruzzone,et al.  Kernel methods for remote sensing data analysis , 2009 .