An ADMM-Newton-CNN numerical approach to a TV model for identifying discontinuous diffusion coefficients in elliptic equations: convex case with gradient observations

Identifying the discontinuous diffusion coefficient in an elliptic equation with observation data of the gradient of the solution is an important nonlinear and ill-posed inverse problem. Models with total variational (TV) regularization have been widely studied for this problem, while the theoretically required nonsmoothness property of the TV regularization and the hidden convexity of the models are usually sacrificed when numerical schemes are considered in the literature. In this paper, we show that the favorable nonsmoothness and convexity properties can be entirely kept if the well-known alternating direction method of multipliers (ADMM) is applied to the TV-regularized models, hence it is meaningful to consider designing numerical schemes based on the ADMM. Moreover, we show that one of the ADMM subproblems can be well solved by the active-set Newton method along with the Schur complement reduction method, and the other one can be efficiently solved by the deep convolutional neural network (CNN). The resulting ADMM-Newton-CNN approach is demonstrated to be easily implementable and very efficient even for higher-dimensional spaces with fine mesh discretization.

[1]  M. Hestenes Multiplier and gradient methods , 1969 .

[2]  Stamatios Lefkimmiatis,et al.  Universal Denoising Networks : A Novel CNN Architecture for Image Denoising , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Lea Fleischer,et al.  Regularization of Inverse Problems , 1996 .

[4]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  John Loustau,et al.  Finite Element Method, the Theory , 2016 .

[6]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[7]  Tran Nhan Tam Quyen,et al.  Finite element methods for coefficient identification in an elliptic equation , 2014 .

[8]  Zhiming Chen,et al.  An Augmented Lagrangian Method for Identifying Discontinuous Parameters in Elliptic Systems , 1999 .

[9]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[10]  K. Kunisch,et al.  The augmented lagrangian method for parameter estimation in elliptic systems , 1990 .

[11]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[12]  Jun Zou,et al.  Numerical methods for elliptic inverse problems , 1998, Int. J. Comput. Math..

[13]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[14]  I. Knowles Parameter identification for elliptic problems , 2001 .

[15]  Jun Zou,et al.  An Efficient Linear Solver for Nonlinear Parameter Identification Problems , 2000, SIAM J. Sci. Comput..

[16]  Dinh Nho Hào,et al.  Convergence rates for Tikhonov regularization of coefficient identification problems in Laplace-type equations , 2010 .

[17]  Kazufumi Ito,et al.  Augmented Lagrangian-SQP-Methods in Hilbert Spaces and Application to Control in the Coefficients Problems , 1996, SIAM J. Optim..

[18]  W. Ziemer Weakly Differentiable Functions: Sobolev Spaces and Functions of Bounded Variation , 1989 .

[19]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[20]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[21]  T. Chan,et al.  Overlapping domain decomposition and multigrid methods for inverse problems , 1998 .

[22]  Giorgio C. Buttazzo,et al.  Variational Analysis in Sobolev and BV Spaces - Applications to PDEs and Optimization, Second Edition , 2014, MPS-SIAM series on optimization.

[23]  R. Guenther,et al.  Remarks on parameter identification. I , 1985 .

[24]  C. Vogel,et al.  Analysis of bounded variation penalty methods for ill-posed problems , 1994 .

[25]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[26]  Sören Bartels,et al.  Stability and Experimental Comparison of Prototypical Iterative Schemes for Total Variation Regularized Problems , 2016, Comput. Methods Appl. Math..

[27]  NUMERIGAL ANAIYS,et al.  A variational method for parameter identification , 2009 .

[28]  R. Kohn,et al.  Relaxation of a variational method for impedance computed tomography , 1987 .

[29]  Sören Bartels,et al.  Total Variation Minimization with Finite Elements: Convergence and Iterative Solution , 2012, SIAM J. Numer. Anal..

[30]  Xue-Cheng Tai,et al.  Identification of Discontinuous Coefficients in Elliptic Problems Using Total Variation Regularization , 2003, SIAM J. Sci. Comput..

[31]  Gene H. Golub,et al.  Numerical solution of saddle point problems , 2005, Acta Numerica.

[32]  Guodong Wang,et al.  Multiplicative noise removal using deep CNN denoiser prior , 2017, 2017 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS).

[33]  Kazufumi Ito,et al.  A Numerical Study of an Augmented Lagrangian Method for the Estimation of Parameters in Elliptic Systems , 1991, SIAM J. Sci. Comput..

[34]  Tran Nhan Tam Quyen Variational method for multiple parameter identification in elliptic PDEs , 2017, 1704.00525.

[35]  G. M.,et al.  Partial Differential Equations I , 2023, Applied Mathematical Sciences.

[36]  Jun Zou,et al.  Numerical identifications of parameters in parabolic systems , 1998 .

[37]  Ivan P. Gavrilyuk,et al.  Variational analysis in Sobolev and BV spaces , 2007, Math. Comput..

[38]  Robert V. Kohn,et al.  A variational method for parameter identification , 1988 .

[39]  Robert V. Kohn,et al.  Determining conductivity by boundary measurements , 1984 .

[40]  Arnd Rösch,et al.  Primal-Dual Active Set Strategy for a General Class of Constrained Optimal Control Problems , 2002, SIAM J. Optim..

[41]  T. Chan,et al.  Augmented Lagrangian and total variation methods for recovering discontinuous coefficients from elliptic equations , 1997 .

[42]  P. Clément Approximation by finite element functions using local regularization , 1975 .

[43]  Michael Hinze,et al.  Matrix coefficient identification in an elliptic equation with the convex energy functional method , 2015, 1510.05489.

[44]  Barbara Kaltenbacher,et al.  A saddle point variational formulation for projection–regularized parameter identification , 2002, Numerische Mathematik.

[45]  M. Powell A method for nonlinear constraints in minimization problems , 1969 .

[46]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[47]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[48]  Ronald B. Guenther,et al.  Remarks on Parameter Identification , 1979 .

[49]  Tran Nhan Tam Quyen,et al.  Convergence rates for total variation regularization of coefficient identification problems in elliptic equations I , 2011 .

[50]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[51]  Sabine Fenstermacher,et al.  Estimation Techniques For Distributed Parameter Systems , 2016 .

[52]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[53]  Stefan Ulbrich,et al.  Optimization with PDE Constraints , 2008, Mathematical modelling.