Deep Learning Solution of the Eigenvalue Problem for Differential Operators

Abstract Solving the eigenvalue problem for differential operators is a common problem in many scientific fields. Classical numerical methods rely on intricate domain discretization and yield nonanalytic or nonsmooth approximations. We introduce a novel neural network–based solver for the eigenvalue problem of differential self-adjoint operators, where the eigenpairs are learned in an unsupervised end-to-end fashion. We propose several training procedures for solving increasingly challenging tasks toward the general eigenvalue problem. The proposed solver is capable of finding the M smallest eigenpairs for a general differential operator. We demonstrate the method on the Laplacian operator, which is of particular interest in image processing, computer vision, and shape analysis among many other applications. In addition, we solve the Legendre differential equation. Our proposed method simultaneously solves several eigenpairs and can be easily used on free-form domains. We exemplify it on L-shape and circular cut domains. A significant contribution of this work is an analysis of the numerical error of this method. In particular an upper bound for the (unknown) solution error is given in terms of the (measured) truncation error of the partial differential equation and the network structure.

[1]  Yang Wang,et al.  Error Analysis of Deep Ritz Methods for Elliptic Equations , 2021, Analysis and Applications.

[2]  Mones Raslan,et al.  Approximation Rates for Neural Networks with Encodable Weights in Smoothness Spaces , 2020, Neural Networks.

[3]  Zuowei Shen,et al.  Deep Network with Approximation Error Being Reciprocal of Width to Power of Square Root of Depth , 2020, Neural Computation.

[4]  Gordon Wetzstein,et al.  Implicit Neural Representations with Periodic Activation Functions , 2020, NeurIPS.

[5]  Jianfeng Lu,et al.  Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion Monte Carlo like approach , 2020, J. Comput. Phys..

[6]  Zuowei Shen,et al.  Deep Network Approximation for Smooth Functions , 2020, SIAM J. Math. Anal..

[7]  Dawei Li,et al.  Sub-Optimal Local Minima Exist for Neural Networks with Almost All Non-Linear Activations , 2019 .

[8]  G. Carleo,et al.  Fermionic neural-network states for ab-initio electronic structure , 2019, Nature Communications.

[9]  F. Noé,et al.  Deep-neural-network solution of the electronic Schrödinger equation , 2019, Nature Chemistry.

[10]  Guy Gilboa,et al.  Numeric Solutions of Eigenvalue Problems for Generic Nonlinear Operators , 2019, ArXiv.

[11]  Yongdai Kim,et al.  Smooth Function Approximation by Deep Neural Networks with General Activation Functions , 2019, Entropy.

[12]  Zuowei Shen,et al.  Deep Network Approximation Characterized by Number of Neurons , 2019, Communications in Computational Physics.

[13]  Nicolas Papadakis,et al.  Rayleigh quotient minimization for absolutely one-homogeneous functionals , 2019, Inverse Problems.

[14]  Leah Bar,et al.  Unsupervised Deep Learning Algorithm for PDE-based Forward and Inverse Problems , 2019, ArXiv.

[15]  E Weinan,et al.  Solving many-electron Schrödinger equation using deep neural networks , 2018, J. Comput. Phys..

[16]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[17]  David Pfau,et al.  Spectral Inference Networks: Unifying Deep and Spectral Learning , 2018, ICLR.

[18]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations , 2017, ArXiv.

[19]  Roberto Cipolla,et al.  Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Farid Bozorgnia,et al.  Convergence of Inverse Power Method for First Eigenvalue of p-Laplace Operator , 2016 .

[21]  Kevin Gimpel,et al.  Gaussian Error Linear Units (GELUs) , 2016, 1606.08415.

[22]  Yang Song,et al.  Improving the Robustness of Deep Neural Networks via Stability Training , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Quan Yuan,et al.  Bounds to eigenvalues of the Laplacian on L-shaped domain by variational methods , 2009, J. Comput. Appl. Math..

[26]  Donald Estep,et al.  A power method for nonlinear operators , 2007 .

[27]  D. Sorensen.  Numerical methods for large eigenvalue problems , 2002, Acta Numerica.

[28]  G. W. Stewart,et al.  A Krylov-Schur Algorithm for Large Eigenproblems , 2001, SIAM J. Matrix Anal. Appl..

[29]  Andrew V. Knyazev,et al.  Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method , 2001, SIAM J. Sci. Comput..

[30]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Alan J. Laub,et al.  Power methods for calculating eigenvalues and eigenvectors of spectral operators on Hilbert spaces , 1995 .

[32]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[33]  Leah Bar,et al.  Strong Solutions for PDE-Based Tomography by Unsupervised Learning , 2021, SIAM J. Imaging Sci..

[34]  M. Sadd.  Formulation and solution strategies , 2021, Elasticity.

[35]  G. Miller,et al.  Spectral Graph Theory and The Laplacian Paradigm Spring 2020 Lecture 9 : Bounding Eigenvalues , 2020 : Feb 5 , 2020 .

[36]  Karin Rothschild,et al.  A Course In Functional Analysis , 2016 .

[37]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[38]  R. Bronson.  Chapter 4 – Eigenvalues, Eigenvectors, and Differential Equations , 2014 .

[39]  N. Benouhiba,et al.  On the solutions of the (p,q)-Laplacian problem at resonance☆ , 2013 .

[40]  Ahmed K. Noor,et al.  Computational Structural Mechanics , 2019 .

[41]  P. Bassanini,et al.  Elliptic Partial Differential Equations of Second Order , 1997 .

[42]  G. M.,et al.  Partial Differential Equations I , 2023, Applied Mathematical Sciences.