PINNeik: Eikonal solution using physics-informed neural networks

The eikonal equation is utilized across a wide spectrum of science and engineering disciplines. In seismology, it regulates seismic wave traveltimes needed for applications like source localization, imaging, and inversion. Several numerical algorithms have been developed over the years to solve the eikonal equation. However, these methods require considerable modifications to incorporate additional physics, such as anisotropy, and may even breakdown for certain complex forms of the eikonal equation, requiring approximation methods. Moreover, they suffer from computational bottleneck when repeated computations are needed for perturbations in the velocity model and/or the source location, particularly in large 3D models. Here, we propose an algorithm to solve the eikonal equation based on the emerging paradigm of physics-informed neural networks (PINNs). By minimizing a loss function formed by imposing the eikonal equation, we train a neural network to output traveltimes that are consistent with the underlying partial differential equation. We observe sufficiently high traveltime accuracy for most applications of interest. We also demonstrate how the proposed algorithm harnesses machine learning techniques like transfer learning and surrogate modeling to speed up traveltime computations for updated velocity models and source locations. Furthermore, we use a locally adaptive activation function and adaptive weighting of the terms in the loss function to improve convergence rate and solution accuracy. We also show the flexibility of the method in incorporating medium anisotropy and free-surface topography compared to conventional methods that require significant algorithmic modifications. These properties of the proposed PINN eikonal solver are highly desirable in obtaining a flexible and efficient forward modeling engine for seismological applications.

[1]  Stéphane Operto,et al.  3D ray+Born migration/inversion—Part 1: Theory , 2003 .

[2]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[3]  E. Rouy,et al.  A viscosity solutions approach to shape-from-shading , 1992 .

[4]  B. Santer,et al.  Statistical significance of climate sensitivity predictors obtained by data mining , 2014 .

[5]  Liwei Wang,et al.  The Expressive Power of Neural Networks: A View from the Width , 2017, NIPS.

[6]  V. Arnold Mathematical Methods of Classical Mechanics , 1974 .

[7]  T. Alkhalifah,et al.  Solving the frequency-domain acoustic VTI wave equation using physics-informed neural networks , 2021, Geophysical Journal International.

[8]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[9]  J. A. Sethian,et al.  Level set methods for etching, deposition and photolithography development , 1996, Journal of Technology Computer Aided Design TCAD.

[10]  Naif Alajlan,et al.  Artificial Neural Network Methods for the Solution of Second Order Boundary Value Problems , 2019, Computers, Materials & Continua.

[11]  Aamir Ahmad,et al.  Towards Optimal Robot Navigation in Domestic Spaces , 2014, RoboCup.

[12]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[13]  Paris Perdikaris,et al.  When and why PINNs fail to train: A neural tangent kernel perspective , 2020, J. Comput. Phys..

[14]  P. Roux,et al.  Relative location of microseismicity , 2015 .

[15]  Yan Pailhas,et al.  Path Planning for Autonomous Underwater Vehicles , 2007, IEEE Transactions on Robotics.

[16]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[17]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[18]  Javier V. Gómez,et al.  Fast Methods for Eikonal Equations: An Experimental Survey , 2015, IEEE Access.

[19]  Jianliang Qian,et al.  An adaptive finite-difference method for traveltimes and amplitudes , 2002 .

[20]  James A. Sethian,et al.  A unified approach to noise removal, image enhancement, and shape recovery , 1996, IEEE Trans. Image Process..

[21]  Charles C. Margossian,et al.  A review of automatic differentiation and its efficient implementation , 2018, WIREs Data Mining Knowl. Discov..

[22]  Hongkai Zhao,et al.  Fast sweeping method for the factored eikonal equation , 2009, J. Comput. Phys..

[23]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[24]  Jingye Li,et al.  Physics informed neural networks for velocity inversion , 2019, SEG Technical Program Expanded Abstracts 2019.

[25]  Haiqiang Lan,et al.  Topography-dependent eikonal equation and its solver for calculating first-arrival traveltimes with an irregular surface , 2013 .

[26]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[27]  R. Juanes,et al.  SciANN: A Keras/TensorFlow wrapper for scientific computations and physics-informed deep learning using artificial neural networks , 2020, Computer Methods in Applied Mechanics and Engineering.

[28]  J. Sethian,et al.  A Fast Level Set Method for Propagating Interfaces , 1995 .

[29]  J. Tsitsiklis,et al.  Efficient algorithms for globally optimal trajectories , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[30]  Taghi M. Khoshgoftaar,et al.  Deep learning applications and challenges in big data analytics , 2015, Journal of Big Data.

[31]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[32]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[33]  Ruben Juanes,et al.  A nonlocal physics-informed deep learning framework using the peridynamic differential operator , 2020, ArXiv.

[34]  J A Sethian,et al.  A fast marching level set method for monotonically advancing fronts. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Ken Perlin,et al.  Accelerating Eulerian Fluid Simulation With Convolutional Networks , 2016, ICML.

[36]  D. Lawton Computation of refraction static corrections using first-break traveltime differences , 1989 .

[37]  Stéphane Operto,et al.  3 D ray + Born migration / inversion — Part 1 : Theory , 2003 .

[38]  Paris Perdikaris,et al.  Understanding and mitigating gradient pathologies in physics-informed neural networks , 2020, ArXiv.

[39]  Pejman Tahmasebi,et al.  Accelerating geostatistical modeling using geostatistics-informed machine Learning , 2021, Comput. Geosci..

[40]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[41]  Christopher V. Alvino,et al.  Efficient segmentation based on Eikonal and diffusion equations , 2007, Int. J. Comput. Math..

[42]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[43]  Alexander M. Bronstein,et al.  Affine-invariant geodesic geometry of deformable 3D shapes , 2010, Comput. Graph..

[44]  Can Evren Yarman,et al.  An iterative, fast-sweeping-based eikonal solver for 3D tilted anisotropic media , 2015 .

[45]  Fan Yang,et al.  First arrival traveltime tomography using supervised descent learning technique , 2019, Inverse Problems.

[46]  J. Masoliver,et al.  From classical to quantum mechanics through optics , 2009, 0909.3258.

[47]  Jeffrey M. McMahon Basic Electromagnetic Theory , 2011 .

[48]  R. Kimmel,et al.  An efficient solution to the eikonal equation on parametric manifolds , 2004 .

[49]  D. Lazer,et al.  The Parable of Google Flu: Traps in Big Data Analysis , 2014, Science.

[50]  Luis Moreno,et al.  Path Planning for Mars Rovers Using the Fast Marching Method , 2015, ROBOT.

[51]  Nagiza F. Samatova,et al.  Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data , 2016, IEEE Transactions on Knowledge and Data Engineering.

[52]  Benoit M. Dawant,et al.  Registration of medical images using an interpolated closest point transform: method and validation , 2003, SPIE Medical Imaging.

[53]  Richard A. Geyer,et al.  To: “LESSONS IN SEISMIC COMPUTING” , 1959 .

[54]  George Em Karniadakis,et al.  Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks , 2020, Proceedings of the Royal Society A.

[55]  Yoshua Bengio,et al.  On the Spectral Bias of Neural Networks , 2018, ICML.

[56]  Hyuk Lee,et al.  Neural algorithm for solving differential equations , 1990 .

[57]  Conal Elliott,et al.  The simple essence of automatic differentiation , 2018, Proc. ACM Program. Lang..

[58]  P. Tahmasebi,et al.  Physics informed machine learning: Seismic wave equation , 2020, Geoscience Frontiers.

[59]  Barak A. Pearlmutter,et al.  Automatic differentiation in machine learning: a survey , 2015, J. Mach. Learn. Res..

[60]  Phillip Colella,et al.  Two new methods for simulating photolithography development in 3D , 1996, Advanced Lithography.

[61]  Hongkai Zhao,et al.  A fast sweeping method for Eikonal equations , 2004, Math. Comput..