Physics-Informed Neural Operator for Learning Partial Differential Equations

Machine learning methods have recently shown promise in solving partial differential equations (PDEs). They can be classified into two broad categories: approximating the solution function and learning the solution operator. The Physics-Informed Neural Network (PINN) is an example of the former while the Fourier neural operator (FNO) is an example of the latter. Both these approaches have shortcomings. The optimization in PINN is challenging and prone to failure, especially on multi-scale dynamic systems. FNO does not suffer from this optimization issue since it carries out supervised learning on a given dataset, but obtaining such data may be too expensive or infeasible. In this work, we propose the physics-informed neural operator (PINO), where we combine the operating-learning and function-optimization frameworks. This integrated approach improves convergence rates and accuracy over both PINN and FNO models. In the operator-learning phase, PINO learns the solution operator over multiple instances of the parametric PDE family. In the test-time optimization phase, PINO optimizes the pre-trained operator ansatz for the querying instance of the PDE. Experiments show PINO outperforms previous ML methods on many popular PDE families while retaining the extraordinary speed-up of FNO compared to solvers. In particular, PINO accurately solves long temporal transient flows and Kolmogorov flows.

[1]  Luning Sun,et al.  Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data , 2019, Computer Methods in Applied Mechanics and Engineering.

[2]  Balaji Srinivasan,et al.  Physics Informed Extreme Learning Machine (PIELM) - A rapid method for the numerical solution of partial differential equations , 2019, Neurocomputing.

[3]  Eric C. Cyr,et al.  A physics-informed operator regression framework for extracting data-driven continuum models , 2020, ArXiv.

[4]  Kamyar Azizzadenesheli,et al.  Fourier Neural Operator for Parametric Partial Differential Equations , 2021, ICLR.

[5]  Weiwei Sun,et al.  Stability and Convergence of the Crank-Nicolson/Adams-Bashforth scheme for the Time-Dependent Navier-Stokes Equations , 2007, SIAM J. Numer. Anal..

[6]  Ronen Basri,et al.  Learning to Optimize Multigrid PDE Solvers , 2019, ICML.

[7]  George Em Karniadakis,et al.  Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations , 2020, Science.

[8]  E Weinan,et al.  86 PFLOPS Deep Potential Molecular Dynamics simulation of 100 million atoms with ab initio accuracy , 2020, Comput. Phys. Commun..

[9]  Kamyar Azizzadenesheli,et al.  Markov Neural Operators for Learning Chaotic Systems , 2021, ArXiv.

[10]  Kamyar Azizzadenesheli,et al.  Multipole Graph Neural Operator for Parametric Partial Differential Equations , 2020, NeurIPS.

[11]  Kamyar Azizzadenesheli,et al.  Neural Operator: Graph Kernel Network for Partial Differential Equations , 2020, ICLR 2020.

[12]  Karthik Kashinath,et al.  Towards Physics-informed Deep Learning for Turbulent Flow Prediction , 2020, KDD.

[13]  Paris Perdikaris,et al.  Learning the solution operator of parametric partial differential equations with physics-informed DeepONets , 2021, Science advances.

[14]  Zhiping Mao,et al.  DeepXDE: A Deep Learning Library for Solving Differential Equations , 2019, AAAI Spring Symposium: MLPS.

[15]  Nils Thuerey,et al.  Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers , 2020, NeurIPS.

[16]  G. Roberts,et al.  MCMC Methods for Functions: ModifyingOld Algorithms to Make Them Faster , 2012, 1202.0709.

[17]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[18]  Stephan Hoyer,et al.  Machine learning–accelerated computational fluid dynamics , 2021, Proceedings of the National Academy of Sciences.

[19]  Paris Perdikaris,et al.  When and why PINNs fail to train: A neural tangent kernel perspective , 2020, J. Comput. Phys..

[20]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Kamyar Azizzadenesheli,et al.  HypoSVI: Hypocenter inversion with Stein variational inference and Physics Informed Neural Networks , 2021, ArXiv.

[23]  H. Tchelepi,et al.  LIMITATIONS OF PHYSICS INFORMED MACHINE LEARNING FOR NONLINEAR TWO-PHASE TRANSPORT IN POROUS MEDIA , 2020 .

[24]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[25]  Paris Perdikaris,et al.  Understanding and mitigating gradient pathologies in physics-informed neural networks , 2020, ArXiv.

[26]  E Weinan,et al.  The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems , 2017, Communications in Mathematics and Statistics.

[27]  Luning Sun,et al.  PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain , 2021, J. Comput. Phys..

[28]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[29]  Karthik Duraisamy,et al.  Non-linear Independent Dual System (NIDS) for Discretization-independent Surrogate Modeling over Complex Geometries , 2021, ArXiv.

[30]  Minglang Yin,et al.  Physics-informed neural networks (PINNs) for fluid mechanics: a review , 2021, Acta Mechanica Sinica.

[31]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[32]  Jan S. Hesthaven,et al.  FC-based shock-dynamics solver with neural-network localized artificial-viscosity assignment , 2021, J. Comput. Phys. X.

[33]  Lexing Ying,et al.  A Multiscale Neural Network Based on Hierarchical Matrices , 2018, Multiscale Model. Simul..

[34]  Petros Koumoutsakos,et al.  Machine Learning for Fluid Mechanics , 2019, Annual Review of Fluid Mechanics.

[35]  Yaoyu Zhang,et al.  Data-informed Deep Optimization , 2021, ArXiv.

[36]  George Em Karniadakis,et al.  NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations , 2020, J. Comput. Phys..

[37]  George Em Karniadakis,et al.  DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators , 2019, ArXiv.

[38]  Kamyar Azizzadenesheli,et al.  Neural Operator: Learning Maps Between Function Spaces , 2021, ArXiv.

[39]  C. Bruneau,et al.  The 2D lid-driven cavity problem revisited , 2006 .