Fourier Neural Operator for Parametric Partial Differential Equations

The classical development of neural networks has primarily focused on learning mappings between finite-dimensional Euclidean spaces. Recently, this has been generalized to neural operators that learn mappings between function spaces. For partial differential equations (PDEs), neural operators directly learn the mapping from any functional parametric dependence to the solution. Thus, they learn an entire family of PDEs, in contrast to classical methods which solve one instance of the equation. In this work, we formulate a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture. We perform experiments on Burgers' equation, Darcy flow, and the Navier-Stokes equation (including the turbulent regime). Our Fourier neural operator shows state-of-the-art performance compared to existing neural network methodologies and it is up to three orders of magnitude faster compared to traditional PDE solvers.

[1]  Stephan Hoyer,et al.  Machine learning–accelerated computational fluid dynamics , 2021, Proceedings of the National Academy of Sciences.

[2]  Nicholas H. Nelsen,et al.  The Random Feature Model for Input-Output Maps between Banach Spaces , 2020, SIAM J. Sci. Comput..

[3]  Nikola B. Kovachki,et al.  Model Reduction and Neural Networks for Parametric PDEs , 2020, The SMAI journal of computational mathematics.

[4]  Kamyar Azizzadenesheli,et al.  EikoNet: Solving the Eikonal Equation With Deep Neural Networks , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Ravi G. Patel,et al.  A physics-informed operator regression framework for extracting data-driven continuum models , 2020, ArXiv.

[6]  Gordon Wetzstein,et al.  Implicit Neural Representations with Periodic Activation Functions , 2020, NeurIPS.

[7]  Nikola B. Kovachki,et al.  Multipole Graph Neural Operator for Parametric Partial Differential Equations , 2020, NeurIPS.

[8]  Karthik Kashinath,et al.  MESHFREEFLOWNET: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework , 2020, SC20: International Conference for High Performance Computing, Networking, Storage and Analysis.

[9]  Kamyar Azizzadenesheli,et al.  Neural Operator: Graph Kernel Network for Partial Differential Equations , 2020, ICLR 2020.

[10]  Karthik Kashinath,et al.  Enforcing Physical Constraints in CNNs through Differentiable PDE Layer , 2020, ICLR 2020.

[11]  Rui Wang,et al.  Towards Physics-informed Deep Learning for Turbulent Flow Prediction , 2019, KDD.

[12]  Karthik Duraisamy,et al.  Physics-Informed Probabilistic Learning of Linear Embeddings of Nonlinear Dynamics with Guaranteed Stability , 2019, SIAM J. Appl. Dyn. Syst..

[13]  Lexing Ying,et al.  Solving parametric PDE problems with artificial neural networks , 2017, European Journal of Applied Mathematics.

[14]  Yadong Mu,et al.  Fast Fourier Convolution , 2020, NeurIPS.

[15]  George Em Karniadakis,et al.  DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators , 2019, ArXiv.

[16]  Karthik Duraisamy,et al.  Prediction of aerodynamic flow fields using convolutional neural networks , 2019, Computational Mechanics.

[17]  Leah Bar,et al.  Unsupervised Deep Learning Algorithm for PDE-based Forward and Inverse Problems , 2019, ArXiv.

[18]  Ronen Basri,et al.  Learning to Optimize Multigrid PDE Solvers , 2019, ICML.

[19]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[20]  Lexing Ying,et al.  BCR-Net: a neural network based on the nonstandard wavelet form , 2018, J. Comput. Phys..

[21]  Lexing Ying,et al.  A Multiscale Neural Network Based on Hierarchical Matrices , 2018, Multiscale Model. Simul..

[22]  Nicholas Zabaras,et al.  Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification , 2018, J. Comput. Phys..

[23]  E Weinan,et al.  The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems , 2017, Communications in Mathematics and Statistics.

[24]  Ronald A. DeVore,et al.  Chapter 3: The Theoretical Foundation of Reduced Basis Methods , 2017 .

[25]  Jonas Adler,et al.  Solving ill-posed inverse problems using iterative deep neural networks , 2017, ArXiv.

[26]  Wei Li,et al.  Convolutional Neural Networks for Steady Flow Approximation , 2016, KDD.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[29]  Yann LeCun,et al.  Fast Training of Convolutional Networks through FFTs , 2013, ICLR.

[30]  G. Roberts,et al.  MCMC Methods for Functions: ModifyingOld Algorithms to Make Them Faster , 2012, 1202.0709.

[31]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[32]  L. F. Mingo,et al.  FOURIER NEURAL NETWORKS: AN APPROACH WITH SINUSOIDAL ACTIVATION FUNCTIONS 1 , 2003 .

[33]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[34]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.