NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation framework

We present SimNet, an AI-driven multi-physics simulation framework, to accelerate simulations across a wide range of disciplines in science and engineering. Compared to traditional numerical solvers, SimNet addresses a wide range of use cases - coupled forward simulations without any training data, inverse and data assimilation problems. SimNet offers fast turnaround time by enabling parameterized system representation that solves for multiple configurations simultaneously, as opposed to the traditional solvers that solve for one configuration at a time. SimNet is integrated with parameterized constructive solid geometry as well as STL modules to generate point clouds. Furthermore, it is customizable with APIs that enable user extensions to geometry, physics and network architecture. It has advanced network architectures that are optimized for high-performance GPU computing, and offers scalable performance for multi-GPU and multi-Node implementation with accelerated linear algebra as well as FP32, FP64 and TF32 computations. In this paper we review the neural network solver methodology, the SimNet architecture, and the various features that are needed for effective solution of the PDEs. We present real-world use cases that range from challenging forward multi-physics simulations with turbulence and complex 3D geometries, to industrial design optimization and inverse problems that are not addressed efficiently by the traditional solvers. Extensive comparisons of SimNet results with open source and commercial solvers show good correlation.

[1]  Quoc V. Le,et al.  Swish: a Self-Gated Activation Function , 2017, 1710.05941.

[2]  Paris Perdikaris,et al.  When and why PINNs fail to train: A neural tangent kernel perspective , 2020, J. Comput. Phys..

[3]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[4]  Quoc V. Le,et al.  Searching for Activation Functions , 2018, arXiv.

[5]  Paris Perdikaris,et al.  Understanding and mitigating gradient pathologies in physics-informed neural networks , 2020, ArXiv.

[6]  George Em Karniadakis,et al.  On the Convergence and generalization of Physics Informed Neural Networks , 2020, ArXiv.

[7]  George Em Karniadakis,et al.  hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition , 2020, Computer Methods in Applied Mechanics and Engineering.

[8]  George Em Karniadakis,et al.  Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations , 2020, Science.

[9]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[10]  Wei Li,et al.  Convolutional Neural Networks for Steady Flow Approximation , 2016, KDD.

[11]  Todd A. Oliver,et al.  Solving differential equations using deep neural networks , 2020, Neurocomputing.

[12]  Changhoon Lee,et al.  Prediction of turbulent heat transfer using convolutional neural networks , 2019, Journal of Fluid Mechanics.

[13]  Yoshua Bengio,et al.  On the Spectral Bias of Neural Networks , 2018, ICML.

[14]  Jonathan T. Barron,et al.  Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains , 2020, NeurIPS.

[15]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[16]  Kaiming He,et al.  Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour , 2017, ArXiv.

[17]  Andy R. Terrel,et al.  SymPy: Symbolic computing in Python , 2017, PeerJ Prepr..

[18]  Zhiping Mao,et al.  DeepXDE: A Deep Learning Library for Solving Differential Equations , 2019, AAAI Spring Symposium: MLPS.

[19]  George Em Karniadakis,et al.  Adaptive activation functions accelerate convergence in deep and physics-informed neural networks , 2019, J. Comput. Phys..

[20]  Ruben Juanes,et al.  SciANN: A Keras wrapper for scientific computations and physics-informed deep learning using artificial neural networks , 2020, ArXiv.

[21]  R. Juanes,et al.  SciANN: A Keras/TensorFlow wrapper for scientific computations and physics-informed deep learning using artificial neural networks , 2020, Computer Methods in Applied Mechanics and Engineering.

[22]  Pratul P. Srinivasan,et al.  NeRF , 2020, ECCV.

[23]  Kaj Nyström,et al.  A unified deep artificial neural network approach to partial differential equations in complex geometries , 2017, Neurocomputing.

[24]  George Em Karniadakis,et al.  NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations , 2020, J. Comput. Phys..

[25]  Gang Bao,et al.  Weak Adversarial Networks for High-dimensional Partial Differential Equations , 2019, J. Comput. Phys..

[26]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  Ruben Juanes,et al.  A deep learning framework for solution and discovery in solid mechanics: linear elasticity , 2020, ArXiv.

[29]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[30]  Gordon Wetzstein,et al.  Implicit Neural Representations with Periodic Activation Functions , 2020, NeurIPS.

[31]  Oliver Hennigh,et al.  Lat-Net: Compressing Lattice Boltzmann Flow Simulations using Deep Neural Networks , 2017, 1705.09036.

[32]  Qing Nie,et al.  DifferentialEquations.jl – A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia , 2017, Journal of Open Research Software.

[33]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[34]  Ruben Juanes,et al.  A deep learning framework for solution and discovery in solid mechanics , 2020 .

[35]  D. L. Young,et al.  A novel vector potential formulation of 3D Navier-Stokes equations with through-flow boundaries by a local meshless method , 2015, J. Comput. Phys..

[36]  Zhao Chen,et al.  GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks , 2017, ICML.