Uniformly accurate machine learning-based hydrodynamic models for kinetic equations

Significance This paper addresses 2 very important issues of current interest: multiscale modeling in the absence of scale separation and building interpretable and truly reliable physical models using machine learning. We demonstrate that machine learning can indeed help us to build reliable multiscale models for problems with which classical multiscale methods have had trouble. To this end, one has to develop the appropriate models or algorithms for each of the 3 major components in the machine-learning procedure: labeling the data, learning from the data, and exploring the state space. We use the kinetic equation as an example and demonstrate that uniformly accurate moment systems can be constructed this way. A framework is introduced for constructing interpretable and truly reliable reduced models for multiscale problems in situations without scale separation. Hydrodynamic approximation to the kinetic equation is used as an example to illustrate the main steps and issues involved. To this end, a set of generalized moments are constructed first to optimally represent the underlying velocity distribution. The well-known closure problem is then solved with the aim of best capturing the associated dynamics of the kinetic equation. The issue of physical constraints such as Galilean invariance is addressed and an active-learning procedure is introduced to help ensure that the dataset used is representative enough. The reduced system takes the form of a conventional moment system and works regardless of the numerical discretization used. Numerical results are presented for the BGK (Bhatnagar–Gross–Krook) model and binary collision of Maxwell molecules. We demonstrate that the reduced model achieves a uniform accuracy in a wide range of Knudsen numbers spanning from the hydrodynamic limit to free molecular flow.

[1]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[2]  E Weinan,et al.  Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics , 2017, Physical review letters.

[3]  Stéphane Mallat,et al.  Solid Harmonic Wavelet Scattering for Predictions of Molecule Properties , 2018, The Journal of chemical physics.

[4]  Naoya Takeishi,et al.  Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition , 2017, NIPS.

[5]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[6]  Shi Jin,et al.  A class of asymptotic-preserving schemes for kinetic equations and related problems with stiff sources , 2009, J. Comput. Phys..

[7]  Ruo Li,et al.  Globally Hyperbolic Regularization of Grad's Moment System , 2012 .

[8]  Francis J. Alexander,et al.  The direct simulation Monte Carlo method , 1997 .

[9]  Ruo Li,et al.  Globally hyperbolic regularized moment method with applications to microflow simulation , 2013 .

[10]  E. Weinan,et al.  Deep Potential: a general representation of a many-body potential energy surface , 2017, 1707.01478.

[11]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[12]  R. Caflisch The fluid dynamic limit of the nonlinear boltzmann equation , 1980 .

[13]  Ruo Li,et al.  Globally Hyperbolic Regularization of Grad's Moment System , 2011, 1111.3409.

[14]  Zheng Ma,et al.  A fast spectral method for the inelastic Boltzmann collision operator and application to heated granular gases , 2019, J. Comput. Phys..

[15]  Ruo Li,et al.  A Framework on Moment Model Reduction for Kinetic Equation , 2015, SIAM J. Appl. Math..

[16]  Jingwei Hu,et al.  A Fast Spectral Method for the Boltzmann Collision Operator with General Collision Kernels , 2016, SIAM J. Sci. Comput..

[17]  R. C. Macridis A review , 1963 .

[18]  P. Bhatnagar,et al.  A Model for Collision Processes in Gases. I. Small Amplitude Processes in Charged and Neutral One-Component Systems , 1954 .

[19]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[21]  Lorenzo Pareschi,et al.  An introduction to Monte Carlo method for the Boltzmann equation , 2001 .

[22]  François Golse,et al.  Kinetic equations and asympotic theory , 2000 .

[23]  T. G. Cowling,et al.  The mathematical theory of non-uniform gases : an account of the kinetic theory of viscosity, thermal conduction, and diffusion in gases , 1954 .

[24]  Petros Koumoutsakos,et al.  Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[25]  Jinlong Wu,et al.  Physics-informed machine learning approach for reconstructing Reynolds stress modeling discrepancies based on DNS data , 2016, 1606.07987.

[26]  Karthik Duraisamy,et al.  Turbulence Modeling in the Age of Data , 2018, Annual Review of Fluid Mechanics.

[27]  E Weinan,et al.  Model Reduction with Memory and the Machine Learning of Dynamical Systems , 2018, Communications in Computational Physics.

[28]  Steven L. Brunton,et al.  Data-Driven Identification of Parametric Partial Differential Equations , 2018, SIAM J. Appl. Dyn. Syst..

[29]  Ruo Li,et al.  Numerical Regularized Moment Method of Arbitrary Order for Boltzmann-BGK Equation , 2010, SIAM J. Sci. Comput..

[30]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[31]  Naoya Takeishi,et al.  Sparse nonnegative dynamic mode decomposition , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[32]  D. Burnett,et al.  The Distribution of Velocities in a Slightly Non‐Uniform Gas , 1935 .

[33]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[34]  Ioannis G Kevrekidis,et al.  Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator. , 2017, Chaos.

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  E Weinan,et al.  End-to-end Symmetry Preserving Inter-atomic Potential Energy Model for Finite and Extended Systems , 2018, NeurIPS.

[37]  K. Carlson,et al.  Turbulent Flows , 2020, Finite Analytic Method in Flows and Heat Transfer.

[38]  E. Toro The HLL and HLLC Riemann Solvers , 1997 .

[39]  C. D. Levermore,et al.  Moment closure hierarchies for kinetic theories , 1996 .

[40]  Ruo Li,et al.  An Efficient NRxx Method for Boltzmann-BGK Equation , 2012, J. Sci. Comput..

[41]  Katepalli R. Sreenivasan,et al.  Deep learning in turbulent convection networks , 2019, Proceedings of the National Academy of Sciences.

[42]  Bin Dong,et al.  PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network , 2018, J. Comput. Phys..

[43]  C. Cercignani Small and Large Mean Free Paths , 1988 .

[44]  E Weinan,et al.  Heterogeneous multiscale methods: A review , 2007 .

[45]  H. Grad On the kinetic theory of rarefied gases , 1949 .

[46]  François Golse,et al.  Fluid dynamic limits of kinetic equations II convergence proofs for the boltzmann equation , 1993 .

[47]  Alexander J. Smola,et al.  Deep Sets , 2017, 1703.06114.

[48]  Lexing Ying,et al.  A fast spectral algorithm for the quantum Boltzmann collision operator , 2012 .

[49]  Raphael Aronson,et al.  Theory and application of the Boltzmann equation , 1976 .

[50]  G. Bird Molecular Gas Dynamics and the Direct Simulation of Gas Flows , 1994 .

[51]  E Weinan,et al.  Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation , 2018, Physical Review Materials.

[52]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[53]  Steven L. Brunton,et al.  Data-driven discovery of coordinates and governing equations , 2019, Proceedings of the National Academy of Sciences.

[54]  E. Weinan Principles of Multiscale Modeling , 2011 .

[55]  Steven L. Brunton,et al.  Discovery of Nonlinear Multiscale Systems: Sampling Strategies and Embeddings , 2018, SIAM J. Appl. Dyn. Syst..

[56]  R. LeVeque Finite Volume Methods for Hyperbolic Problems: Characteristics and Riemann Problems for Linear Hyperbolic Equations , 2002 .

[57]  T. Nishida Fluid dynamical limit of the nonlinear Boltzmann equation to the level of the compressible Euler equation , 1978 .

[58]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[59]  Randall J. LeVeque,et al.  Clawpack: building an open source ecosystem for solving hyperbolic PDEs , 2016, PeerJ Comput. Sci..