SURFNet: Super-Resolution of Turbulent Flows with Transfer Learning using Small Datasets

Deep Learning (DL) algorithms are emerging as a key alternative to computationally expensive CFD simulations. However, state-of-the-art DL approaches require large and high-resolution training data to learn accurate models. The size and availability of such datasets are a major limitation for the development of next-generation data-driven surrogate models for turbulent flows. This paper introduces SURFNet, a transfer learning-based super-resolution flow network. SURFNet primarily trains the DL model on low-resolution datasets and transfer learns the model on a handful of high-resolution flow problems-accelerating the traditional numerical solver independent of the input size. We propose two approaches to transfer learning for the task of super-resolution, namely one-shot and incremental learning. Both approaches entail transfer learning on only one geometry to account for fine-grid flow fields requiring 15× less training data on high-resolution inputs compared to the tiny resolution ($64\times 256$) of the coarse model significantly, reducing the time for both data collection and training. We empirically evaluate SURFNet's performance by solving the Navier-Stokes equations in the turbulent regime on input resolutions up to 256× larger than the coarse model. On four test geometries and eight flow configurations unseen during training, we observe a consistent 2–2.1× speedup over the OpenFOAM physics solver independent of the test geometry and the resolution size (up to $2048 \times 2048$), demonstrating both resolution-invariance and generalization capabilities. Moreover, compared to the baseline model (aka oracle) that collects large training data at $256 \times 256$ and $512 \times 512$ grid resolutions, SURFNet achieves the same performance gain while reducing the combined data collection and training time by 3.6× and 10.2×, respectively. Our approach addresses the challenge of reconstructing high-resolution solutions from coarse grid models trained using low-resolution inputs (i.e., super-resolution) without loss of accuracy and requiring limited computational resources.

[1]  Myoungkyu Lee,et al.  Petascale direct numerical simulation of turbulent channel flow on up to 786K cores , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[2]  George Em Karniadakis,et al.  NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations , 2020, J. Comput. Phys..

[3]  Karthik Kashinath,et al.  Towards Physics-informed Deep Learning for Turbulent Flow Prediction , 2020, KDD.

[4]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[5]  Hui Xiong,et al.  A Comprehensive Survey on Transfer Learning , 2019, Proceedings of the IEEE.

[6]  Michael I. Jordan,et al.  Deep Transfer Learning with Joint Adaptation Networks , 2016, ICML.

[7]  O. Reynolds II. On the dynamical theory of incompressible viscous fluids and the determination of the criterion , 1894, Proceedings of the Royal Society of London.

[8]  Aleksandar Jemcov,et al.  OpenFOAM: A C++ Library for Complex Physics Simulations , 2007 .

[9]  Ruqiang Yan,et al.  Highly Accurate Machine Fault Diagnosis Using Deep Transfer Learning , 2019, IEEE Transactions on Industrial Informatics.

[10]  Wei Li,et al.  Convolutional Neural Networks for Steady Flow Approximation , 2016, KDD.

[11]  Paris Perdikaris,et al.  Understanding and mitigating gradient pathologies in physics-informed neural networks , 2020, ArXiv.

[12]  P. Spalart A One-Equation Turbulence Model for Aerodynamic Flows , 1992 .

[13]  Tomer Michaeli,et al.  Deep-STORM: super-resolution single-molecule microscopy by deep learning , 2018, 1801.09631.

[14]  Linyang Zhu,et al.  Machine learning methods for turbulence modeling in subsonic flows around airfoils , 2018, Physics of Fluids.

[15]  Ken Perlin,et al.  Accelerating Eulerian Fluid Simulation With Convolutional Networks , 2016, ICML.

[16]  Andrew M. Stuart,et al.  SMAI-JCM SMAI Journal of Computational Mathematics Model Reduction And Neural Networks For Parametric PDEs , 2022 .

[17]  Abhinav Vishnu,et al.  CFDNet: a deep learning-based accelerator for fluid simulations , 2020, ICS.

[18]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[19]  Chao Yang,et al.  A Survey on Deep Transfer Learning , 2018, ICANN.

[20]  Nils Thuerey,et al.  Deep Learning Methods for Reynolds-Averaged Navier–Stokes Simulations of Airfoil Flows , 2018, AIAA Journal.

[21]  Thomas Wolf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[22]  Kamyar Azizzadenesheli,et al.  Fourier Neural Operator for Parametric Partial Differential Equations , 2021, ICLR.

[23]  G. Park,et al.  Wall-Modeled Large-Eddy Simulation for Complex Turbulent Flows. , 2018, Annual review of fluid mechanics.

[24]  J. Casacuberta,et al.  Transitional Flow Dynamics Behind a Micro-Ramp , 2019, Flow, Turbulence and Combustion.

[25]  Piyush Kumar Shukla,et al.  Deep Transfer Learning Based Classification Model for COVID-19 Disease , 2020, IRBM.

[26]  Feng Liu,et al.  Roofline Guided Design and Analysis of a Multi-stencil CFD Solver for Multicore Performance , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[27]  Ruben Juanes,et al.  SciANN: A Keras wrapper for scientific computations and physics-informed deep learning using artificial neural networks , 2020, ArXiv.

[28]  Sebastian Ruder,et al.  Neural transfer learning for natural language processing , 2019 .

[29]  Aparna Chandramowlishwaran,et al.  Unsteady Navier-Stokes Computations on GPU Architectures , 2017 .

[30]  Kamyar Azizzadenesheli,et al.  Neural Operator: Graph Kernel Network for Partial Differential Equations , 2020, ICLR 2020.

[31]  M. Boussinesq Essai sur la théorie des eaux courantes , 1873 .

[32]  Steven C. H. Hoi,et al.  Deep Learning for Image Super-Resolution: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Zhen Xie,et al.  Adaptive neural network-based approximation to accelerate eulerian fluid simulation , 2019, SC.

[34]  Mw Hirsch,et al.  Chaos In Dynamical Systems , 2016 .

[35]  George Em Karniadakis,et al.  DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators , 2019, ArXiv.

[36]  James Demmel,et al.  ImageNet Training in Minutes , 2017, ICPP.

[37]  Jiri Blazek,et al.  Computational Fluid Dynamics: Principles and Applications , 2001 .

[38]  E. Jennings,et al.  Accelerating RANS turbulence modeling using potential flow and machine learning , 2019, 1910.10878.

[39]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Cihat Duru,et al.  CNNFOIL: convolutional encoder decoder modeling for pressure fields around airfoils , 2020, Neural Computing and Applications.

[41]  Zhiping Mao,et al.  DeepXDE: A Deep Learning Library for Solving Differential Equations , 2019, AAAI Spring Symposium: MLPS.

[42]  Youjun Xu,et al.  Transfer Learning for Drug Discovery. , 2020, Journal of medicinal chemistry.

[43]  MESHFREEFLOWNET: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework , 2020, SC20: International Conference for High Performance Computing, Networking, Storage and Analysis.

[44]  Jian-Xun Wang,et al.  Super-resolution and denoising of fluid flow using physics-informed convolutional neural networks without high-resolution labels , 2020, Physics of Fluids.

[45]  Ramin Bostanabad,et al.  Train Once and Use Forever: Solving Boundary Value Problems in Unseen Domains with Pre-trained Deep Learning Models , 2021, ArXiv.

[46]  Michel Desmarais,et al.  Spatial Convolution Neural Network for Efficient Prediction of Aerodynamic Coefficients , 2020, AIAA Scitech 2021 Forum.

[47]  Wei Wang,et al.  Deep Learning for Single Image Super-Resolution: A Brief Review , 2018, IEEE Transactions on Multimedia.