Diffusion model based data generation for partial differential equations

In a preliminary attempt to address the problem of data scarcity in physics-based machine learning, we introduce a novel methodology for data generation in physics-based simulations. Our motivation is to overcome the limitations posed by the limited availability of numerical data. To achieve this, we leverage a diffusion model that allows us to generate synthetic data samples and test them for two canonical cases: (a) the steady 2-D Poisson equation, and (b) the forced unsteady 2-D Navier-Stokes (NS) {vorticity-transport} equation in a confined box. By comparing the generated data samples against outputs from classical solvers, we assess their accuracy and examine their adherence to the underlying physics laws. In this way, we emphasize the importance of not only satisfying visual and statistical comparisons with solver data but also ensuring the generated data's conformity to physics laws, thus enabling their effective utilization in downstream tasks.

[1]  Nikolaos N. Vlassis,et al.  Denoising diffusion algorithm for inverse design of microstructures with fine-tuned nonlinear material properties , 2023, Computer Methods in Applied Mechanics and Engineering.

[2]  Jae Hyun Lim,et al.  Score-based Diffusion Models in Function Space , 2023, ArXiv.

[3]  S. Sommer,et al.  A Denoising Diffusion Model for Fluid Field Prediction , 2023, ArXiv.

[4]  A. Farimani,et al.  A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction , 2022, J. Comput. Phys..

[5]  C. Hill,et al.  A composable machine-learning approach for steady-state simulations on high-resolution grids , 2022, NeurIPS.

[6]  David J. Fleet,et al.  Video Diffusion Models , 2022, NeurIPS.

[7]  B. Ommer,et al.  High-Resolution Image Synthesis with Latent Diffusion Models , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  G. Weymouth.  Data-driven Multi-Grid solver for accelerated pressure projection , 2021, Computers & Fluids.

[9]  Sergio Hoyas,et al.  Towards extraction of orthogonal and parsimonious non-linear modes from turbulent flows , 2021, Expert Syst. Appl..

[10]  Prafulla Dhariwal,et al.  Diffusion Models Beat GANs on Image Synthesis , 2021, NeurIPS.

[11]  David J. Fleet,et al.  Image Super-Resolution via Iterative Refinement , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Nikola B. Kovachki,et al.  Fourier Neural Operator for Parametric Partial Differential Equations , 2020, ICLR.

[13]  Jiaming Song,et al.  Denoising Diffusion Implicit Models , 2020, ICLR.

[14]  Karthik Duraisamy,et al.  Perspectives on machine learning-augmented Reynolds-averaged and large eddy simulation models of turbulence , 2020, Physical Review Fluids.

[15]  Heiga Zen,et al.  WaveGrad: Estimating Gradients for Waveform Generation , 2020, ICLR.

[16]  Pieter Abbeel,et al.  Denoising Diffusion Probabilistic Models , 2020, NeurIPS.

[17]  Stefano Ermon,et al.  Improved Techniques for Training Score-Based Generative Models , 2020, NeurIPS.

[18]  Chris Hill,et al.  DiscretizationNet: A Machine-Learning based solver for Navier-Stokes Equations using Finite Volume Discretization , 2020, Computer Methods in Applied Mechanics and Engineering.

[19]  George Em Karniadakis,et al.  DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators , 2019, ArXiv.

[20]  Stephan Hoyer,et al.  Learning data-driven discretizations for partial differential equations , 2018, Proceedings of the National Academy of Sciences.

[21]  Kai Fukami,et al.  Nonlinear mode decomposition with convolutional neural networks for fluid dynamics , 2019, Journal of Fluid Mechanics.

[22]  Prakash Vedula,et al.  Subgrid modelling for two-dimensional turbulence using neural networks , 2018, Journal of Fluid Mechanics.

[23]  Chen Sun,et al.  Revisiting Unreasonable Effectiveness of Data in Deep Learning Era , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[25]  Surya Ganguli,et al.  Deep Unsupervised Learning using Nonequilibrium Thermodynamics , 2015, ICML.