Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow

We propose a method for the data‐driven inference of temporal evolutions of physical functions with deep learning. More specifically, we target fluid flow problems, and we propose a novel LSTM‐based approach to predict the changes of the pressure field over time. The central challenge in this context is the high dimensionality of Eulerian space‐time data sets. We demonstrate for the first time that dense 3D+time functions of physics system can be predicted within the latent spaces of neural networks, and we arrive at a neural‐network based simulation algorithm with significant practical speed‐ups. We highlight the capabilities of our method with a series of complex liquid simulations, and with a set of single‐phase buoyancy simulations. With a set of trained networks, our method is more than two orders of magnitudes faster than a traditional pressure solver. Additionally, we present and discuss a series of detailed evaluations for the different components of our algorithm.

[1]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[2]  Gavin S. P. Miller,et al.  Rapid, stable fluid dynamics for computer graphics , 1990, SIGGRAPH.

[3]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Dimitris N. Metaxas,et al.  Realistic Animation of Liquids , 1996, Graphics Interface.

[5]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[6]  Jos Stam,et al.  Stable fluids , 1999, SIGGRAPH.

[7]  Ronald Fedkiw,et al.  Practical animation of liquids , 2001, SIGGRAPH.

[8]  R. Fedkiw,et al.  USING THE PARTICLE LEVEL SET METHOD AND A SECOND ORDER ACCURATE PRESSURE BOUNDARY CONDITION FOR FREE SURFACE FLOWS , 2003 .

[9]  Markus H. Gross,et al.  Particle-based fluid simulation for interactive applications , 2003, SCA '03.

[10]  K. Willcox,et al.  Aerodynamic Data Reconstruction and Inverse Design Using Proper Orthogonal Decomposition , 2004 .

[11]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[12]  Yongning Zhu,et al.  Animating sand as a fluid , 2005, SIGGRAPH 2005.

[13]  Z. Popovic,et al.  Model reduction for real-time fluids , 2006, SIGGRAPH 2006.

[14]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[15]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[16]  Adrien Treuille,et al.  Model reduction for real-time fluids , 2006, ACM Trans. Graph..

[17]  Robert Bridson,et al.  A fast variational framework for accurate solid-fluid coupling , 2007, ACM Trans. Graph..

[18]  Robert Bridson,et al.  A fast variational framework for accurate solid-fluid coupling , 2007, SIGGRAPH 2007.

[19]  Ronald Fedkiw,et al.  Two-way coupling of fluids to rigid and deformable solids and shells , 2008, ACM Trans. Graph..

[20]  Doug L. James,et al.  Wavelet turbulence for fluid simulation , 2008, SIGGRAPH 2008.

[21]  Robert Bridson,et al.  Fluid Simulation for Computer Graphics , 2008 .

[22]  Nancy Argüelles,et al.  Author ' s , 2008 .

[23]  Ken Museth,et al.  Guiding of smoke animations through variational coupling of simulations at different resolutions , 2009, SCA '09.

[24]  Adrien Treuille,et al.  Modular bases for fluid dynamics , 2009, SIGGRAPH 2009.

[25]  Adrien Treuille,et al.  Modular bases for fluid dynamics , 2009, ACM Trans. Graph..

[26]  Ronald Fedkiw,et al.  A novel algorithm for incompressible flow using only a coarse grid projection , 2010, SIGGRAPH 2010.

[27]  Eftychios Sifakis,et al.  A parallel multigrid Poisson solver for fluids simulation on large grids , 2010, SCA '10.

[28]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[29]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[30]  Hujun Bao,et al.  Interactive localized liquid motion editing , 2013, ACM Trans. Graph..

[31]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[32]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[33]  Pieter Abbeel,et al.  Tracking deformable objects with point clouds , 2013, 2013 IEEE International Conference on Robotics and Automation.

[34]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[35]  Katsushi Ikeuchi,et al.  Detecting potential falling objects by inferring human action and natural disturbance , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Matthias Teschner,et al.  Implicit Incompressible SPH , 2014, IEEE Transactions on Visualization and Computer Graphics.

[37]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[38]  Tae-Yong Kim,et al.  Unified particle physics for real-time applications , 2014, ACM Trans. Graph..

[39]  Greg Turk,et al.  Blending liquids , 2014, ACM Trans. Graph..

[40]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[41]  Christopher Wojtan,et al.  A Dimension‐reduced Pressure Solver for Liquid Simulations , 2015, Comput. Graph. Forum.

[42]  Leonidas J. Guibas,et al.  Database‐Assisted Object Retrieval for Real‐Time 3D Reconstruction , 2015, Comput. Graph. Forum.

[43]  Barbara Solenthaler,et al.  Data-driven fluid simulations using regression forests , 2015, ACM Trans. Graph..

[44]  Tsuhan Chen,et al.  3D Reasoning from Blocks to Stability , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[46]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[47]  Theodore Kim,et al.  Eulerian solid-fluid coupling , 2016, ACM Trans. Graph..

[48]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[49]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[50]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[51]  John Flynn,et al.  Deep Stereo: Learning to Predict New Views from the World's Imagery , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Jiajun Wu,et al.  Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling , 2016, NIPS.

[53]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[54]  Xiaohui S. Xie,et al.  DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences , 2015, bioRxiv.

[55]  Cheng Yang,et al.  Data‐driven projection method in fluid simulation , 2016, Comput. Animat. Virtual Worlds.

[56]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[57]  Sophie Papst,et al.  Computational Methods For Fluid Dynamics , 2016 .

[58]  Rüdiger Westermann,et al.  Narrow Band FLIP for Liquid Simulations , 2016, Comput. Graph. Forum.

[59]  Vincent Dumoulin,et al.  Deconvolution and Checkerboard Artifacts , 2016 .

[60]  Lin Yang,et al.  Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation , 2016, NIPS.

[61]  Nils Thürey,et al.  Pre-computed Liquid Spaces with Generative Neural Networks and Optical Flow , 2017, ArXiv.

[62]  Razvan Pascanu,et al.  Visual Interaction Networks: Learning a Physics Simulator from Video , 2017, NIPS.

[63]  Yisong Yue,et al.  Long-term Forecasting using Tensor-Train RNNs , 2017, ArXiv.

[64]  Connor Schenck,et al.  Reasoning About Liquids via Closed-Loop Simulation , 2017, Robotics: Science and Systems.

[65]  Taku Komura,et al.  Phase-functioned neural networks for character control , 2017, ACM Trans. Graph..

[66]  Hao Li,et al.  Photorealistic Facial Texture Inference Using Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  Eitan Grinspun,et al.  Supplemental : A Multi-Scale Model for Simulating Liquid-Hair Interactions , 2017 .

[68]  Sylvain Paris,et al.  Deep Photo Style Transfer , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Glen Berseth,et al.  DeepLoco , 2017, ACM Trans. Graph..

[70]  Niloy J. Mitra,et al.  Learning A Physical Long-term Predictor , 2017, ArXiv.

[71]  Joshua B. Tenenbaum,et al.  A Compositional Object-Based Approach to Learning Physical Dynamics , 2016, ICLR.

[72]  Ken Perlin,et al.  Accelerating Eulerian Fluid Simulation With Convolutional Networks , 2016, ICML.

[73]  Robert Pless,et al.  Deep Feature Interpolation for Image Content Changes , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[74]  Amir Barati Farimani,et al.  Deep Learning the Physics of Transport Phenomena , 2017, ArXiv.

[75]  Yongdong Zhang,et al.  Learning Multimodal Attention LSTM Networks for Video Captioning , 2017, ACM Multimedia.

[76]  N. Thürey,et al.  Data-driven synthesis of smoke flows with CNN-based feature descriptors , 2017, ACM Transactions on Graphics.

[77]  Nils Thürey,et al.  tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , 2018, ACM Trans. Graph..

[78]  Mykel J. Kochenderfer,et al.  Deep Dynamical Modeling and Control of Unsteady Fluid Flows , 2018, NeurIPS.

[79]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[80]  Ahmed H. Elsheikh,et al.  Reduced order modeling of subsurface multiphase flow models using deep residual recurrent neural networks , 2018, ArXiv.

[81]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[82]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.

[83]  Bo Ren,et al.  Fluid directed rigid body control using deep reinforcement learning , 2018, ACM Trans. Graph..

[84]  Nils Thürey,et al.  Generating Liquid Simulations with Deformation-aware Neural Networks , 2017, ICLR.