Alternating ConvLSTM: Learning Force Propagation with Alternate State Updates

Data-driven simulation is an important step-forward in computational physics when traditional numerical methods meet their limits. Learning-based simulators have been widely studied in past years; however, most previous works view simulation as a general spatial-temporal prediction problem and take little physical guidance in designing their neural network architectures. In this paper, we introduce the alternating convolutional Long Short-Term Memory (Alt-ConvLSTM) that models the force propagation mechanisms in a deformable object with near-uniform material properties. Specifically, we propose an accumulation state, and let the network update its cell state and the accumulation state alternately. We demonstrate how this novel scheme imitates the alternate updates of the first and second-order terms in the forward Euler method of numerical PDE solvers. Benefiting from this, our network only requires a small number of parameters, independent of the number of the simulated particles, and also retains the essential features in ConvLSTM, making it naturally applicable to sequential data with spatial inputs and outputs. We validate our Alt-ConvLSTM on human soft tissue simulation with thousands of particles and consistent body pose changes. Experimental results show that Alt-ConvLSTM efficiently models the material kinetic features and greatly outperforms vanilla ConvLSTM with only the single state update.

[1]  Charles K. Birdsall,et al.  Particle-in-cell charged-particle simulations, plus Monte Carlo collisions with neutral atoms, PIC-MCC , 1991 .

[2]  Miguel A. Otaduy,et al.  Learning‐Based Animation of Clothing for Virtual Try‐On , 2019, Comput. Graph. Forum.

[3]  Jürgen Schmidhuber,et al.  Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions , 2018, ICLR.

[4]  Miguel A. Otaduy,et al.  Learning Nonlinear Soft-Tissue Dynamics for Interactive Avatars , 2018, PACMCGIT.

[5]  Miles Macklin,et al.  Position based fluids , 2013, ACM Trans. Graph..

[6]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[7]  Ronald Fedkiw,et al.  Two-Way Coupled SPH and Particle Level Set Fluid Simulation , 2008, IEEE Transactions on Visualization and Computer Graphics.

[8]  Guirong Liu,et al.  Smoothed Particle Hydrodynamics (SPH): an Overview and Recent Developments , 2010 .

[9]  Daniel Cremers,et al.  DeepWrinkles: Accurate and Realistic Clothing Modeling , 2018, ECCV.

[10]  Ronald Fedkiw,et al.  A Pixel‐Based Framework for Data‐Driven Clothing , 2018, Comput. Graph. Forum.

[11]  Connor Schenck,et al.  SPNets: Differentiable Fluid Dynamics for Deep Neural Networks , 2018, CoRL.

[12]  Jiayu Zhou,et al.  Recurrent Encoder-Decoder Networks for Time-Varying Dense Prediction , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[13]  Michael J. Black,et al.  Dynamic FAUST: Registering Human Bodies in Motion , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Ronald Fedkiw,et al.  Visual simulation of smoke , 2001, SIGGRAPH.

[15]  Jiajun Wu,et al.  Propagation Networks for Model-Based Control Under Partial Observation , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[16]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[17]  Jiajun Wu,et al.  Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids , 2018, ICLR.

[18]  J. Brackbill,et al.  Flip: A low-dissipation, particle-in-cell method for fluid flow , 1988 .

[19]  Yu Zheng,et al.  Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction , 2016, AAAI.

[20]  Alexey Stomakhin,et al.  A material point method for snow simulation , 2013, ACM Trans. Graph..

[21]  Leonidas J. Guibas,et al.  Predicting the Physical Dynamics of Unseen 3D Objects , 2020, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[22]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling , 2015, CVPR 2015.

[23]  C. Karen Liu,et al.  Animating human dressing , 2015, ACM Trans. Graph..

[24]  Joshua B. Tenenbaum,et al.  A Compositional Object-Based Approach to Learning Physical Dynamics , 2016, ICLR.

[25]  R. Zemel,et al.  Neural Relational Inference for Interacting Systems , 2018, ICML.

[26]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[27]  Leonidas J. Guibas,et al.  SAPIEN: A SimulAted Part-Based Interactive ENvironment , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Ronald Fedkiw,et al.  Practical animation of liquids , 2001, SIGGRAPH.

[29]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[30]  Ronald Fedkiw,et al.  Coercing Machine Learning to Output Physically Accurate Results , 2020, J. Comput. Phys..

[31]  In-Kwon Lee,et al.  Efficient Cloth Simulation using Miniature Cloth and Upscaling Deep Neural Networks , 2019, ArXiv.

[32]  Dinesh K. Pai,et al.  The human touch , 2018, ACM Trans. Graph..

[33]  Meekyoung Kim,et al.  Data-driven physics for human soft tissue animation , 2017, ACM Trans. Graph..

[34]  Xiaolin Hu,et al.  Recurrent convolutional neural network for object recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Daniel L. K. Yamins,et al.  Flexible Neural Representation for Physics Prediction , 2018, NeurIPS.

[36]  Miguel A. Otaduy,et al.  SoftSMPL: Data‐driven Modeling of Nonlinear Soft‐tissue Dynamics for Parametric Humans , 2020, Comput. Graph. Forum.

[37]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Ronald Fedkiw,et al.  Recovering Geometric Information with Learned Texture Perturbations , 2020, Proc. ACM Comput. Graph. Interact. Tech..

[39]  Jan Kautz,et al.  Video-to-Video Synthesis , 2018, NeurIPS.

[40]  Ilya Kostrikov,et al.  Surface Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Alexei A. Efros,et al.  Everybody Dance Now , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Greg Turk,et al.  Learning to Collaborate From Simulation for Robot-Assisted Dressing , 2019, IEEE Robotics and Automation Letters.

[43]  C. Karen Liu,et al.  Assistive Gym: A Physics Simulation Framework for Assistive Robotics , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[44]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[45]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[46]  Ronald Fedkiw,et al.  A review of level-set methods and some recent applications , 2018, J. Comput. Phys..