Differentiable Cloth Simulation for Inverse Problems

We propose a differentiable cloth simulator that can be embedded as a layer in deep neural networks. This approach provides an effective, robust framework for modeling cloth dynamics, self-collisions, and contacts. Due to the high dimensionality of the dynamical system in modeling cloth, traditional gradient computation for collision response can become impractical. To address this problem, we propose to compute the gradient directly using QR decomposition of a much smaller matrix. Experimental results indicate that our method can speed up backpropagation by two orders of magnitude. We demonstrate the presented approach on a number of inverse problems, including parameter estimation and motion control for cloth.

[1]  Eitan Grinspun,et al.  Robust treatment of simultaneous collisions , 2008, ACM Trans. Graph..

[2]  Jonas Degrave,et al.  A DIFFERENTIABLE PHYSICS ENGINE FOR DEEP LEARNING IN ROBOTICS , 2016, Front. Neurorobot..

[3]  Daniel L. K. Yamins,et al.  Flexible Neural Representation for Physics Prediction , 2018, NeurIPS.

[4]  Wolfgang Straßer,et al.  A fast finite element solution for cloth modelling , 2003, 11th Pacific Conference onComputer Graphics and Applications, 2003. Proceedings..

[5]  J. Nathan Kutz,et al.  Deep learning in fluid dynamics , 2017, Journal of Fluid Mechanics.

[6]  Joshua B. Tenenbaum,et al.  End-to-End Differentiable Physics for Learning and Control , 2018, NeurIPS.

[7]  Mathieu Desbrun,et al.  Discrete shells , 2003, SCA '03.

[8]  Michael I. Jordan,et al.  RLlib: Abstractions for Distributed Reinforcement Learning , 2017, ICML.

[9]  James F. O'Brien,et al.  Bringing clothing into desired configurations with limited perception , 2011, 2011 IEEE International Conference on Robotics and Automation.

[10]  Marc Toussaint,et al.  Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning , 2018, Robotics: Science and Systems.

[11]  Miguel A. Otaduy,et al.  Learning‐Based Animation of Clothing for Virtual Try‐On , 2019, Comput. Graph. Forum.

[12]  Connor Schenck,et al.  SPNets: Differentiable Fluid Dynamics for Deep Neural Networks , 2018, CoRL.

[13]  Jiajun Wu,et al.  Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids , 2018, ICLR.

[14]  Daniel Cremers,et al.  DeepWrinkles: Accurate and Realistic Clothing Modeling , 2018, ECCV.

[15]  Richard W. Cottle,et al.  Linear Complementarity Problem , 2009, Encyclopedia of Optimization.

[16]  Ken Perlin,et al.  Accelerating Eulerian Fluid Simulation With Convolutional Networks , 2016, ICML.

[17]  Debora S. Marks,et al.  Learning Protein Structure with a Differentiable Simulator , 2018, ICLR.

[18]  William T. Freeman,et al.  Estimating the Material Properties of Fabric from Video , 2013, 2013 IEEE International Conference on Computer Vision.

[19]  J. Zico Kolter,et al.  OptNet: Differentiable Optimization as a Layer in Neural Networks , 2017, ICML.

[20]  Dinesh Manocha,et al.  Fast continuous collision detection using deforming non-penetration filters , 2010, I3D '10.

[21]  Pascal Fua,et al.  GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Draping , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[22]  James F. O'Brien,et al.  Adaptive anisotropic remeshing for cloth simulation , 2012, ACM Trans. Graph..

[23]  Andrew P. Witkin,et al.  Large steps in cloth simulation , 1998, SIGGRAPH.

[24]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[25]  Huamin Wang,et al.  Data-driven elastic models for cloth: modeling and measurement , 2011, ACM Trans. Graph..

[26]  Mykel J. Kochenderfer,et al.  Deep Dynamical Modeling and Control of Unsteady Fluid Flows , 2018, NeurIPS.

[27]  Jiancheng Liu,et al.  ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[28]  Trevor Darrell,et al.  A geometric approach to robotic laundry folding , 2012, Int. J. Robotics Res..

[29]  Ming C. Lin,et al.  Learning-Based Cloth Material Recovery from Video , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Michael J. Black,et al.  ClothCap , 2017, ACM Trans. Graph..

[31]  Michael Bradley Cline Rigid Body Simulation with Contact and Constraints , 2002 .