DiffTaichi: Differentiable Programming for Physical Simulation

We present DiffTaichi, a new differentiable programming language tailored for building high-performance differentiable physical simulators. Based on an imperative programming language, DiffTaichi generates gradients of simulation steps using source code transformations that preserve arithmetic intensity and parallelism. A light-weight tape is used to record the whole simulation program structure and replay the gradient kernels in a reversed order, for end-to-end backpropagation. We demonstrate the performance and productivity of our language in gradient-based learning and optimization tasks on 10 different physical simulators. For example, a differentiable elastic object simulator written in our language is 4.2x shorter than the hand-engineered CUDA version yet runs as fast, and is 188x faster than the TensorFlow implementation. Using our differentiable programs, neural network controllers are typically optimized within only tens of iterations.

[1]  Chenfanfu Jiang,et al.  Multi-species simulation of porous sand and water mixtures , 2017, ACM Trans. Graph..

[2]  Joshua B. Tenenbaum,et al.  End-to-End Differentiable Physics for Learning and Control , 2018, NeurIPS.

[3]  Tatsuya Harada,et al.  Neural 3D Mesh Renderer , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Connor Schenck,et al.  SPNets: Differentiable Fluid Dynamics for Deep Neural Networks , 2018, CoRL.

[5]  Jos Stam,et al.  Stable fluids , 1999, SIGGRAPH.

[6]  Jaakko Lehtinen,et al.  Differentiable Monte Carlo ray tracing through edge sampling , 2018, ACM Trans. Graph..

[7]  G. Herman,et al.  Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography. , 1970, Journal of theoretical biology.

[8]  G. M. Ostrovskii,et al.  Automatic computation of derivatives with the use of the multilevel differentiating technique—1. Algorithmic basis , 1985 .

[9]  Andre Pradhana,et al.  A moving least squares material point method with displacement discontinuity and two-way rigid body coupling , 2018, ACM Trans. Graph..

[10]  Jean Utke,et al.  OpenAD/F: A Modular Open-Source Tool for Automatic Differentiation of Fortran Codes , 2008, TOMS.

[11]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[12]  Michael J. Black,et al.  OpenDR: An Approximate Differentiable Renderer , 2014, ECCV.

[13]  Laurent Hascoët,et al.  The Tapenade automatic differentiation tool: Principles, model, and specification , 2013, TOMS.

[14]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[15]  Steven M. Seitz,et al.  Interactive manipulation of rigid body simulations , 2000, SIGGRAPH.

[16]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[17]  Eftychios Sifakis,et al.  Computing the Singular Value Decomposition of 3x3 matrices with minimal branching and elementary floating point operations , 2011 .

[18]  Razvan Pascanu,et al.  Theano: A CPU and GPU Math Compiler in Python , 2010, SciPy.

[19]  Abderrahmane Kheddar,et al.  Fast Continuous Collision Detection between Rigid Bodies , 2002, Comput. Graph. Forum.

[20]  Joshua B. Tenenbaum,et al.  A Compositional Object-Based Approach to Learning Physical Dynamics , 2016, ICLR.

[21]  R. E. Wengert,et al.  A simple automatic derivative evaluation program , 1964, Commun. ACM.

[22]  Jonas Degrave,et al.  A DIFFERENTIABLE PHYSICS ENGINE FOR DEEP LEARNING IN ROBOTICS , 2016, Front. Neurorobot..

[23]  Daniel L. K. Yamins,et al.  Flexible Neural Representation for Physics Prediction , 2018, NeurIPS.

[24]  Sebastian Hack,et al.  Whole-function vectorization , 2011, International Symposium on Code Generation and Optimization (CGO 2011).

[25]  Frédo Durand,et al.  Taichi , 2019, ACM Trans. Graph..

[26]  Alan Edelman,et al.  A Differentiable Programming System to Bridge Machine Learning and Scientific Computing , 2019, ArXiv.

[27]  Frédo Durand,et al.  Differentiable programming for image processing and deep learning in halide , 2018, ACM Trans. Graph..

[28]  M. Pharr,et al.  ispc: A SPMD compiler for high-performance CPU programming , 2012, 2012 Innovative Parallel Computing (InPar).

[29]  Jiancheng Liu,et al.  ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[30]  Doug L. James,et al.  Toward wave-based sound synthesis for computer animation , 2018, ACM Trans. Graph..

[31]  Merlin Nimier-David,et al.  Mitsuba 2 , 2019, ACM Trans. Graph..

[32]  R. Errico.  What is an adjoint model , 1997 .

[33]  Barak A. Pearlmutter,et al.  Reverse-mode AD in a functional framework: Lambda the ultimate backpropagator , 2008, TOPL.

[34]  Chenfanfu Jiang,et al.  The Material Point Method for the Physics-Based Simulation of Solids and Fluids , 2015 .

[35]  Frédo Durand,et al.  Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines , 2013, PLDI 2013.

[36]  Ming C. Lin,et al.  Differentiable Cloth Simulation for Inverse Problems , 2019, NeurIPS.