Coercing Machine Learning to Output Physically Accurate Results

Many machine/deep learning artificial neural networks are trained to simply be interpolation functions that map input variables to output values interpolated from the training data in a linear/nonlinear fashion. Even when the input/output pairs of the training data are physically accurate (e.g. the results of an experiment or numerical simulation), interpolated quantities can deviate quite far from being physically accurate. Although one could project the output of a network into a physically feasible region, such a postprocess is not captured by the energy function minimized when training the network; thus, the final projected result could incorrectly deviate quite far from the training data. We propose folding any such projection or postprocess directly into the network so that the final result is correctly compared to the training data by the energy function. Although we propose a general approach, we illustrate its efficacy on a specific convolutional neural network that takes in human pose parameters (joint rotations) and outputs a prediction of vertex positions representing a triangulated cloth mesh. While the original network outputs vertex positions with erroneously high stretching and compression energies, the new network trained with our physics prior remedies these issues producing highly improved results.

[1]  Ilias Bilionis,et al.  Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification , 2018, J. Comput. Phys..

[2]  Ronald Fedkiw,et al.  High-Quality Face Capture Using Anatomical Muscles , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[4]  Stephen J. Wright Coordinate descent algorithms , 2015, Mathematical Programming.

[5]  Stephen P. Boyd,et al.  Differentiating Through a Conic Program , 2019 .

[6]  Daniel Thalmann,et al.  Joint-dependent local deformations for hand animation and object grasping , 1989 .

[7]  Thomas J. R. Hughes,et al.  A simple and efficient finite element for plate bending , 1977 .

[8]  Ronald Fedkiw,et al.  Improved Search Strategies with Application to Estimating Facial Blendshape Parameters. , 2020 .

[9]  Stephen P. Boyd,et al.  Differentiating through a cone program , 2019, Journal of Applied and Numerical Optimization.

[10]  Hans C. van Houwelingen,et al.  The Elements of Statistical Learning, Data Mining, Inference, and Prediction. Trevor Hastie, Robert Tibshirani and Jerome Friedman, Springer, New York, 2001. No. of pages: xvi+533. ISBN 0‐387‐95284‐5 , 2004 .

[11]  J. Franklin,et al.  The elements of statistical learning: data mining, inference and prediction , 2005 .

[12]  Ronald Fedkiw,et al.  Invertible finite elements for robust simulation of large deformation , 2004, SCA '04.

[13]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[14]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[15]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[16]  Ronald Fedkiw,et al.  Inequality cloth , 2017, Symposium on Computer Animation.

[17]  Ronald Fedkiw,et al.  Simulation of clothing with folds and wrinkles , 2003, SCA '03.

[18]  Stephen P. Boyd,et al.  ECOS: An SOCP solver for embedded systems , 2013, 2013 European Control Conference (ECC).

[19]  Haibin Chang,et al.  Identification of physical processes via combined data-driven and data-assimilation methods , 2018, J. Comput. Phys..

[20]  Karthik Duraisamy,et al.  A paradigm for data-driven predictive modeling using field inversion and machine learning , 2016, J. Comput. Phys..

[21]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[22]  Boris Polyak Some methods of speeding up the convergence of iteration methods , 1964 .

[24]  Paris Perdikaris,et al.  Machine learning of linear differential equations using Gaussian processes , 2017, J. Comput. Phys..

[25]  Ronald Fedkiw,et al.  Sharp interface approaches and deep learning techniques for multiphase flows , 2019, J. Comput. Phys..

[26]  C. G. Broyden The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[27]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.

[28]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[29]  Timothy Dozat,et al.  Incorporating Nesterov Momentum into Adam , 2016 .

[30]  Ronald Fedkiw,et al.  Robust treatment of collisions, contact and friction for cloth animation , 2002, SIGGRAPH Courses.

[31]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[32]  Alexandre M. Tartakovsky,et al.  Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks , 2018, J. Comput. Phys..

[33]  Jirí Zára,et al.  Skinning with dual quaternions , 2007, SI3D.

[34]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[35]  Ronald Fedkiw,et al.  Robust quasistatic finite elements and flesh simulation , 2005, SCA '05.

[36]  H. Robbins A Stochastic Approximation Method , 1951 .

[37]  Ronald Fedkiw,et al.  Automatic determination of facial muscle activations from sparse motion capture marker data , 2005, ACM Trans. Graph..

[38]  J. Zico Kolter,et al.  OptNet: Differentiable Optimization as a Layer in Neural Networks , 2017, ICML.

[39]  J. P. Lewis,et al.  Direct delta mush skinning and variants , 2019, ACM Trans. Graph..

[40]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[41]  Ronald Fedkiw,et al.  Ieee Transactions on Visualization and Computer Graphics 1 Robust High-resolution Cloth Using Parallelism, History-based Collisions and Accurate Friction , 2022 .

[42]  Ronald Fedkiw,et al.  Finite volume methods for the simulation of skeletal muscle , 2003, SCA '03.

[43]  Ronald Fedkiw,et al.  Art-directed muscle simulation for high-end facial animation , 2016, Symposium on Computer Animation.

[44]  Julia Ling,et al.  Machine learning strategies for systems with invariance properties , 2016, J. Comput. Phys..

[45]  Kyoungmin Lee,et al.  Scalable muscle-actuated human simulation and control , 2019, ACM Trans. Graph..

[46]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[47]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[48]  Ruben Scardovelli,et al.  Computing curvature for volume of fluid methods using machine learning , 2018, J. Comput. Phys..

[49]  Matthew D. Zeiler ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[50]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[51]  Ronald Fedkiw,et al.  Automatic determination of facial muscle activations from sparse motion capture marker data , 2005, SIGGRAPH '05.

[52]  Andrew Selle,et al.  To appear in the ACM SIGGRAPH conference proceedings A Mass Spring Model for Hair Simulation , 2008 .

[53]  Jessica K. Hodgins,et al.  Real-time skeletal skinning with optimized centers of rotation , 2016, ACM Trans. Graph..

[54]  Ronald Fedkiw,et al.  Improved Search Strategies for Determining Facial Expression , 2018, ArXiv.

[55]  S. Delp,et al.  Three-Dimensional Representation of Complex Muscle Architectures and Geometries , 2005, Annals of Biomedical Engineering.

[56]  Ronald Fedkiw,et al.  A Pixel‐Based Framework for Data‐Driven Clothing , 2018, Comput. Graph. Forum.

[57]  Igor Melnyk,et al.  Deep learning algorithm for data-driven simulation of noisy dynamical system , 2018, J. Comput. Phys..

[58]  Ian Stavness,et al.  Unified skinning of rigid and deformable models for anatomical simulations , 2014, SIGGRAPH ASIA Technical Briefs.

[59]  Y. Nesterov A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .