Hamiltonian Neural Networks

Even though neural networks enjoy widespread use, they still struggle to learn the basic laws of physics. How might we endow them with better inductive biases? In this paper, we draw inspiration from Hamiltonian mechanics to train models that learn and respect exact conservation laws in an unsupervised manner. We evaluate our models on problems where conservation of energy is important, including the two-body problem and pixel observations of a pendulum. Our model trains faster and generalizes better than a regular neural network. An interesting side effect is that our model is perfectly reversible in time.

[1]  E. Batschelet.  Über die numerische Auflösung von Randwertproblemen bei elliptischen partiellen Differentialgleichungen , 1952 .

[2]  Emmy Noether,et al.  Invariant Variation Problems , 2005, physics/0503066.

[3]  Michael Brereton,et al.  A Modern Course in Statistical Physics , 1981 .

[4]  R. Salmon.  HAMILTONIAN FLUID MECHANICS , 1988 .

[5]  C. cohen-tannoudji,et al.  Photons and Atoms: Introduction to Quantum Electrodynamics , 1989 .

[6]  J. J. Sakurai,et al.  Modern Quantum Mechanics, Revised Edition , 1995 .

[7]  Geoffrey E. Hinton,et al.  NeuroAnimator: fast neural network emulation and control of physics-based models , 1998, SIGGRAPH.

[8]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[9]  J. Taylor.  Classical Mechanics , 2002 .

[10]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[11]  A. Pukrittayakamee,et al.  Simultaneous fitting of a potential-energy surface and its corresponding force fields using feedforward neural networks. , 2009, The Journal of chemical physics.

[12]  Hod Lipson,et al.  Evolving robot gaits in hardware: the HyperNEAT generative encoding vs. parameter optimization , 2011, ECAL.

[13]  J. Behler.  Neural network potential-energy surfaces in chemistry: a tool for large-scale simulations. , 2011, Physical chemistry chemical physics : PCCP.

[14]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[15]  K. Müller,et al.  Fast and accurate modeling of molecular atomization energies with machine learning. , 2011, Physical review letters.

[16]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[17]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[18]  M. Gastegger,et al.  High-Dimensional Neural Network Potentials for Organic Reactions and an Improved Training Algorithm. , 2015, Journal of chemical theory and computation.

[19]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[20]  Razvan Pascanu,et al.  A simple neural network module for relational reasoning , 2017, NIPS.

[21]  Razvan Pascanu,et al.  Visual Interaction Networks: Learning a Physics Simulator from Video , 2017, NIPS.

[22]  Alexandre Tkatchenko,et al.  Quantum-chemical insights from deep tensor neural networks , 2016, Nature Communications.

[23]  Klaus-Robert Müller,et al.  Machine learning of accurate energy-conserving molecular force fields , 2016, Science Advances.

[24]  Raquel Urtasun,et al.  The Reversible Residual Network: Backpropagation Without Storing Activations , 2017, NIPS.

[25]  Joshua B. Tenenbaum,et al.  A Compositional Object-Based Approach to Learning Physical Dynamics , 2016, ICLR.

[26]  Ken Perlin,et al.  Accelerating Eulerian Fluid Simulation With Convolutional Networks , 2016, ICML.

[27]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[28]  J S Smith,et al.  ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost , 2016, Chemical science.

[29]  Roger B. Grosse,et al.  Reversible Recurrent Neural Networks , 2018, NeurIPS.

[30]  Joshua B. Tenenbaum,et al.  End-to-End Differentiable Physics for Learning and Control , 2018, NeurIPS.

[31]  Arnold W. M. Smeulders,et al.  i-RevNet: Deep Invertible Networks , 2018, ICLR.

[32]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[33]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[34]  Jessica B. Hamrick,et al.  Relational inductive bias for physical construction in humans and machines , 2018, CogSci.

[35]  Jason Yosinski,et al.  An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution , 2018, NeurIPS.

[36]  David W Toth,et al.  The TensorMol-0.1 model chemistry: a neural network augmented with long-range physics , 2017, Chemical science.

[37]  Jürgen Schmidhuber,et al.  Recurrent World Models Facilitate Policy Evolution , 2018, NeurIPS.

[38]  Austen Lamacraft,et al.  Learning Symmetries of Classical Integrable Systems , 2019, ArXiv.

[39]  S. Girvin,et al.  Modern Condensed Matter Physics , 2019 .

[40]  Jan Peters,et al.  Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning , 2019, ICLR.

[41]  Frank Noé,et al.  Machine Learning of Coarse-Grained Molecular Dynamics Force Fields , 2018, ACS central science.

[42]  Ruben Villegas,et al.  Learning Latent Dynamics for Planning from Pixels , 2018, ICML.

[43]  Renato Renner,et al.  Discovering physical concepts with neural networks , 2018, Physical review letters.

[44]  Jakub W. Pachocki,et al.  Learning dexterous in-hand manipulation , 2018, Int. J. Robotics Res..