Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitions

Effective control and prediction of dynamical systems often require appropriate handling of continuous–time and discrete, event–triggered processes. Stochastic hybrid systems (SHSs), common across engineering domains, provide a formalism for dynamical systems subject to discrete, possibly stochastic, state jumps and multi–modal continuous–time flows. Despite the versatility and importance of SHSs across applications, a general procedure for the explicit learning of both discrete events and multi–mode continuous dynamics remains an open problem. This work introduces Neural Hybrid Automata (NHAs), a recipe for learning SHS dynamics without a priori knowledge on the number of modes and intermodal transition dynamics. NHAs provide a systematic inference method based on normalizing flows, neural differential equations and self–supervision. We showcase NHAs on several tasks, including mode recovery and flow learning in systems with stochastic transitions, and end–to–end learning of hierarchical robot controllers.

[1]  A. Yamashita,et al.  Identification of a Class of Hybrid Dynamical Systems , 2020 .

[2]  David Duvenaud,et al.  Latent Ordinary Differential Equations for Irregularly-Sampled Time Series , 2019, NeurIPS.

[3]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[4]  Ali Ramadhan,et al.  Universal Differential Equations for Scientific Machine Learning , 2020, ArXiv.

[5]  Diane J. Cook,et al.  A survey of methods for time series change point detection , 2017, Knowledge and Information Systems.

[6]  Julia Hirschberg,et al.  V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure , 2007, EMNLP.

[7]  Yaofeng Desmond Zhong,et al.  A Differentiable Contact Model to Extend Lagrangian and Hamiltonian Neural Networks for Modeling Hybrid Dynamics , 2021, ArXiv.

[8]  Stephan Günnemann,et al.  Intensity-Free Learning of Temporal Point Processes , 2020, ICLR.

[9]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[10]  J. Dormand,et al.  A family of embedded Runge-Kutta formulae , 1980 .

[11]  Jianghai Hu,et al.  Stochastic Hybrid Systems , 2013 .

[12]  Christos G. Cassandras,et al.  Perturbation analysis and optimization of stochastic flow networks , 2004, IEEE Transactions on Automatic Control.

[13]  Jharna Majumdar,et al.  Kinematics, Localization and Control of Differential Drive Mobile Robot , 2014 .

[14]  J. Hespanha,et al.  Stochastic models for chemically reacting systems using polynomial stochastic hybrid systems , 2005 .

[15]  Edward R. Dougherty,et al.  Review of stochastic hybrid systems with applications in biological systems modeling and analysis , 2017, EURASIP J. Bioinform. Syst. Biol..

[16]  Hod Lipson,et al.  Learning symbolic representations of hybrid dynamical systems , 2012, J. Mach. Learn. Res..

[17]  Daniel E. Koditschek,et al.  A hybrid systems model for simple manipulation and self-manipulation systems , 2015, Int. J. Robotics Res..

[18]  Ingmar Posner,et al.  First Steps: Latent-Space Control with Semantic Constraints for Quadruped Locomotion , 2020, ArXiv.

[19]  Edward Choi,et al.  Neural Ordinary Differential Equations for Intervention Modeling , 2020, ArXiv.

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Hajime Asama,et al.  Dissecting Neural ODEs , 2020, NeurIPS.

[22]  L. Shampine,et al.  Event location for ordinary differential equations , 2000 .

[23]  Ralf L. M. Peeters,et al.  Identification of Piecewise Linear Models of Complex Dynamical Systems , 2011, ArXiv.

[24]  Kathryn Fraughnaugh,et al.  Introduction to graph theory , 1973, Mathematical Gazette.

[25]  Iain Murray,et al.  Neural Spline Flows , 2019, NeurIPS.

[26]  John Guckenheimer,et al.  The Dynamics of Legged Locomotion: Models, Analyses, and Challenges , 2006, SIAM Rev..

[27]  Marc Peter Deisenroth,et al.  Learning Contact Dynamics using Physically Structured Neural Networks , 2021, AISTATS.

[28]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[29]  René Vidal,et al.  Identification of Hybrid Systems: A Tutorial , 2007, Eur. J. Control.

[30]  Ricky T. Q. Chen,et al.  Neural Spatio-Temporal Point Processes , 2020, ICLR.

[31]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[32]  Esko Valkeila,et al.  An Introduction to the Theory of Point Processes, Volume II: General Theory and Structure, 2nd Edition by Daryl J. Daley, David Vere‐Jones , 2008 .

[33]  René Vidal,et al.  A continuous optimization framework for hybrid system identification , 2011, Autom..

[34]  Yee Whye Teh,et al.  Augmented Neural ODEs , 2019, NeurIPS.

[35]  Ricky T. Q. Chen,et al.  Learning Neural Event Functions for Ordinary Differential Equations , 2020, ICLR.

[36]  Austin R. Benson,et al.  Neural Jump Stochastic Differential Equations , 2019, NeurIPS.

[37]  Fionn Murtagh,et al.  Algorithms for hierarchical clustering: an overview , 2012, WIREs Data Mining Knowl. Discov..

[38]  Stefan Schaal,et al.  Policy Gradient Methods for Robotics , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  Noah D. Goodman,et al.  Pyro: Deep Universal Probabilistic Programming , 2018, J. Mach. Learn. Res..

[40]  Gérard Bloch,et al.  Hybrid System Identification , 2019 .

[41]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[42]  Naomi Ehrich Leonard,et al.  Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Control , 2020, NeurIPS.

[43]  João Pedro Hespanha,et al.  Stochastic Hybrid Systems: Application to Communication Networks , 2004, HSCC.

[44]  Derya Birant,et al.  ST-DBSCAN: An algorithm for clustering spatial-temporal data , 2007, Data Knowl. Eng..

[45]  Yoshua Bengio,et al.  Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation , 2013, ArXiv.