Supervised Learning for Dynamical System Learning

Recently there has been substantial interest in spectral methods for learning dynamical systems. These methods are popular since they often offer a good tradeoff between computational and statistical efficiency. Unfortunately, they can be difficult to use and extend in practice: e.g., they can make it difficult to incorporate prior information such as sparsity or structure. To address this problem, we present a new view of dynamical system learning: we show how to learn dynamical systems by solving a sequence of ordinary supervised learning problems, thereby allowing users to incorporate prior knowledge via standard techniques such as L1 regularization. Many existing spectral methods are special cases of this new framework, using linear regression as the supervised learner. We demonstrate the effectiveness of our framework by showing examples where nonlinear regression or lasso let us learn better state representations than plain linear regression does; the correctness of these instances follows directly from our general analysis.

[1]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[2]  Sebastian Thrun,et al.  Learning low dimensional predictive representations , 2004, ICML.

[3]  Le Song,et al.  Kernel Bayes' rule: Bayesian inference with positive definite kernels , 2013, J. Mach. Learn. Res..

[4]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[5]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[6]  Joelle Pineau,et al.  Methods of Moments for Learning Stochastic Languages: Unified Presentation and Empirical Comparison , 2014, ICML.

[7]  Daniel J. Hsu,et al.  Tail inequalities for sums of random matrices that depend on the intrinsic dimension , 2012 .

[8]  John Langford,et al.  Learning nonlinear dynamic models , 2009, ICML '09.

[9]  Le Song,et al.  Hilbert Space Embeddings of Hidden Markov Models , 2010, ICML.

[10]  Herbert Jaeger,et al.  Observable Operator Models for Discrete Stochastic Time Series , 2000, Neural Computation.

[11]  Byron Boots,et al.  Spectral Approaches to Learning Predictive Representations , 2011 .

[12]  Byron Boots,et al.  Reduced-Rank Hidden Markov Models , 2009, AISTATS.

[13]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[14]  Shien-Ming Wu,et al.  Time series and system analysis with applications , 1983 .

[15]  Byron Boots,et al.  Hilbert Space Embeddings of Predictive State Representations , 2013, UAI.

[16]  Dean Alderucci.  A SPECTRAL ALGORITHM FOR LEARNING HIDDEN MARKOV MODELS THAT HAVE SILENT STATES , 2015 .

[17]  Byron Boots,et al.  Closing the learning-planning loop with predictive state representations , 2009, Int. J. Robotics Res..

[18]  Anima Anandkumar,et al.  Tensor decompositions for learning latent variable models , 2012, J. Mach. Learn. Res..

[19]  Kenneth R. Koedinger,et al.  A Data Repository for the EDM Community: The PSLC DataShop , 2010 .

[20]  Byron Boots,et al.  Two Manifold Problems with Applications to Nonlinear System Identification , 2012, ICML.

[21]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[22]  Bart De Moor,et al.  Subspace Identification for Linear Systems: Theory ― Implementation ― Applications , 2011 .

[23]  Alexander J. Smola,et al.  Hilbert space embeddings of conditional distributions with applications to dynamical systems , 2009, ICML '09.

[24]  J. Tropp.  User-Friendly Tools for Random Matrices: An Introduction , 2012 .

[25]  Byron Boots,et al.  An Online Spectral Learning Algorithm for Partially Observable Nonlinear Dynamical Systems , 2011, AAAI.