Learning Linear Dynamical Systems with Semi-Parametric Least Squares

We analyze a simple prefiltered variation of the least squares estimator for the problem of estimation with biased, semi-parametric noise, an error model studied more broadly in causal statistics and active learning. We prove an oracle inequality which demonstrates that this procedure provably mitigates the variance introduced by long-term dependencies. We then demonstrate that prefiltered least squares yields, to our knowledge, the first algorithm that provably estimates the parameters of partially-observed linear systems that attains rates which do not not incur a worst-case dependence on the rate at which these dependencies decay. The algorithm is provably consistent even for systems which satisfy the weaker marginal stability condition obeyed by many classical models based on Newtonian mechanics. In this context, our semi-parametric framework yields guarantees for both stochastic and worst-case noise.

[1]  Sun-Yuan Kung,et al.  A new identification and model reduction algorithm via singular value decomposition , 1978 .

[2]  M. Hautus.  Strong detectability and observers , 1983 .

[3]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[4]  Lei Guo,et al.  Least-squares identification for ARMAX models without the positive real condition , 1989 .

[5]  Joshua D. Angrist,et al.  Identification of Causal Effects Using Instrumental Variables , 1993 .

[6]  J. Doyle,et al.  Robust and optimal control , 1995, Proceedings of 35th IEEE Conference on Decision and Control.

[7]  Bo Wahlberg,et al.  Analysis of state space system identification methods based on instrumental variables and subspace fitting , 1997, Autom..

[8]  John Shawe-Taylor,et al.  Structural Risk Minimization Over Data-Dependent Hierarchies , 1998, IEEE Trans. Inf. Theory.

[9]  P. Tilli.  Singular values and eigenvalues of non-hermitian block Toeplitz matrices , 1996 .

[10]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[11]  Lex Weaver,et al.  The Optimal Reward Baseline for Gradient-Based Reinforcement Learning , 2001, UAI.

[12]  Peter L. Bartlett,et al.  Variance Reduction Techniques for Gradient Estimates in Reinforcement Learning , 2001, J. Mach. Learn. Res..

[13]  W. Zheng.  A revisit to least-squares parameter estimation of ARMAX systems , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[14]  Marco Lovera,et al.  On the role of prefiltering in nonlinear system identification , 2005, IEEE Transactions on Automatic Control.

[15]  Si-Zhao Joe Qin,et al.  An overview of subspace identification , 2006, Comput. Chem. Eng..

[16]  J. Geanakoplos,et al.  Generalized Instrumental Variables Estimation of Nonlinear Rational Expectations Models , 2007 .

[17]  Csaba Szepesvári,et al.  Online Least Squares Estimation with Self-Normalized Processes: An Application to Bandit Problems , 2011, ArXiv.

[18]  Yong Zhang,et al.  Unbiased identification of a class of multi-input single-output systems with correlated disturbances using bias compensation methods , 2011, Math. Comput. Model..

[19]  D. Wang.  Brief paper: Lleast squares-based recursive and iterative estimation for output error moving average systems using data filtering , 2011 .

[20]  Parikshit Shah,et al.  Linear system identification via atomic norm regularization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[21]  Holger Rauhut,et al.  Suprema of Chaos Processes and the Restricted Isometry Property , 2012, ArXiv.

[22]  F. Ding.  Two-stage least squares based iterative estimation algorithm for CARARMA system modeling ☆ , 2013 .

[23]  M. Talagrand.  Upper and Lower Bounds for Stochastic Processes: Modern Methods and Classical Problems , 2014 .

[24]  Håkan Hjalmarsson,et al.  A weighted least-squares method for parameter estimation in structured models , 2014, 53rd IEEE Conference on Decision and Control.

[25]  Miguel Galrinho,et al.  Least Squares Methods for System Identification of Structured Models , 2016 .

[26]  Christian Hansen,et al.  Double/Debiased/Neyman Machine Learning of Treatment Effects , 2017, 1701.08687.

[27]  Karan Singh,et al.  Learning Linear Dynamical Systems via Spectral Filtering , 2017, NIPS.

[28]  Jascha Sohl-Dickstein,et al.  REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models , 2017, NIPS.

[29]  Alexander Rakhlin,et al.  How fast can linear dynamical systems be learned? , 2018, ArXiv.

[30]  Ambuj Tewari,et al.  Finite Time Identification in Unstable Linear Systems , 2017, Autom..

[31]  Michael I. Jordan,et al.  Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification , 2018, COLT.

[32]  Tengyu Ma,et al.  Gradient Descent Learns Linear Dynamical Systems , 2016, J. Mach. Learn. Res..

[33]  Akshay Krishnamurthy,et al.  Semiparametric Contextual Bandits , 2018, ICML.

[34]  Xian Wu,et al.  Variance reduced value iteration and faster algorithms for solving Markov decision processes , 2017, SODA.

[35]  Sham M. Kakade,et al.  Variance Reduction Methods for Sublinear Reinforcement Learning , 2018, ArXiv.

[36]  Roman Vershynin,et al.  High-Dimensional Probability , 2018 .

[37]  Yi Zhang,et al.  Spectral Filtering for General Linear Dynamical Systems , 2018, NeurIPS.

[38]  Samet Oymak,et al.  Stochastic Gradient Descent Learns State Equations with Nonlinear Activations , 2018, COLT.

[39]  Alexander Rakhlin,et al.  Near optimal finite time identification of arbitrary linear dynamical systems , 2018, ICML.

[40]  Munther A. Dahleh,et al.  Finite-Time System Identification for Partially Observed LTI Systems of Unknown Order , 2019, ArXiv.

[41]  Samet Oymak,et al.  Non-asymptotic Identification of LTI Systems from a Single Trajectory , 2018, 2019 American Control Conference (ACC).