Online Instrumental Variable Regression with Applications to Online Linear System Identification

Instrumental variable regression (IVR) is a statistical technique utilized to recover unbiased estimators when there are errors in the independent variables. Estimator bias in learned time series models can yield poor performance in applications such as long-term prediction and filtering where the recursive use of the model results in the accumulation of propagated error. However, prior work addressed the IVR objective in the batch setting, where it is necessary to store the entire dataset in memory - an infeasible requirement in large dataset scenarios. In this work, we develop Online Instrumental Variable Regression (OIVR), an algorithm that is capable of updating the learned estimator with streaming data. We show that the online adaptation of IVR enjoys a no-regret performance guarantee with respect to the original batch setting by taking advantage of any no-regret online learning algorithm inside OIVR for the underlying update steps. We experimentally demonstrate the efficacy of our algorithm in combination with popular no-regret online algorithms for the task of learning predictive dynamical system models and on a prototypical econometrics instrumental variable regression problem.

[1]  Byron Boots,et al.  Two Manifold Problems with Applications to Nonlinear System Identification , 2012, ICML.

[2]  Geoffrey J. Gordon,et al.  A New View of Predictive State Methods for Dynamical System Learning , 2015, ArXiv.

[3]  Byron Boots,et al.  Closing the learning-planning loop with predictive state representations , 2009, Int. J. Robotics Res..

[4]  Shalabh,et al.  Linear Models and Generalizations: Least Squares and Alternatives , 2007 .

[5]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[6]  A. Cameron,et al.  Microeconometrics: Methods and Applications , 2005 .

[7]  Byron Boots,et al.  An Online Spectral Learning Algorithm for Partially Observable Nonlinear Dynamical Systems , 2011, AAAI.

[8]  Byron Boots,et al.  Predictive State Temporal Difference Learning , 2010, NIPS.

[9]  Peter L. Bartlett,et al.  Implicit Online Learning , 2010, ICML.

[10]  Elad Hazan,et al.  Logarithmic regret algorithms for online convex optimization , 2006, Machine Learning.

[11]  Satinder P. Singh,et al.  Kernel Predictive Linear Gaussian models for nonlinear stochastic dynamical systems , 2006, ICML.

[12]  Achim Zeileis,et al.  Applied Econometrics with R , 2008 .

[13]  Elad Hazan,et al.  An optimal algorithm for stochastic strongly-convex optimization , 2010, 1006.2425.

[14]  T. Söderström,et al.  Instrumental variable methods for system identification , 1983 .

[15]  Pieter Abbeel,et al.  Exploration and apprenticeship learning in reinforcement learning , 2005, ICML.

[16]  Richard S. Sutton,et al.  Predictive Representations of State , 2001, NIPS.

[17]  Michael R. James,et al.  Predictive State Representations: A New Theory for Modeling Dynamical Systems , 2004, UAI.

[18]  David Card,et al.  Using Geographic Variation in College Proximity to Estimate the Return to Schooling , 1993 .

[19]  Sander Greenland,et al.  An introduction to instrumental variables for epidemiologists. , 2018, International journal of epidemiology.

[20]  Byron Boots,et al.  Closing the learning-planning loop with predictive state representations , 2009, Int. J. Robotics Res..

[21]  H. R. Pitt.  Divergent Series , 1951, Nature.

[22]  Liva Ralaivola,et al.  Dynamical Modeling with Kernels for Nonlinear Time Series Prediction , 2003, NIPS.

[23]  Claudio Gentile,et al.  On the generalization ability of on-line learning algorithms , 2001, IEEE Transactions on Information Theory.

[24]  Nick Littlestone,et al.  From on-line to batch learning , 1989, COLT '89.

[25]  E. Miguel,et al.  Economic Shocks and Civil Conflict: An Instrumental Variables Approach , 2004, Journal of Political Economy.

[26]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.