Statistics for sparse, high-dimensional, and nonparametric system identification

Local linearization techniques are an important class of nonparametric system identification. Identifying local linearizations in practice involves solving a linear regression problem that is ill-posed. The problem can be ill-posed either if the dynamics of the system lie on a manifold of lower dimension than the ambient space or if there are not enough measurements of all the modes of the dynamics of the system. We describe a set of linear regression estimators that can handle data lying on a lower-dimension manifold. These estimators differ from previous estimators, because these estimators are able to improve estimator performance by exploiting the sparsity of the system - the existence of direct interconnections between only some of the states - and can work in the “large p, small n” setting in which the number of states is comparable to the number of data points. We describe our system identification procedure, which consists of a presmoothing step and a regression step, and then we apply this procedure to data taken from a quadrotor helicopter. We use this data set to compare our procedure with existing procedures.

[1]  P. Bickel,et al.  Local polynomial regression on unknown manifolds , 2007, 0708.0983.

[2]  Claire J. Tomlin,et al.  Quadrotor Helicopter Flight Dynamics and Control: Theory and Experiment , 2007 .

[3]  Stefan Schaal,et al.  A Bayesian approach to empirical local linearization for robotics , 2008, 2008 IEEE International Conference on Robotics and Automation.

[4]  Sabine Van Huffel,et al.  Total least squares problem - computational aspects and analysis , 1991, Frontiers in applied mathematics.

[5]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[6]  W. Massy.  Principal Components Regression in Exploratory Statistical Research , 1965 .

[7]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[8]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[9]  Noureddine El Karoui.  Tracy–Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices , 2005, math/0503109.

[10]  Claire J. Tomlin,et al.  Quadrotor Helicopter Trajectory Tracking Control , 2008 .

[11]  Ja-Yong Koo,et al.  Optimal Rates of Convergence for Nonparametric Statistical Inverse Problems , 1993 .

[12]  W. F. Phillips,et al.  Review of Attitude Representations Used for Aircraft Kinematics , 2001 .

[13]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[14]  P. Bickel,et al.  Regression on manifolds: Estimation of the exterior derivative , 2011, 1103.1457.

[15]  John M. Lee.  Introduction to Smooth Manifolds , 2002 .

[16]  Andrew W. Moore,et al.  Locally Weighted Learning for Control , 1997, Artificial Intelligence Review.

[17]  Tor Arne Johansen,et al.  Design and analysis of gain-scheduled control using local controller networks , 1997 .

[18]  C. Desoer,et al.  Linear System Theory , 1963 .

[19]  John G. Proakis,et al.  Digital signal processing (3rd ed.): principles, algorithms, and applications , 1996 .

[20]  N. Meinshausen,et al.  LASSO-TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH-DIMENSIONAL DATA , 2008, 0806.0145.

[21]  I. Johnstone,et al.  Sparse Principal Components Analysis , 2009, 0901.4392.

[22]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[23]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[24]  M. Wand,et al.  Multivariate Locally Weighted Least Squares Regression , 1994 .

[25]  H. Schneeweiß,et al.  Consistent estimation of a regression with errors in the variables , 1976 .

[26]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[27]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[28]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[29]  P. Bickel,et al.  Covariance regularization by thresholding , 2009, 0901.3079.

[30]  Stefan Schaal,et al.  Incremental Online Learning in High Dimensions , 2005, Neural Computation.

[31]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .