Noise-induced bias in last principal component modeling of linear system

Last principal component (LPC) modeling relies on principal component transformation, and utilizes the eigenvectors associated with the last (smallest) principal components. When applied to experimental data, it may be considered an alternative to least squares based estimation of model parameters. Experimental results in the literature (cited in the body of the paper) suggest that LPC modeling is inferior to LS, in terms of estimation bias, in the presence of noise. Other results show that LPC produces unbiased estimates only in a very special case. In this paper, we derive explicit expressions for noise-induced bias in LPC-based identification. We investigate static systems with input actuator and measurement noise, and discrete dynamic systems with output measurement noise. We show that, indeed, LPC-based estimates are biased even when LS-based ones are not, and when the LS estimate is also biased, the LPC estimate has the LS bias plus an additional term. The theoretical results are supported by simulation studies.

[1]  Weihua Li,et al.  Isolation enhanced principal component analysis , 1999 .

[2]  Umberto Soverini,et al.  The frisch scheme in dynamic system identification , 1990, Autom..

[3]  M. Moonen,et al.  QSVD approach to on- and off-line state-space identification , 1990 .

[4]  Petre Stoica,et al.  System identification from noisy measurements by using instrumental variables and subspace fitting , 1996 .

[5]  Chun-Bo Feng,et al.  Unbiased parameter estimation of linear systems in the presence of input and output noise , 1989 .

[6]  Sabine Van Huffel,et al.  Comparison of total least squares and instrumental variable methods for parameter estimation of transfer function models , 1989 .

[7]  G. C. Tiao,et al.  Modeling Multiple Time Series with Applications , 1981 .

[8]  S. Joe Qin,et al.  Consistent dynamic PCA based on errors-in-variables subspace identification , 2001 .

[9]  K. Roeder,et al.  Journal of the American Statistical Association: Comment , 2006 .

[10]  Janos Gertler,et al.  Principal Component Analysis and Parity Relations - A Strong Duality , 1997 .

[11]  Torsten Söderström,et al.  Identification of stochastic linear systems in presence of input noise , 1981, Autom..

[12]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[13]  I. Jolliffe.  Principal Component Analysis , 2002 .

[14]  Venkataramanan Balakrishnan,et al.  System identification: theory for the user (second edition): Lennart Ljung; Prentice-Hall, Englewood Cliffs, NJ, 1999, ISBN 0-13-656695-2 , 2002, Autom..

[15]  Theodora Kourti,et al.  Process analysis, monitoring and diagnosis, using multivariate projection methods , 1995 .

[16]  Christos Georgakis,et al.  Disturbance detection and isolation by dynamic principal component analysis , 1995 .

[17]  Janos Gertler,et al.  Noise-induced bias in PCA modeling of linear system , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[18]  Alison J. Burnham,et al.  LATENT VARIABLE MULTIVARIATE REGRESSION MODELING , 1999 .

[19]  W. H. Weinberg,et al.  Combinatorial materials science: Paradigm shift in materials discovery and optimization , 1999 .

[20]  Janos Gertler,et al.  Fault detection and diagnosis in engineering systems , 1998 .

[21]  Michel Verhaegen,et al.  Subspace Algorithms for the Identification of Multivariable Dynamic Errors-in-Variables Models , 1997, Autom..

[22]  Petre Stoica,et al.  Combined instrumental variable and subspace fitting approach to parameter estimation of noisy input-output systems , 1995, IEEE Trans. Signal Process..

[23]  G. P. King,et al.  Extracting qualitative dynamics from experimental data , 1986 .

[24]  P. Holmes,et al.  Suppression of bursting , 1997, Autom..

[25]  W. Zheng,et al.  Identification of a class of dynamic errors-in-variables models , 1992 .

[26]  A. Negiz,et al.  Statistical monitoring of multivariable dynamic processes with state-space models , 1997 .

[27]  Seongkyu Yoon,et al.  Statistical and causal model‐based approaches to fault detection and isolation , 2000 .