Generalized principal components analysis and its application in approximate stochastic realization

The state of a linear system is an information interface betwen the past and the future, and approximate realization is essentially a problem of approximating an apparently high-dimensional interface by a low-order partial state. In this chapter, we generalize the ideas of principal components to the problem of approximating the information interface between two random vectors. Two such generalizations exist in the statistical literature [1,2]. Applications of these generalizations to the partial-state selection problem lead to three approximate stochastic realization methods. We discuss these methods and the partial-state selection criterion that each optimizes. We also study their relation to determinstic identification and balanced model reduction.

[1]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[2]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[3]  N. Levinson.  The Wiener (Root Mean Square) Error Criterion in Filter Design and Prediction , 1946 .

[4]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[5]  Jr. J. L. Brown.  Mean Square Truncation Error in Series Expansions of Random Functions , 1960 .

[6]  Calyampudi R. Rao.  The use and interpretation of principal component analysis in applied research , 1964 .

[7]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[8]  J. P. Burg,et al.  Maximum entropy spectral analysis. , 1967 .

[9]  A. Yaglom.  Outline of some topics in linear extrapolation of stationary random processes , 1967 .

[10]  T. Kailath.  The innovations approach to detection and estimation theory , 1970 .

[11]  B. Anderson,et al.  The choice of signal-process models in Kalman-Bucy filtering , 1971 .

[12]  R. J. Bell,et al.  Introductory Fourier transform spectroscopy , 1972 .

[13]  T. Kailath,et al.  An innovations approach to least-squares estimation--Part VI: Discrete-time innovations representations and recursive estimation , 1973 .

[14]  Thomas Kailath,et al.  A view of three decades of linear filtering theory , 1974, IEEE Trans. Inf. Theory.

[15]  H. Akaike.  Markovian Representation of Stochastic Processes and Its Application to the Analysis of Autoregressive Moving Average Processes , 1974 .

[16]  H. Akaike.  Markovian Representation of Stochastic Processes by Canonical Variables , 1975 .

[17]  Yoshikazu Sawaragi,et al.  System-theoretical approach to model reduction and system-order determination , 1975 .

[18]  Clifford T. Mullis,et al.  Synthesis of minimum roundoff noise fixed point digital filters , 1976 .

[19]  P. Faurre.  Stochastic Realization Algorithms , 1976 .

[20]  D. Lainiotis,et al.  System identification : advances and case studies , 1976 .

[21]  Sun-Yuan Kung,et al.  A new identification and model reduction algorithm via singular value decomposition , 1978 .

[22]  S. Haykin.  Nonlinear Methods of Spectral Analysis , 1980 .

[23]  S. Haykin,et al.  Prediction-Error Filtering and Maximum-Entropy Spectral Estimation (With 16 Figures) , 1979 .

[24]  John C. Viney,et al.  The Principles of Interferometric Spectroscopy , 1979 .

[25]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[26]  Harold W. Sorenson,et al.  Parameter estimation: Principles and problems , 1980 .

[27]  Thomas Kailath,et al.  Linear Systems , 1980 .

[28]  B. Moore.  Principal component analysis in linear systems: Controllability, observability, and model reduction , 1981 .

[29]  Y. Baram.  Realization and reduction of Markovian models from nonstationary data , 1981 .

[30]  U. Desai,et al.  A realization approach to stochastic model reduction and balanced stochastic realizations , 1982, 1982 21st IEEE Conference on Decision and Control.

[31]  J. Cadzow,et al.  Spectral estimation: An overdetermined rational model equation approach , 1982, Proceedings of the IEEE.

[32]  Albert Benveniste,et al.  Identification of vibrating structures subject to non stationary excitation : A non stationary stochastic realization problem , 1982, ICASSP.

[33]  K. Arun,et al.  State-space and singular-value decomposition-based approximation methods for the harmonic retrieval problem , 1983 .

[34]  J. L. Hock,et al.  An exact recursion for the composite nearest‐neighbor degeneracy for a 2×N lattice space , 1984 .

[35]  U. Desai,et al.  A transformation approach to stochastic model reduction , 1984 .

[36]  A. Benveniste,et al.  Single sample modal identification of a nonstationary stochastic process , 1985, IEEE Transactions on Automatic Control.