On adaptive covariance and spectrum estimation of locally stationary multivariate processes

Abstract When estimating the correlation/spectral structure of a locally stationary process, one has to make two important decisions. First, one should choose the so-called estimation bandwidth, inversely proportional to the effective width of the local analysis window, in the way that complies with the degree of signal nonstationarity. Too small bandwidth may result in an excessive estimation bias, while too large bandwidth may cause excessive estimation variance. Second, but equally important, one should choose the appropriate order of the spectral representation of the signal so as to correctly model its resonant structure–when the order is too small, the estimated spectrum may not reveal some important signal components (resonances), and when it is too high, it may indicate the presence of some nonexistent components. When the analyzed signal is not stationary, with a possibly time-varying degree of nonstationarity, both the bandwidth and order parameters should be adjusted in an adaptive fashion. The paper presents and compares three approaches allowing for unified treatment of the problem of adaptive bandwidth and order selection for the purpose of identification of nonstationary vector autoregressive processes: the cross-validation approach, the full cross-validation approach, and the approach that incorporates the multivariate version of the generalized Akaike’s final prediction error criterion. It is shown that the latter solution yields the best results and, at the same time, is very attractive from the computational viewpoint.

[1]  H. Akaike Autoregressive model fitting for control , 1971 .

[2]  R. Dahlhaus,et al.  On the Optimal Segment Length for Parameter Estimates for Locally Stationary Time Series , 1998 .

[3]  Maciej Niedwiecki,et al.  Brief paper: Easy recipes for cooperative smoothing , 2010 .

[4]  R. Dahlhaus Locally Stationary Processes , 2011, 1109.4174.

[5]  Maciej Niedzwiecki,et al.  Identification of Time-Varying Processes , 2000 .

[6]  R. Dahlhaus Local inference for locally stationary time series based on the empirical spectral measure , 2009 .

[7]  Robert L. Taylor Stochastic Convergence of Weighted Sums of Random Elements in Linear Spaces , 1978 .

[8]  Vassilis S. Kouikoglou,et al.  An exact discrete-event model and control policies for production lines with buffers , 1991 .

[9]  Maciej Niedzwiecki,et al.  On noncausal weighted least squares identification of nonstationary stochastic systems , 2011, Autom..

[10]  Maciej Niedzwiecki,et al.  Bayesian-like autoregressive spectrum estimation in the case of unknown process order , 1984, The 23rd IEEE Conference on Decision and Control.

[11]  Michele Pavon,et al.  Time and Spectral Domain Relative Entropy: A New Approach to Multivariate Spectral Estimation , 2011, IEEE Transactions on Automatic Control.

[12]  Maciej Niedzwiecki,et al.  Nonasymptotic results for finite-memory WLS filters , 1991 .

[13]  Maciej Niedzwiecki,et al.  On the localized estimators and generalized Akaike's criterions , 1981, CDC 1981.

[14]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[15]  A. Goldenshluger On Spatial Adaptive Estimation of Nonparametric Regression , 2004 .

[16]  J. P. Burg,et al.  Maximum entropy spectral analysis. , 1967 .

[17]  R. Dahlhaus Fitting time series models to nonstationary processes , 1997 .

[18]  Vladimir Katkovnik,et al.  A new method for varying adaptive bandwidth selection , 1999, IEEE Trans. Signal Process..

[19]  Petre Stoica,et al.  Introduction to spectral analysis , 1997 .

[20]  Jörg Polzehl,et al.  Model Selection, Transformations and Variance Estimation in Nonlinear Regression , 1999 .

[21]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[22]  Maciej Niedzwiecki,et al.  Statistical Reconstruction of Multivariate Time Series , 1993, IEEE Trans. Signal Process..

[23]  Zening Fu,et al.  Adaptive Covariance Estimation of Non-Stationary Processes and its Application to Infer Dynamic Connectivity From fMRI , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[24]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[25]  H. Akaike A new look at the statistical model identification , 1974 .

[26]  Maciej Niedzwiecki,et al.  New Approach to Noncausal Identification of Nonstationary Stochastic FIR Systems Subject to Both Smooth and Abrupt Parameter Changes , 2012, IEEE Transactions on Automatic Control.

[27]  Alina A. von Davier,et al.  Cross-Validation , 2014 .

[28]  Ljubisa Stankovic,et al.  Performance analysis of the adaptive algorithm for bias-to-variance tradeoff , 2004, IEEE Transactions on Signal Processing.

[29]  V. A. Epanechnikov Non-Parametric Estimation of a Multivariate Probability Density , 1969 .

[30]  Maciej Niedzwiecki,et al.  Locally Adaptive Cooperative Kalman Smoothing and Its Application to Identification of Nonstationary Stochastic Systems , 2012, IEEE Transactions on Signal Processing.

[31]  M. Priestley Evolutionary Spectra and Non‐Stationary Processes , 1965 .