Monotonicity of the Trace–Inverse of Covariance Submatrices and Two-Sided Prediction

It is common to assess the "memory strength" of a stationary process looking at how fast the normalized log-determinant of its covariance submatrices (i.e., entropy rate) decreases. In this work, we propose an alternative characterization in terms of the normalized trace-inverse of the covariance submatrices. We show that this sequence is monotonically non-decreasing and is constant if and only if the process is white. Furthermore, while the entropy rate is associated with one-sided prediction errors (present from past), the new measure is associated with two-sided prediction errors (present from past and future). This measure can be used as an alternative to Burg's maximum-entropy principle for spectral estimation. We also propose a counterpart for non-stationary processes, by looking at the average trace-inverse of subsets.

[1]  Bishwa Nath Mukherjee,et al.  On some properties of positive definite toeplitz matrices and their possible applications , 1988 .

[2]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[3]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[4]  Steven Kay,et al.  Some results in linear interpolation theory , 1983 .

[5]  D. L. Helsey,et al.  Linear estimation filters in spectral analysis , 1976, ICASSP.

[6]  R. Galbraith,et al.  ON THE INVERSES OF SOME PATTERNED MATRICES ARISING IN THE THEORY OF STATIONARY TIME SERIES , 1974 .

[7]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[8]  Bernard C. Picinbono,et al.  Some properties of prediction and interpolation errors , 1988, IEEE Trans. Acoust. Speech Signal Process..

[9]  Claude Brezinski,et al.  Moments of a linear operator, with applications to the trace of the inverse of matrices and the solution of equations , 2012, Numer. Linear Algebra Appl..

[10]  Alfred M. Bruckstein,et al.  Fast matrix factorizations via discrete transmission lines , 1986 .

[11]  M. Morf,et al.  Inverses of Toeplitz operators, innovations, and orthogonal polynomials , 1975, 1975 IEEE Conference on Decision and Control including the 14th Symposium on Adaptive Processes.

[12]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[13]  G. Szegö.  Ein Grenzwertsatz über die Toeplitzschen Determinanten einer reellen positiven Funktion , 1915 .

[14]  Constantine Bekas,et al.  Accelerating data uncertainty quantification by solving linear systems with multiple right-hand sides , 2013, Numerical Algorithms.

[15]  M. M. Siddiqui.  On the Inversion of the Sample Covariance Matrix in a Stationary Autoregressive Process , 1958 .

[16]  S. Brendle,et al.  Calculus of Variations , 1927, Nature.

[17]  Athanasios Papoulis,et al.  Spectral estimation from nonconsecutive data , 1987, IEEE Trans. Inf. Theory.

[18]  R. Weinstock.  Calculus of Variations: with Applications to Physics and Engineering , 1952 .

[19]  U. Grenander,et al.  Toeplitz Forms And Their Applications , 1958 .

[20]  Ram Zamir,et al.  Random subsets of structured deterministic frames have MANOVA spectra , 2017, Proceedings of the National Academy of Sciences.

[21]  Carolo Friederico Gauss.  Theoria Motus Corporum Coelestium in Sectionibus Conicis Solem Ambientium , 2014 .

[22]  Julius S. Bendat,et al.  Stationary Random Processes , 2012 .

[23]  J. Wise,et al.  THE AUTOCORRELATION FUNCTION AND THE SPECTRAL DENSITY FUNCTION , 1955 .

[24]  Gérard Meurant,et al.  Estimates of the trace of the inverse of a symmetric matrix using the modified Chebyshev algorithm , 2009, Numerical Algorithms.

[25]  D. G. Champernowne,et al.  Sampling Theory Applied to Autoregressive Sequences , 1948 .

[26]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[27]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[28]  Andrew E. Yagle,et al.  Similarities and differences between one-sided and two-sided linear prediction , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[29]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..