A Proof of Burg’s Theorem

There are now many proofs that the maximum entropy stationary stochastic process, subject to a finite number of autocorrelation constraints, is the Gauss Markov process of appropriate order. The associated spectrum is Burg’s maximum entropy spectral density. We pose a somewhat broader entropy maximization problem, in which stationarity, for example, is not assumed, and shift the burden of proof from the previous focus on the calculus of variations and time series techniques to a string of information theoretic inequalities. This results in a simple proof.

[1]  Mats Hamrud,et al.  A remark on the correspondence between the maximum entropy method and the autoregressive model (Corresp.) , 1980, IEEE Trans. Inf. Theory.

[2]  R. Gallager Information Theory and Reliable Communication , 1968 .

[3]  D. E. Smylie,et al.  Analysis of Irregularities in the Earth's Rotation , 1973 .

[4]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[5]  Adriaan van den Bos,et al.  Alternative interpretation of maximum entropy spectral analysis (Corresp.) , 1971, IEEE Trans. Inf. Theory.

[6]  J. P. Burg,et al.  Maximum entropy spectral analysis. , 1967 .

[7]  On the finite maximum entropy extrapolation , 1984, Proceedings of the IEEE.

[8]  Michael M. Fitelson,et al.  Notes on maximum-entropy processing (Corresp.) , 1973, IEEE Trans. Inf. Theory.

[9]  R. McDonough Application of the maximum-likelihood method and the maximum-entropy method to array processing , 1979 .

[10]  S. Haykin,et al.  Prediction-Error Filtering and Maximum-Entropy Spectral Estimation (With 16 Figures) , 1979 .

[11]  D. B. Preston Spectral Analysis and Time Series , 1983 .

[12]  Hirotugu Akaike,et al.  On entropy maximization principle , 1977 .

[13]  T. Ulrich,et al.  Maximum entropy spectral analy-sis and autoregressive decomposition , 1975 .

[14]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[15]  S. Haykin Nonlinear Methods of Spectral Analysis , 1980 .

[16]  T. Ulrych,et al.  Autoregressive and Mixed Autoregressive-Moving Average Models and Spectra (With 13 Figures) , 1979 .

[17]  E. Robinson,et al.  A historical perspective of spectrum estimation , 1982, Proceedings of the IEEE.

[18]  B.S. Choi,et al.  An information-theoretic proof of Burg's maximum entropy spectrum , 1984, Proceedings of the IEEE.