Predictability and Information Theory. Part I: Measures of Predictability

Abstract This paper gives an introduction to the connection between predictability and information theory, and derives new connections between these concepts. A system is said to be unpredictable if the forecast distribution, which gives the most complete description of the future state based on all available knowledge, is identical to the climatological distribution, which describes the state in the absence of time lag information. It follows that a necessary condition for predictability is for the forecast and climatological distributions to differ. Information theory provides a powerful framework for quantifying the difference between two distributions that agrees with intuition about predictability. Three information theoretic measures have been proposed in the literature: predictive information, relative entropy, and mutual information. These metrics are discussed with the aim of clarifying their similarities and differences. All three metrics have attractive properties for defining predictability, i...

[1]  R. Kleeman.  Measuring Dynamical Prediction Utility Using Relative Entropy , 2002 .

[2]  Robert M. Chervin,et al.  On Determining the Statistical Significance of Climate Experiments with General Circulation Models , 1976 .

[3]  Michael J. Rycroft,et al.  Storms in Space , 2004 .

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  A. Jazwinski.  Stochastic Processes and Filtering Theory , 1970 .

[6]  Jeffrey L. Anderson,et al.  Evaluating the Potential Predictive Utility of Ensemble Forecasts , 1996 .

[7]  Andrew M. Moore,et al.  Stochastic forcing of ENSO by the intraseasonal oscillation , 1999 .

[8]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics (Revised Edition) , 1999 .

[9]  Andrew J. Majda,et al.  A mathematical framework for quantifying predictability through relative entropy , 2002 .

[10]  E. Epstein,et al.  Stochastic dynamic prediction , 1969 .

[11]  E. Lorenz.  A study of the predictability of a 28-variable atmospheric model , 1965 .

[12]  S. Griffies,et al.  A Conceptual Framework for Predictability Studies , 1999 .

[13]  E. Lorenz.  The predictability of a flow which possesses many scales of motion , 1969 .

[14]  J. Shukla,et al.  Dynamical predictability of monthly means , 1981 .

[15]  E. Epstein,et al.  Stochastic dynamic prediction1 , 1969 .

[16]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[17]  Prashant D. Sardeshmukh,et al.  Changes of Probability Associated with El Niño , 2000 .

[18]  Fazlollah M. Reza,et al.  Introduction to Information Theory , 2004, Lecture Notes in Electrical Engineering.

[19]  G. North,et al.  Information Theory and Climate Prediction , 1990 .

[20]  Timothy DelSole,et al.  Predictable Component Analysis, Canonical Correlation Analysis, and Autoregressive Models , 2003 .

[21]  Prashant D. Sardeshmukh,et al.  The Optimal Growth of Tropical Sea Surface Temperature Anomalies , 1995 .

[22]  Andrew J. Majda,et al.  A framework for predictability through relative entropy , 2002 .

[23]  G. Uhlenbeck,et al.  On the Theory of the Brownian Motion II , 1945 .

[24]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[25]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .

[26]  Michael K. Tippett,et al.  Some theoretical considerations on predictability of linear stochastic dynamics , 2003 .

[27]  K. Hasselmann.  Optimal Fingerprints for the Detection of Time-dependent Climate Change , 1993 .

[28]  T. DelSole,et al.  Stochastic Models of Quasigeostrophic Turbulence , 2004 .

[29]  T. Barnett,et al.  Origins and Levels of Monthly and Seasonal Forecast Skill for United States Surface Air Temperatures Determined by Canonical Correlation Analysis , 1987 .

[30]  Richard A. Johnson,et al.  Applied Multivariate Statistical Analysis , 1983 .

[31]  A. H. Murphy,et al.  What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting , 1993 .

[32]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[33]  Stanford Goldman,et al.  Information theory , 1953 .