Probabilistic evaluation of time series models: a comparison of several approaches.

Several methods are examined which allow to produce forecasts for time series in the form of probability assignments. The necessary concepts are presented, addressing questions such as how to assess the performance of a probabilistic forecast. A particular class of models, cluster weighted models (CWMs), is given particular attention. CWMs, originally proposed for deterministic forecasts, can be employed for probabilistic forecasting with little modification. Two examples are presented. The first involves estimating the state of (numerically simulated) dynamical systems from noise corrupted measurements, a problem also known as filtering. There is an optimal solution to this problem, called the optimal filter, to which the considered time series models are compared. (The optimal filter requires the dynamical equations to be known.) In the second example, we aim at forecasting the chaotic oscillations of an experimental bronze spring system. Both examples demonstrate that the considered time series models, and especially the CWMs, provide useful probabilistic information about the underlying dynamical relations. In particular, they provide more than just an approximation to the conditional mean.

[1]  B. Silverman Density estimation for statistics and data analysis , 1986 .

[2]  A. Jazwinski Stochastic Processes and Filtering Theory , 1970 .

[3]  A. N. Sharkovskiĭ Dynamic systems and turbulence , 1989 .

[4]  Christian Musso,et al.  Improving Regularised Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[5]  Leonard A. Smith,et al.  Scoring Probabilistic Forecasts: The Importance of Being Proper , 2007 .

[6]  L. J. Savage Elicitation of Personal Probabilities and Expectations , 1971 .

[7]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[8]  Martienssen,et al.  Controlling chaos experimentally in systems exhibiting large effective Lyapunov exponents. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[9]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[10]  J. Levine,et al.  Exact finite dimensional filters for a class of nonlinear discrete-time systems † , 1986 .

[11]  G. Brier VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[12]  G. Kallianpur Stochastic Filtering Theory , 1980 .

[13]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[14]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[15]  A. H. Murphy,et al.  Hedging and Skill Scores for Probability Forecasts , 1973 .

[16]  Tracking unstable periodic orbits in a bronze ribbon experiment. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[17]  N. Gershenfeld,et al.  Cluster-weighted modelling for time-series analysis , 1999, Nature.

[18]  U. Parlitz,et al.  Nonlinear noise reduction , 2002, Proc. IEEE.

[19]  Jochen Bröcker,et al.  Reliability, sufficiency, and the decomposition of proper scores , 2009 .

[20]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[21]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[22]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[23]  Neil Gershenfeld,et al.  Cluster-Weighted Modeling: Probabilistic Time Series Prediction, Characterization, and Synthesis , 2001 .

[24]  W. Martienssen,et al.  QUASICONTINUOUS CONTROL OF A BRONZE RIBBON EXPERIMENT USING TIME-DELAY COORDINATES , 1997 .