High-dimensional delay selection for regression models with mutual information and distance-to-diagonal criteria

Delay selection for time series phase space reconstruction may be performed using a mutual information (MI) criterion. However, the delay selection is in that case limited to the estimation of a single delay using MI between two variables only. A high-dimensional estimator of the MI may be used to select more than one delay between more than two variables but this approach is rather time consuming. In this paper, an alternative fast criterion is proposed to optimize all delays for a high-dimensional phase space reconstruction: the distance-to-diagonal (DD) criterion, based on a geometrical heuristic. The use of the distance to diagonal criterion is illustrated and compared to MI on artificial and benchmark time series.

[1]  Amaury Lendasse,et al.  Mutual information and gamma test for input selection , 2005, ESANN.

[2]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[3]  J M Nichols,et al.  Attractor reconstruction for non-linear systems: a methodological note. , 2001, Mathematical biosciences.

[4]  Mehmet Emre Çek,et al.  Analysis of observed chaotic data , 2004 .

[5]  Fraser,et al.  Independent coordinates for strange attractors from mutual information. , 1986, Physical review. A, General physics.

[6]  Michel Verleysen,et al.  Lag selection for regression models using high-dimensional mutual information , 2006, ESANN.

[7]  H. Kantz,et al.  Nonlinear time series analysis , 1997 .

[8]  Amaury Lendasse,et al.  Mutual Information and k-Nearest Neighbors Approximator for Time Series Prediction , 2005, ICANN.

[9]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[10]  P. Grassberger,et al.  Measuring the Strangeness of Strange Attractors , 1983 .

[11]  Amaury Lendasse,et al.  Input and Structure Selection for k-NN Approximator , 2005, IWANN.

[12]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[13]  L. Cao Practical method for determining the minimum embedding dimension of a scalar time series , 1997 .

[14]  Michel Verleysen,et al.  On the Kernel Widths in Radial-Basis Function Networks , 2003, Neural Processing Letters.

[15]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[16]  H. Abarbanel,et al.  Determining embedding dimension for phase-space reconstruction using a geometrical construction. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[17]  Mark J. L. Orr Optimising the widths of radial basis functions , 1998, Proceedings 5th Brazilian Symposium on Neural Networks (Cat. No.98EX209).

[18]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[19]  F. Takens Detecting strange attractors in turbulence , 1981 .

[20]  Amaury Lendasse,et al.  Direct and Recursive Prediction of Time Series Using Mutual Information Selection , 2005, IWANN.

[21]  Alexander Kraskov,et al.  Least-dependent-component analysis based on mutual information. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.