Correlation dimension: a pivotal statistic for non-constrained realizations of composite hypotheses in surrogate data analysis

Abstract Currently surrogate data analysis can be used to determine if data is consistent with various linear systems, or something else (a nonlinear system). In this paper we propose an extension of these methods in an attempt to make more specific classifications within the class of nonlinear systems. In the method of surrogate data one estimates the probability distribution of values of a test statistic for a set of experimental data under the assumption that the data is consistent with a given hypothesis. If the probability distribution of the test statistic is different for different dynamical systems consistent with the hypothesis, one must ensure that the surrogate generation technique generates surrogate data that are a good approximation to the data. This is often achieved with a careful choice of surrogate generation method and for noise driven linear surrogates such methods are commonly used. This paper argues that, in many cases (particularly for nonlinear hypotheses), it is easier to select a test statistic for which the probability distribution of test statistic values is the same for all systems consistent with the hypothesis. For most linear hypotheses one can use a reliable estimator of a dynamic invariant of the underlying class of processes. For more complex, nonlinear hypothesis it requires suitable restatement (or cautious statement) of the hypothesis. Using such statistics one can build nonlinear models of the data and apply the methods of surrogate data to determine if the data is consistent with a simulation from a broad class of models. These ideas are illustrated with estimates of probability distribution functions for correlation dimension estimates of experimental and artificial data, and linear and nonlinear hypotheses.