Detecting Signals from Data with Noise: Theory and Applications

AbstractSignal detection from noisy data by rejecting a noise null hypothesis depends critically on a priori assumptions regarding the background noise and the associated statistical methods. Rejecting one kind of noise null hypothesis cannot rule out the possibility that the detected oscillations are generated from the stochastic processes of another kind. This calls for an adaptive null hypothesis based on general characteristics of the noise that is present. In this paper, a new method is developed for identifying signals from data based on the finding that true physical signals in a well-sampled time series cannot be destroyed or eliminated by resampling the time series with fractional sampling rates through linear interpolation. Therefore, the significance of signals could be tested by checking whether the signals persist in the true time–frequency spectral representation during resampling. This hypothesis is based on the general characteristics of noise as revealed by empirical mode decomposition, a...

[1]  Ka Kit Tung,et al.  Solar Cycles in 150 Years of Global Sea Surface Temperature Data , 2010 .

[2]  Leonard A. Smith,et al.  Monte Carlo SSA: Detecting irregular oscillations in the Presence of Colored Noise , 1996 .

[3]  Norden E. Huang,et al.  Some Considerations on Physical Analysis of Data , 2011, Adv. Data Sci. Adapt. Anal..

[4]  W. Root,et al.  An introduction to the theory of random signals and noise , 1958 .

[5]  Norden E. Huang,et al.  On Instantaneous Frequency , 2009, Adv. Data Sci. Adapt. Anal..

[6]  Mike Lockwood,et al.  SOLAR INFLUENCES ON CLIMATE , 2009 .

[7]  Katrine K Andersen,et al.  The DO-climate events are probably noise induced: statistical investigation of the claimed 1470 years cycle , 2006 .

[8]  Norden E. Huang,et al.  Ensemble Empirical Mode Decomposition: a Noise-Assisted Data Analysis Method , 2009, Adv. Data Sci. Adapt. Anal..

[9]  Milan Paluš,et al.  Detecting Oscillations Hidden in Noise: Common Cycles in Atmospheric, Geomagnetic and Solar Data , 2008 .

[10]  C. Tucker,et al.  The modulated annual cycle: an alternative reference frame for climate anomalies , 2007 .

[11]  N. Huang,et al.  A study of the characteristics of white noise using the empirical mode decomposition method , 2004, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[12]  Jürgen Kurths,et al.  Limitations of red noise in analysing Dansgaard-Oeschger events , 2009 .

[13]  S. Feldstein.  The Timescale, Power Spectra, and Climate Noise Properties of Teleconnection Patterns , 2000 .

[14]  Jonathan M. Lees,et al.  Robust estimation of background noise and signal detection in climatic time series , 1996 .

[15]  Alejandro J. Souza,et al.  Tidal mixing modulation of sea-surface temperature and diatom abundance in Southern California , 2001 .

[16]  Gerald A. Meehl,et al.  The response in the Pacific to the sun's decadal peaks and contrasts to cold events in the Southern Oscillation , 2008 .

[17]  O. Eisen,et al.  Ground‐based measurements of spatial and temporal variability of snow accumulation in East Antarctica , 2008 .

[18]  Gabriel Rilling,et al.  Empirical mode decomposition as a filter bank , 2004, IEEE Signal Processing Letters.

[19]  Smith,et al.  Satellite measurements of sea surface temperature through clouds , 2000, Science.

[20]  Norden E. Huang,et al.  A review on Hilbert‐Huang transform: Method and its applications to geophysical studies , 2008 .

[21]  Norden E. Huang,et al.  The Time-Dependent Intrinsic Correlation Based on the Empirical Mode Decomposition , 2010, Adv. Data Sci. Adapt. Anal..

[22]  Jiansong Zhou,et al.  Constraining model transient climate response using independent observations of solar‐cycle forcing and response , 2008 .

[23]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[24]  J. Beer,et al.  The role of the sun in climate forcing , 2000 .

[25]  S. Childress,et al.  Topics in geophysical fluid dynamics. Atmospheric dynamics, dynamo theory, and climate dynamics. , 1987 .

[26]  S. Vaughan.  A simple test for periodic signals in red noise , 2005 .

[27]  Judith L. Lean,et al.  Variations in the Sun's radiative output , 1991 .

[28]  Judith L. Lean,et al.  Solar ultraviolet irradiance variations: A review , 1987 .

[29]  B. Mandelbrot,et al.  Fractional Brownian Motions, Fractional Noises and Applications , 1968 .

[30]  Norden E. Huang,et al.  On Hilbert Spectral Representation: a True Time-Frequency Representation for nonlinear and nonstationary Data , 2011, Adv. Data Sci. Adapt. Anal..

[31]  K. Coughlin,et al.  Eleven-year solar cycle signal throughout the lower atmosphere , 2004 .

[32]  Michael Schulz,et al.  REDFIT: estimating red-noise spectra directly from unevenly spaced paleoclimatic time series , 2002 .

[33]  Hans von Storch,et al.  Taking Serial Correlation into Account in Tests of the Mean. , 1995 .

[34]  Arnold L. Gordon,et al.  Tidal Mixing Signatures in the Indonesian Seas , 1996 .

[35]  David B. Stephenson,et al.  Is the North Atlantic Oscillation a random walk , 2000 .

[36]  K. Hasselmann.  Stochastic climate models Part I. Theory , 1976 .