Data processing inequality and stochastic resonance

The data processing inequality of information theory states that given random variables X, Y and Z which form a Markov chain in the order X-->Y-->Z, then the mutual information between X and Y is greater than or equal to the mutual information between X and Z. That is I(X) >= I(X;Z) . In practice, this means that no more information can be obtained out of a set of data then was there to begin with, or in other words, there is a bound on how much can be accomplished with signal processing. However, in the field of stochastic resonance, it has been reported that a signal to noise ratio gain can occur in some nonlinear systems due to the addition of noise. Such an observation appears to contradict the data processing inequality. In this paper, we investigate this question by using an example model system.

[1]  Zoltan Gingl,et al.  Signal-to-noise ratio gain by stochastic resonance in a bistable system , 2000 .

[2]  François Chapeau-Blondeau,et al.  Noise-enhanced capacity via stochastic resonance in an asymmetric binary channel , 1997 .

[3]  Vincenzo Galdi,et al.  Evaluation of stochastic-resonance-based detectors of weak harmonic signals in additive white Gaussian noise , 1998 .

[4]  Bulsara,et al.  Noise-Controlled Resonance Behavior in Nonlinear Dynamical Systems with Broken Symmetry. , 1996, Physical review letters.

[5]  J. R. Cogdell.  Foundations of electrical engineering , 1990 .

[6]  Bulsara,et al.  Nonlinear stochastic resonance: the saga of anomalous output-input gain , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[7]  Xi-Cheng Zhang,et al.  T-Ray Sensing and Imaging , 2003 .

[8]  Adi R. Bulsara,et al.  DC SIGNAL DETECTION VIA DYNAMICAL ASYMMETRY IN A NONLINEAR DEVICE , 1998 .

[9]  Derek Abbott,et al.  Analysis of system trade-offs for terahertz imaging , 2000 .

[10]  Péter Makra,et al.  HIGH SIGNAL-TO-NOISE RATIO GAIN BY STOCHASTIC RESONANCE IN A DOUBLE WELL , 2001 .

[11]  François Chapeau-Blondeau,et al.  Input-output gains for signal in noise in stochastic resonance , 1997 .

[12]  William Bialek,et al.  Information flow in sensory neurons , 1995 .

[13]  François Chapeau-Blondeau,et al.  PERIODIC AND APERIODIC STOCHASTIC RESONANCE WITH OUTPUT SIGNAL-TO-NOISE RATIO EXCEEDING THAT AT THE INPUT , 1999 .

[14]  F Liu,et al.  Signal-to-noise ratio gain in neuronal systems. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  L. Kish,et al.  SIGNAL-TO-NOISE RATIO GAIN IN NON-DYNAMICAL AND DYNAMICAL BISTABLE STOCHASTIC RESONATORS , 2002, The Random and Fluctuating World.

[16]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[17]  François Chapeau-Blondeau,et al.  Theory of stochastic resonance in signal transmission by static nonlinear systems , 1997 .

[18]  Zoltan Gingl,et al.  A stochastic resonator is able to greatly improve signal-to- noise ratio , 1996 .

[19]  Péter Makra,et al.  HIGH SIGNAL-TO-NOISE RATIO GAIN BY STOCHASTIC RESONANCE IN A DOUBLE WELL , 2001 .

[20]  Bulsara,et al.  Nonlinear dynamic elements with noisy sinusoidal forcing: Enhancing response via nonlinear coupling. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[21]  Peter Hänggi,et al.  Gain in stochastic resonance: precise numerics versus linear response theory beyond the two-mode approximation. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  Carson C. Chow,et al.  Aperiodic stochastic resonance in excitable systems. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[23]  Adi R. Bulsara,et al.  Information-Theoretic Distance Measures and a Generalization of Stochastic Resonance , 1998 .

[24]  J. Rung,et al.  General measures for signal-noise separation in nonlinear dynamical systems. , 2000, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  Jung,et al.  Stochastic resonance in optical bistable systems. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[26]  L. B. Kiss,et al.  Possible breakthrough: significant improvement of signal to noise ratio by stochastic resonance , 1996 .

[27]  Vilar,et al.  Divergent Signal-to-Noise Ratio and Stochastic Resonance in Monostable Systems. , 1996, Physical review letters.

[28]  Peter Hänggi,et al.  Stochastic resonance in optical bistable systems. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[29]  Riccardo Mannella,et al.  Stochastic resonance in perspective , 1995 .