Exploring $Bona~Fide$ Optimal Noise for Bayesian Parameter Estimation

In this paper, we investigate the benefit of intentionally added noise to observed data in various scenarios of Bayesian parameter estimation. For optimal estimators, we theoretically demonstrate that the Bayesian Cramér-Rao bound for the case with added noise is never smaller than for the original data, and the updated minimum mean-square error (MSE) estimator performs no better. This motivates us to explore the feasibility of noise benefit in some useful suboptimal estimators. Several Bayesian estimators established from one-bit-quantizer sensors are considered, and for different types of pre-existing background noise, optimal distributions are determined for the added noise in order to improve the performance in estimation. With a single sensor, it is shown that the optimal added noise for reducing the MSE is actually a constant bias. However, with parallel arrays of such sensors, bona fide optimal added noise, no longer a constant bias, is shown to reduce the MSE. Moreover, it is found that the designed Bayesian estimators can benefit from the optimal added noise to effectively approach the performance of the minimum MSE estimator, even when the assembled sensors possess different quantization thresholds.

[1]  Olivier White,et al.  The Promise of Stochastic Resonance in Falls Prevention , 2019, Front. Physiol..

[2]  Johann F. Böhme,et al.  A pseudo-noise approach to direction finding , 1998, Signal Process..

[3]  Lawrence G. Roberts,et al.  Picture coding using pseudo-random noise , 1962, IRE Trans. Inf. Theory.

[4]  L. M. Ward,et al.  Stochastic resonance and sensory information processing: a tutorial and review of application , 2004, Clinical Neurophysiology.

[5]  Ashok Patel,et al.  Optimal Noise Benefits in Neyman–Pearson and Inequality-Constrained Statistical Signal Detection , 2009, IEEE Transactions on Signal Processing.

[6]  Pramod K. Varshney,et al.  Theory of the Stochastic Resonance Effect in Signal Detection: Part I—Fixed Detectors , 2007, IEEE Transactions on Signal Processing.

[7]  Qiqing Zhai,et al.  Noise effect on signal quantization in an array of binary quantizers , 2018, Signal Process..

[8]  S. Kay Fundamentals of statistical signal processing: estimation theory , 1993 .

[9]  H. Vincent Poor,et al.  Noise Enhanced Hypothesis-Testing in the Restricted Bayesian Framework , 2010, IEEE Transactions on Signal Processing.

[10]  Haralabos C. Papadopoulos,et al.  Sequential signal encoding from noisy measurements using quantizers with dynamic bias control , 2001, IEEE Trans. Inf. Theory.

[11]  Michael Elad,et al.  MMSE Approximation For Sparse Coding Algorithms Using Stochastic Resonance , 2018, IEEE Transactions on Signal Processing.

[12]  S. Zozor,et al.  Noise-aided processing : Revisiting dithering in a Σ ∆ quantizer , 2004 .

[13]  Derek Abbott,et al.  What Is Stochastic Resonance? Definitions, Misconceptions, Debates, and Its Relevance to Biology , 2009, PLoS Comput. Biol..

[14]  Pramod K. Varshney,et al.  Theory of the Stochastic Resonance Effect in Signal Detection—Part II: Variable Detectors , 2007, IEEE Transactions on Signal Processing.

[15]  Derek Abbott,et al.  Noise Benefits in Combined Nonlinear Bayesian Estimators , 2019, IEEE Transactions on Signal Processing.

[16]  Xiaohong Shen,et al.  Effects of Second-Order Matched Stochastic Resonance for Weak Signal Detection , 2018, IEEE Access.

[17]  P. Landa Mechanism of stochastic resonance , 2004 .

[18]  L. M. Ward,et al.  Physics of neural synchronisation mediated by stochastic resonance , 2009 .

[19]  Carson C. Chow,et al.  Stochastic resonance without tuning , 1995, Nature.

[20]  Sinan Gezici,et al.  Noise benefits in joint detection and estimation problems , 2016, Signal Process..

[21]  L. Schuchman Dither Signals and Their Effect on Quantization Noise , 1964 .

[22]  Pramod K. Varshney,et al.  Noise Enhanced Parameter Estimation , 2008, IEEE Transactions on Signal Processing.

[23]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[24]  Azadeh Vosoughi,et al.  On Bayesian Fisher Information Maximization for Distributed Vector Estimation , 2017, IEEE Transactions on Signal and Information Processing over Networks.

[25]  Derek Abbott,et al.  A review of stochastic resonance: circuits and measurement , 2002, IEEE Trans. Instrum. Meas..

[26]  L. Gammaitoni,et al.  Stochastic resonance and the dithering effect in threshold physical systems. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[27]  Pramod K. Varshney,et al.  Performance Limit for Distributed Estimation Systems With Identical One-Bit Quantizers , 2010, IEEE Transactions on Signal Processing.

[28]  François Chapeau-Blondeau,et al.  Theory of stochastic resonance in signal transmission by static nonlinear systems , 1997 .

[29]  Yaguo Lei,et al.  Applications of stochastic resonance to machinery fault detection: A review and tutorial , 2019, Mechanical Systems and Signal Processing.

[30]  Sinan Gezici,et al.  Optimal Signal Design for Multi-Parameter Estimation Problems , 2015, IEEE Transactions on Signal Processing.

[31]  François Chapeau-Blondeau Noise-aided nonlinear Bayesian estimation. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[32]  C. Pearce,et al.  Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization , 2008 .

[33]  Derek Abbott,et al.  Noise Enhancement in Robust Estimation of Location , 2018, IEEE Transactions on Signal Processing.

[34]  David Rousseau,et al.  Raising the noise to improve performance in optimal processing , 2008 .

[35]  N. Stocks,et al.  Suprathreshold stochastic resonance in multilevel threshold systems , 2000, Physical review letters.

[36]  Rick S. Blum,et al.  Parameter Estimation From Quantized Observations in Multiplicative Noise Environments , 2015, IEEE Transactions on Signal Processing.

[37]  Ashok Patel,et al.  Optimal Mean-Square Noise Benefits in Quantizer-Array Linear Estimation , 2010, IEEE Signal Processing Letters.

[38]  B. Andò,et al.  Adding noise to improve measurement , 2001 .

[39]  François Chapeau-Blondeau,et al.  Noise-enhanced performance for an optimal Bayesian estimator , 2004, IEEE Transactions on Signal Processing.

[40]  Gregoire Nicolis,et al.  Stochastic resonance , 2007, Scholarpedia.

[41]  Shujun Liu,et al.  Noise benefits parameter estimation in LMMSE sense , 2018, Digit. Signal Process..

[42]  Frank Moss,et al.  Stochastic resonance: noise-enhanced order , 1999 .

[43]  Stefan Uhlich Bayes Risk Reduction of Estimators Using Artificial Observation Noise , 2015, IEEE Transactions on Signal Processing.

[44]  Sinan Gezici,et al.  CRLB Based Optimal Noise Enhanced Parameter Estimation Using Quantized Observations , 2010, IEEE Signal Processing Letters.

[45]  Carson C. Chow,et al.  Aperiodic stochastic resonance in excitable systems. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[46]  S. Kay,et al.  Can detectability be improved by adding noise? , 2000, IEEE Signal Processing Letters.

[47]  Siliang Lu,et al.  A review of stochastic resonance in rotating machine fault detection , 2019, Mechanical Systems and Signal Processing.

[48]  S. Engelberg An introduction to dither , 2012, IEEE Instrumentation & Measurement Magazine.

[49]  Pramod K. Varshney,et al.  Noise-Enhanced Information Systems , 2014, Proceedings of the IEEE.

[50]  Yue Meng,et al.  Classification of Hyperspectral Image by CNN Based on Shadow Area Enhancement Through Dynamic Stochastic Resonance , 2019, IEEE Access.

[51]  Pierre-Olivier Amblard,et al.  Stochastic resonance in locally optimal detectors , 2003, IEEE Trans. Signal Process..

[52]  G.B. Giannakis,et al.  Distributed compression-estimation using wireless sensor networks , 2006, IEEE Signal Processing Magazine.

[53]  François Chapeau-Blondeau,et al.  Noise-Improved Bayesian Estimation With Arrays of One-Bit Quantizers , 2007, IEEE Transactions on Instrumentation and Measurement.