Stochastic resonance in Hopfield neural networks for transmitting binary signals

Abstract We investigate the stochastic resonance phenomenon in a discrete Hopfield neural network for transmitting binary amplitude modulated signals, wherein the binary information is represented by two stored patterns. Based on the potential energy function and the input binary signal amplitude, the observed stochastic resonance phenomena involve two general noise-improvement mechanisms. A suitable amount of added noise assists or accelerates the switch of the network state vectors to follow input binary signals more correctly, yielding a lower probability of error. Moreover, at a given added noise level, the probability of error can be further reduced by the increase of the number of neurons. When the binary signals are corrupted by external heavy-tailed noise, it is found that the Hopfield neural network with a large number of neurons can outperform the matched filter in the region of low input signal-to-noise ratios per bit.

[1]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[2]  Haruhiko Nishimura,et al.  Stochastic Resonance in Recurrent Neural Network with Hopfield-Type Memory , 2009, Neural Processing Letters.

[3]  Vladimir Pavlovic,et al.  Stochastic noise Process enhancement of Hopfield neural networks , 2005, IEEE Transactions on Circuits and Systems II: Express Briefs.

[4]  Joaquín J. Torres,et al.  Stochastic Resonance Crossovers in Complex Networks , 2012, PloS one.

[5]  Sancho Salcedo-Sanz,et al.  A hybrid Hopfield network-simulated annealing approach for frequency assignment in satellite communications systems , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Matjaz Perc,et al.  Stochastic resonance on weakly paced scale-free networks. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[7]  Jehoshua Bruck.  On the convergence properties of the Hopfield model , 1990, Proc. IEEE.

[8]  Nikolay V. Kuznetsov,et al.  Hidden chaotic sets in a Hopfield neural system , 2017 .

[9]  Shun-ichi Amari,et al.  Mathematical foundations of neurocomputing , 1990, Proc. IEEE.

[10]  Riani,et al.  Stochastic resonance in the perceptual interpretation of ambiguous figures: A neural network model. , 1994, Physical review letters.

[11]  Masahiko Morita,et al.  Associative memory with nonmonotone dynamics , 1993, Neural Networks.

[12]  Dominique M. Durand,et al.  Stochastic Resonance Can Enhance Information Transmission in Neural Networks , 2011, IEEE Transactions on Biomedical Engineering.

[13]  Kazuyuki Aihara,et al.  Deterministic SR phenomena in autoassociative chaotic neural networks , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[14]  S. Kassam.  Signal Detection in Non-Gaussian Noise , 1987 .

[15]  Yangsong Zhang,et al.  Frequency-difference-dependent stochastic resonance in neural systems. , 2017, Physical review. E.

[16]  A. R. Bizzarri.  Convergence properties of a modified Hopfield-Tank model , 2004, Biological Cybernetics.

[17]  S. Fauve,et al.  Stochastic resonance in a bistable system , 1983 .

[18]  Masaki Kobayashi,et al.  Multistate vector product hopfield neural networks , 2018, Neurocomputing.

[19]  Leslie S. Smith,et al.  Precision constrained stochastic resonance in a feedforward neural network , 2005, IEEE Transactions on Neural Networks.

[20]  Elias S. Manolakos,et al.  Hopfield neural network implementation of the optimal CDMA multiuser detector , 1996, IEEE Trans. Neural Networks.

[21]  François Chapeau-Blondeau,et al.  Residual aperiodic stochastic resonance in a bistable dynamic system transmitting a suprathreshold binary signal. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  Yasuo Ogawa,et al.  Application of a modified hopfield neural network to noisy magnetotelluric data , 2007 .

[23]  Juan Humberto Sossa Azuela,et al.  Generating exponentially stable states for a Hopfield Neural Network , 2018, Neurocomputing.

[24]  Mahesan Niranjan,et al.  A theoretical investigation into the performance of the Hopfield model , 1990, IEEE Trans. Neural Networks.

[25]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[26]  Dan Schonfeld,et al.  On the hysteresis and robustness of Hopfield neural networks , 1993 .

[27]  Bing Hu,et al.  Optimum adaptive array stochastic resonance in noisy grayscale image restoration , 2019, Physics Letters A.

[28]  Roy,et al.  Observation of stochastic resonance in a ring laser. , 1988, Physical review letters.

[29]  Amin Karbasi,et al.  Noise-Enhanced Associative Memories , 2013, NIPS.

[30]  N. Stocks,et al.  Suprathreshold stochastic resonance in multilevel threshold systems , 2000, Physical review letters.

[31]  Masaki Kobayashi,et al.  Quaternionic Hopfield neural networks with twin-multistate activation function , 2017, Neurocomputing.

[32]  M. Ozer,et al.  Stochastic resonance on Newman–Watts networks of Hodgkin–Huxley neurons with local periodic driving , 2009 .

[33]  Fabio Marchesoni,et al.  PHASE-SHIFTS IN BISTABLE EPR SYSTEMS AT STOCHASTIC RESONANCE , 1991 .

[34]  Dominique M. Durand,et al.  Enhancement of information transmission of sub-threshold signals applied to distal positions of dendritic trees in hippocampal CA1 neuron models with stochastic resonance , 2010, Biological Cybernetics.

[35]  Gregoire Nicolis,et al.  Stochastic resonance , 2007, Scholarpedia.

[36]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[37]  Peiguo Liu,et al.  State observer for stochastic resonance in bistable system , 2019, Physics Letters A.

[38]  R. Benzi.  Stochastic resonance in complex systems , 2009 .

[39]  N. D. Stein,et al.  Stochastic resonance , 1993, Scholarpedia.

[40]  François Chapeau-Blondeau,et al.  Theory of stochastic resonance in signal transmission by static nonlinear systems , 1997 .

[41]  Derek Abbott,et al.  Noise Benefits in Combined Nonlinear Bayesian Estimators , 2019, IEEE Transactions on Signal Processing.

[42]  Yan-Mei Kang,et al.  Stochastic resonance in underdamped periodic potential systems with alpha stable Lévy noise , 2018, Physics Letters A.

[43]  Carson C. Chow,et al.  Aperiodic stochastic resonance. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[44]  A. Sutera,et al.  The mechanism of stochastic resonance , 1981 .

[45]  Ebrahim A. Soujeri,et al.  Hopfield multiuser detection of asynchronous MC-CDMA signals in multipath fading channels , 2002, IEEE Communications Letters.

[46]  Derek Abbott,et al.  Stochastic resonance in a parallel array of nonlinear dynamical elements , 2008 .

[47]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.