Small-world Hopfield neural networks with weight salience priority and memristor synapses for digit recognition

Abstract A novel systematic design of associative memory networks is addressed in this paper, by incorporating both the biological small-world effect and the recently acclaimed memristor into the conventional Hopfield neural network. More specifically, the original fully connected Hopfield network is diluted by considering the small-world effect, based on a preferential connection removal criteria, i.e., weight salience priority. The generated sparse network exhibits comparable performance in associative memory but with much less connections. Furthermore, a hardware implementation scheme of the small-world Hopfield network is proposed using the experimental threshold adaptive memristor (TEAM) synaptic-based circuits. Finally, performance of the proposed network is validated by illustrative examples of digit recognition.

[1]  Robinson E. Pino,et al.  Compact method for modeling and simulation of memristor devices: Ion conductor chalcogenide-based memristor devices , 2010, 2010 IEEE/ACM International Symposium on Nanoscale Architectures.

[2]  Olaf Sporns,et al.  Small worlds inside big brains , 2006, Proceedings of the National Academy of Sciences.

[3]  Yiran Chen,et al.  An adjustable memristor model and its application in small-world neural networks , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[4]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[5]  Danielle Smith Bassett,et al.  Small-World Brain Networks , 2006, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[6]  Robert Proulx,et al.  NDRAM: nonlinear dynamic recurrent associative memory for learning bipolar and nonbipolar correlated patterns , 2005, IEEE Transactions on Neural Networks.

[7]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[8]  J. Yang,et al.  Switching dynamics in titanium dioxide memristive devices , 2009 .

[9]  Hai Helen Li,et al.  Spintronic Memristor Through Spin-Torque-Induced Magnetization Motion , 2009, IEEE Electron Device Letters.

[10]  R. Williams,et al.  Exponential ionic drift: fast switching and low volatility of thin-film memristors , 2009 .

[11]  Björn Sommer,et al.  Membrane Packing Problems: A short Review on computational Membrane Modeling Methods and Tools , 2013, Computational and structural biotechnology journal.

[12]  Blaise Mouttet,et al.  Proposal for Memristors in Signal Processing , 2008, NanoNet.

[13]  Neil R. Taylor,et al.  Small world network strategies for studying protein structures and binding , 2013, Computational and structural biotechnology journal.

[14]  Chua.  Memristor-The Missing Circuit Element LEON 0 , 1971 .

[15]  Yiran Chen,et al.  The Circuit Realization of a Neuromorphic Computing System with Memristor-Based Synapse Design , 2012, ICONIP.

[16]  Tomer Fekete,et al.  Small-world network properties in prefrontal cortex correlate with predictors of psychopathology risk in young children: A NIRS study , 2014, NeuroImage.

[17]  F. Corinto,et al.  Mathematical models and circuit implementations of memristive systems , 2012, 2012 13th International Workshop on Cellular Nanoscale Networks and their Applications.

[18]  Uri C. Weiser,et al.  TEAM: ThrEshold Adaptive Memristor Model , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[19]  R. Williams,et al.  How We Found The Missing Memristor , 2008, IEEE Spectrum.

[20]  Dalibor Biolek,et al.  SPICE Model of Memristor with Nonlinear Dopant Drift , 2009 .

[21]  Pauline van den Driessche,et al.  Global Attractivity in Delayed Hopfield Neural Network Models , 1998, SIAM J. Appl. Math..

[22]  Shukai Duan,et al.  A Memristor-Based Chaotic System with Boundary Conditions , 2014 .

[23]  Michael J. Berry,et al.  The simplest maximum entropy model for collective behavior in a neural network , 2012, 1207.6319.

[24]  Jason Cong,et al.  mrFPGA: A novel FPGA architecture with memristor-based reconfiguration , 2011, 2011 IEEE/ACM International Symposium on Nanoscale Architectures.

[25]  F. Corinto,et al.  Memristor Model Comparison , 2013, IEEE Circuits and Systems Magazine.

[26]  Matthew D. Pickett,et al.  SPICE modeling of memristors , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[27]  A. Thomas,et al.  Memristor-based neural networks , 2013 .

[28]  Sung-Mo Kang,et al.  Resistive Computing: Memristors-Enabled Signal Multiplication , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[29]  T. Prescott,et al.  The brainstem reticular formation is a small-world, not scale-free, network , 2006, Proceedings of the Royal Society B: Biological Sciences.

[30]  Leon O. Chua,et al.  Memristor Networks , 2014, Springer International Publishing.

[31]  Chuandong Li,et al.  Memristor-based RRAM with applications , 2012, Science China Information Sciences.