The Neuro-Fuzzy Computing System With the Capacity of Implementation on a Memristor Crossbar and Optimization-Free Hardware Training

In this paper, first we present a new explanation for the relationship between logical circuits and artificial neural networks, logical circuits and fuzzy logic, and artificial neural networks and fuzzy inference systems. This shows us that neural networks are working in the same way as logical circuits when the connection between them is through the fuzzy logic. However, the main difference between them is that logical circuits can be constructed without using any kind of optimization-based learning methods. Based on these results, we propose a new neuro-fuzzy computing system. As verified by simulation results, it can effectively be implemented on the memristor crossbar structure and therefore can be a good approach to emulating the computing power of human brain. One important feature of the designed hardware is that it has the potential to be directly trained using the Hebbian learning rule and without the need for any optimization. The system is also very capable of dealing with a large number of input-out training data without facing problems like overtraining or sensitivity to outliers.

[1]  Wei Lu,et al.  Short-term Memory to Long-term Memory Transition in a Nanoscale Memristor , 2022 .

[2]  A. Ayatollahi,et al.  Implementation of biologically plausible spiking neural network models on the memristor crossbar-based CMOS/nano circuits , 2009, 2009 European Conference on Circuit Theory and Design.

[3]  Marcian N. Cirstea,et al.  Direct Neural-Network Hardware-Implementation Algorithm , 2010, IEEE Transactions on Industrial Electronics.

[4]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[5]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[6]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[7]  L. Chua.  Memristor-The missing circuit element , 1971 .

[8]  Robert Kozma,et al.  Adaptive Resonance Theory Design in Mixed Memristive-Fuzzy Hardware , 2012 .

[9]  Ligang Gao,et al.  High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm , 2011, Nanotechnology.

[10]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[11]  Farnood Merrikh-Bayat,et al.  Bottleneck of using a single memristive device as a synapse , 2013, Neurocomputing.

[12]  Edgar Sanchez-Sinencio,et al.  CMOS transconductance multipliers: a tutorial , 1998 .

[13]  Doreen Meier,et al.  Fundamentals Of Neural Networks Architectures Algorithms And Applications , 2016 .

[14]  Wei Zhang,et al.  Design Exploration of Hybrid CMOS and Memristor Circuit by New Modified Nodal Analysis , 2012, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[15]  Farnood Merrikh-Bayat,et al.  Memristive Neuro-Fuzzy System , 2013, IEEE Transactions on Cybernetics.

[16]  Naleih M. Botros,et al.  Hardware implementation of an artificial neural network using field programmable gate arrays (FPGA's) , 1994, IEEE Trans. Ind. Electron..

[17]  B. Bose,et al.  Evaluation of membership functions for fuzzy logic controlled induction motor drive , 2002, IEEE 2002 28th Annual Conference of the Industrial Electronics Society. IECON 02.

[18]  Edgar Sanchez-Sinencio,et al.  Operational transconductance amplifier-based nonlinear function syntheses , 1989 .

[19]  Leszek Rutkowski,et al.  Flexible Neuro-fuzzy Systems: Structures, Learning and Performance Evaluation (Kluwer International Series in Engineering and Computer Science) , 2004 .

[20]  Leon O. Chua,et al.  Memristor Bridge Synapse-Based Neural Network and Its Learning , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Masayuki Murakami,et al.  A study on the modeling ability of the IDS method: A soft computing technique using pattern-based information processing , 2007, Int. J. Approx. Reason..

[22]  Jussi H. Poikonen,et al.  Time-dependency of the threshold voltage in memristive devices , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[23]  J. David Irwin,et al.  Digital Logic Circuit Analysis and Design , 1995 .

[24]  Yannis Tsividis,et al.  A reconfigurable VLSI neural network , 1992 .

[25]  Jenq-Neng Hwang,et al.  Regression modeling in back-propagation and projection pursuit learning , 1994, IEEE Trans. Neural Networks.

[26]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[27]  Leon O. Chua,et al.  Neural Synaptic Weighting With a Pulse-Based Memristor Circuit , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[28]  Yau-Hwang Kuo,et al.  A fuzzy neural network model and its hardware implementation , 1993, IEEE Trans. Fuzzy Syst..

[29]  K. D. Cantley,et al.  Hebbian Learning in Spiking Neural Networks With Nanocrystalline Silicon TFTs and Memristive Synapses , 2011, IEEE Transactions on Nanotechnology.

[30]  Laurene V. Fausett,et al.  Fundamentals Of Neural Networks , 1994 .

[31]  Farnood Merrikh-Bayat,et al.  Memristor Crossbar-Based Hardware Implementation of the IDS Method , 2010, IEEE Transactions on Fuzzy Systems.

[32]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[33]  Bernabé Linares-Barranco,et al.  On neuromorphic spiking architectures for asynchronous STDP memristive systems , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[34]  Bernabé Linares-Barranco,et al.  On Spike-Timing-Dependent-Plasticity, Memristive Devices, and Building a Self-Learning Visual Cortex , 2011, Front. Neurosci..

[35]  L.O. Chua,et al.  Memristive devices and systems , 1976, Proceedings of the IEEE.

[36]  Y. Dan,et al.  Spike-timing-dependent synaptic modification induced by natural spike trains , 2002, Nature.

[37]  James T. Kwok,et al.  Objective functions for training new hidden units in constructive neural networks , 1997, IEEE Trans. Neural Networks.

[38]  Khashayar Khorasani,et al.  Constructive feedforward neural networks using Hermite polynomial activation functions , 2005, IEEE Transactions on Neural Networks.

[39]  Ennio Mingolla,et al.  From Synapses to Circuitry: Using Memristive Memory to Explore the Electronic Brain , 2011, Computer.