Evolving nanoscale associative memories with memristors

An associative memory is an essential building block for high-level networks for cognitive or brain-like computing. In this paper we consider the problem of designing associative memories using nano-scale memristors. Until now, memristors have been exploited solely as a synapse in neural networks. Our approach is novel because it exploits the analog, time-dependent properties of memristors, resulting in more efficient and simpler designs. We have designed two complementary evolutionary frameworks for the automated discovery of circuits. The memristor-based circuits are evaluated using ngspice. Our best circuit only uses three memristors for a fully functional associative memory of two inputs. HP has demonstrated practical memristors working at 3nm × 3nm sizes in terms of area. At these densities our associative memory could easily rival even the current sub-25 nm flash memory technology.

[1]  L.O. Chua,et al.  Memristive devices and systems , 1976, Proceedings of the IEEE.

[2]  John R. Koza,et al.  Evolution by Means of Genetic Programming of Analog Circuits that Perform Digital Functions , 1999, GECCO.

[3]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[4]  Robert B. Allen,et al.  Performance of a Stochastic Learning Microchip , 1990, NIPS.

[5]  James A. M. McHugh,et al.  Synthesis of biconnected graphs , 1974 .

[6]  E. Lehtonen,et al.  Arithmetic operations within memristor-based analog memory , 2010, 2010 12th International Workshop on Cellular Nanoscale Networks and their Applications (CNNA 2010).

[7]  John R. Koza,et al.  Genetic Programming III: Darwinian Invention & Problem Solving , 1999 .

[8]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[9]  Hai Helen Li,et al.  Spintronic Memristor Through Spin-Torque-Induced Magnetization Motion , 2009, IEEE Electron Device Letters.

[10]  R. Williams,et al.  How We Found The Missing Memristor , 2008, IEEE Spectrum.

[11]  J. M. Dolsma.  Nonlinear Controller Design based on Genetic Programming. , 2007 .

[12]  S. Tam,et al.  An electrically trainable artificial neural network (ETANN) with 10240 'floating gate' synapses , 1990, International 1989 Joint Conference on Neural Networks.

[13]  Massimiliano Di Ventra,et al.  Memristive model of amoeba learning. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[15]  D. George,et al.  A hierarchical Bayesian model of invariant pattern recognition in the visual cortex , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[16]  L. Chua.  Memristor-The missing circuit element , 1971 .

[17]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[18]  C. Habel,et al.  Language , 1931, NeuroImage.

[19]  Mandar Deshpande.  FPGA Implementation & Acceleration of Building blocks for Biologically Inspired Computational Models , 2011 .

[20]  Y. Pershin,et al.  Erratum: Memristive model of amoeba learning [Phys. Rev. E 80 , 021926 (2009)] , 2010 .

[21]  György Cserey,et al.  Macromodeling of the Memristor in SPICE , 2010, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[22]  Gregory S. Snider,et al.  ‘Memristive’ switches enable ‘stateful’ logic operations via material implication , 2010, Nature.

[23]  Yoh-Han Pao,et al.  Adaptive pattern recognition and neural networks , 1989 .

[24]  Teuvo Kohonen,et al.  Associative memory. A system-theoretical approach , 1977 .

[25]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[26]  Karl Steinbuch,et al.  Learning Matrices and Their Applications , 1963, IEEE Trans. Electron. Comput..

[27]  R.E. Howard,et al.  A programmable analog neural network chip , 1988, Proceedings of the IEEE 1988 Custom Integrated Circuits Conference.

[28]  J. Bailey,et al.  Why VLSI implementations of associative VLCNs require connection multiplexing , 1988, IEEE 1988 International Conference on Neural Networks.

[29]  Min Jiang,et al.  Automated synthesis of passive analog filters using graph representation , 2010, Expert Syst. Appl..

[30]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[31]  Dan Hammerstrom,et al.  Prospects for building cortex-scale CMOL/CMOS circuits: A design space exploration , 2009, 2009 NORCHIP.

[32]  E. Talbi,et al.  A Unified Model for Evolutionary Multiobjective Optimization and its Implementation in a General Purpose Software Framework: ParadisEO-MOEO , 2009, 0904.2987.

[33]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[34]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[35]  Gérard Dreyfus,et al.  Specification and implementation of a digital Hopfield-type associative memory with on-chip training , 1992, IEEE Trans. Neural Networks.

[36]  A.N. Michel,et al.  Associative memories via artificial neural networks , 1990, IEEE Control Systems Magazine.

[37]  Donald J. Foss,et al.  Human Associative Memory: A Brief Edition. , 1981 .

[38]  M. Duranton,et al.  Learning on VLSI: a general purpose digital neurochip , 1989, International 1989 Joint Conference on Neural Networks.

[39]  Ennio Mingolla,et al.  From Synapses to Circuitry: Using Memristive Memory to Explore the Electronic Brain , 2011, Computer.

[40]  John R. Koza.  Genetic Programming III - Darwinian Invention and Problem Solving , 1999, Evolutionary Computation.

[41]  Patrick D. Roberts,et al.  Spike timing dependent synaptic plasticity in biological systems , 2002, Biological Cybernetics.

[42]  Arthur L. Samuel,et al.  Some studies in machine learning using the game of checkers , 2000, IBM J. Res. Dev..

[43]  El-Ghazali Talbi,et al.  ParadisEO: A Framework for the Reusable Design of Parallel and Distributed Metaheuristics , 2004, J. Heuristics.

[44]  John R. Koza,et al.  Automated synthesis of analog electrical circuits by means of genetic programming , 1997, IEEE Trans. Evol. Comput..

[45]  Stuart Mackie,et al.  A digital implementation of a best match classifier , 1988, Proceedings of the IEEE 1988 Custom Integrated Circuits Conference.

[46]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[47]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[48]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[49]  Richard F. Lyon,et al.  An analog electronic cochlea , 1988, IEEE Trans. Acoust. Speech Signal Process..

[50]  Kunihiko Fukushima,et al.  A model of associative memory in the brain , 1973, Kybernetik.

[51]  Wolfgang Banzhaf,et al.  Linear-Graph GP - A New GP Structure , 2002, EuroGP.

[52]  John R. Koza,et al.  Automated Design of Both the Topology and Sizing of Analog Electrical Circuits Using Genetic Programming , 1996 .

[53]  J J Hopfield,et al.  Neural computation by concentrating information in time. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[54]  John McCarthy,et al.  A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955 , 2006, AI Mag..

[55]  Richard S. Withers,et al.  Analog nonvolatile memory for neural network implementations , 1990 .

[56]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[57]  Toshiyuki Nakagaki,et al.  Amoebae anticipate periodic events. , 2008, Physical review letters.

[58]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[59]  Carver A. Mead,et al.  A novel associative memory implemented using collective computation , 1990 .

[60]  P. Vontobel,et al.  Writing to and reading from a nano-scale crossbar memory based on memristors , 2009, Nanotechnology.

[61]  Massimiliano Di Ventra,et al.  Experimental demonstration of associative memory with memristive neural networks , 2009, Neural Networks.

[62]  V. Erokhin,et al.  Electrochemically controlled polymeric device: a memristor (and more) found two years ago , 2008, 0807.0333.

[63]  Mike Butts,et al.  FPGA IMPLEMENTATION OF VERY LARGE ASSOCIATIVE MEMORIES Scaling Issues , 2003 .

[64]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[65]  Kyung-Joong Kim,et al.  Evolution of Analog Circuit Models of Ion Channels , 2010, ICES.

[66]  John R. Koza,et al.  Automated synthesis of computational circuits using genetic programming , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[67]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[68]  E. Fischer.  Conditioned Reflexes , 1942, American journal of physical medicine.

[69]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .