Toward bio-inspired information processing with networks of nano-scale switching elements

Unconventional computing explores multi-scale platforms connecting molecular-scale devices into networks for the development of scalable neuromorphic architectures, often based on new materials and components with new functionalities. We review some work investigating the functionalities of locally connected networks of different types of switching elements as computational substrates. In particular, we discuss reservoir computing with networks of nonlinear nanoscale components. In usual neuromorphic paradigms, the network synaptic weights are adjusted as a result of a training/learning process. In reservoir computing, the non-linear network acts as a dynamical system mixing and spreading the input signals over a large state space, and only a readout layer is trained. We illustrate the most important concepts with a few examples, featuring memristor networks with time-dependent and history dependent resistances.

[1]  Kazuo Iwama,et al.  CONNECTIVITY , 1996, Graph Theory and Its Applications.

[2]  Z. Konkoli,et al.  A generic simulator for large networks of memristive elements. , 2013, Nanotechnology.

[3]  Herbert Jaeger,et al.  Echo State Property Linked to an Input: Exploring a Fundamental Characteristic of Recurrent Neural Networks , 2013, Neural Computation.

[4]  Fabien Alibart,et al.  Pavlov's Dog Associative Learning Demonstrated on Synaptic-Like Organic Transistors , 2013, Neural Computation.

[5]  A. Thomas,et al.  Memristor-based neural networks , 2013 .

[6]  Jian-Xin Xu,et al.  Effects of synaptic connectivity on liquid state machine performance , 2013, Neural Networks.

[7]  Johannes Schemmel,et al.  Six Networks on a Universal Neuromorphic Computing Substrate , 2012, Front. Neurosci..

[8]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[9]  Stefan J. Kiebel,et al.  Re-visiting the echo state property , 2012, Neural Networks.

[10]  Valeriu Beiu,et al.  Aspects of computing with locally connected networks , 2012 .

[11]  Damien Querlioz,et al.  Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity , 2012, Neural Networks.

[12]  Benjamin Schrauwen,et al.  Information Processing Capacity of Dynamical Systems , 2012, Scientific Reports.

[13]  Christof Teuscher,et al.  Memristor-based reservoir computing , 2012, 2012 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[14]  William R. Dichtel,et al.  High hopes: can molecular electronics realise its potential? , 2012, Chemical Society reviews.

[15]  Mauricio Barahona,et al.  Device Properties of Bernoulli Memristors , 2012, Proceedings of the IEEE.

[16]  Benjamin Schrauwen,et al.  Reservoir Computing Trends , 2012, KI - Künstliche Intelligenz.

[17]  Jakub S. Prauzner-Bechcicki,et al.  Atomic‐ and molecular‐scale devices and systems for single‐molecule electronics , 2012 .

[18]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[19]  Johannes Schemmel,et al.  Is a 4-Bit Synaptic Weight Resolution Enough? – Constraints on Enabling Spike-Timing Dependent Plasticity in Neuromorphic Hardware , 2012, Front. Neurosci..

[20]  Christian Joachim,et al.  The Different Designs of Molecule Logic Gates , 2012, Advanced materials.

[21]  Audrius V. Avizienis,et al.  Emergent Criticality in Complex Turing B‐Type Atomic Switch Networks , 2012, Advanced materials.

[22]  Y. V. Pershin,et al.  Biologically-Inspired Electronics with Memory Circuit Elements , 2011, 1112.4987.

[23]  Benjamin Schrauwen,et al.  Optoelectronic Reservoir Computing , 2011, Scientific Reports.

[24]  Massimiliano Di Ventra,et al.  Neuromorphic, Digital, and Quantum Computation With Memory Circuit Elements , 2010, Proceedings of the IEEE.

[25]  V. Beiu,et al.  SYMONE Project: Synaptic Molecular Networks for Bio-Inspired Information Processing , 2012, Int. J. Unconv. Comput..

[26]  Fabien Alibart,et al.  A Memristive Nanoparticle/Organic Hybrid Synapstor for Neuroinspired Computing , 2011, ArXiv.

[27]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[28]  Wei Lu,et al.  Short-term Memory to Long-term Memory Transition in a Nanoscale Memristor , 2022 .

[29]  M. Mayor,et al.  Resonant photoconductance of molecular junctions formed in gold nanoparticle arrays. , 2011, Journal of the American Chemical Society.

[30]  Stephen Berard,et al.  Implications of Historical Trends in the Electrical Efficiency of Computing , 2011, IEEE Annals of the History of Computing.

[31]  T. Hasegawa,et al.  Chemical wiring and soldering toward all-molecule electronic circuitry. , 2011, Journal of the American Chemical Society.

[32]  R. Ahuja,et al.  Interference effects in phtalocyanine controlled by H-H tautomerization: Potential two-terminal unimolecular electronic switch , 2011, 1104.1441.

[33]  M. Sahimi,et al.  Electric currents in networks of interconnected memristors. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[34]  Massimiliano Di Ventra,et al.  Solving mazes with memristors: a massively-parallel approach , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[35]  C. Joachim,et al.  Manipulating molecular quantum states with classical metal atom inputs: demonstration of a single molecule NOR logic gate. , 2011, ACS nano.

[36]  Johannes Schemmel,et al.  A comprehensive workflow for general-purpose neural modeling with highly configurable neuromorphic hardware systems , 2010, Biological Cybernetics.

[37]  Göran Wendin,et al.  Robustness of logic gates and reconfigurability of neuromorphic switching networks , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[38]  Benjamin Schrauwen,et al.  Connectivity, Dynamics, and Memory in Reservoir Computing with Binary and Analog Neurons , 2010, Neural Computation.

[39]  Gregory S. Snider,et al.  ‘Memristive’ switches enable ‘stateful’ logic operations via material implication , 2010, Nature.

[40]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[41]  C. Schönenberger,et al.  Cyclic conductance switching in networks of redox-active molecular junctions. , 2010, Nano letters.

[42]  C. Gamrat,et al.  An Organic Nanoparticle Transistor Behaving as a Biological Spiking Synapse , 2009, 0907.2540.

[43]  Massimiliano Di Ventra,et al.  Experimental demonstration of associative memory with memristive neural networks , 2009, Neural Networks.

[44]  D. Stewart,et al.  The missing memristor found , 2009, Nature.

[45]  Edward T. Bullmore,et al.  Broadband Criticality of Human Brain Network Synchronization , 2009, PLoS Comput. Biol..

[46]  Konrad Szaciłowski,et al.  Digital Information Processing in Molecular Systems , 2008 .

[47]  K. Szaciłowski Digital information processing in molecular systems. , 2008, Chemical reviews.

[48]  J. Yang,et al.  Memristive switching mechanism for metal/oxide/metal nanodevices. , 2008, Nature nanotechnology.

[49]  Ilya Shmulevich,et al.  Critical networks exhibit maximal information diversity in structure-dynamics relationships. , 2008, Physical review letters.

[50]  G. Wendin,et al.  Reconfigurable logic in nanoelectronic switching networks , 2007 .

[51]  J. M. Herrmann,et al.  Dynamical synapses causing self-organized criticality in neural networks , 2007, 0712.1003.

[52]  Carl Önnheim,et al.  Nanocell Devices and Architecture for Configurable Computing With Molecular Electronics , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[53]  R. Waser,et al.  Nanoionics-based resistive switching memories. , 2007, Nature materials.

[54]  Tadashi Yamazaki,et al.  The cerebellum as a liquid state machine , 2007, Neural Networks.

[55]  Bonnie A. Sheriff,et al.  A 160-kilobit molecular electronic memory patterned at 1011 bits per square centimetre , 2007, Nature.

[56]  Robert A. Legenstein,et al.  At the Edge of Chaos: Real-time Computations and Self-Organized Criticality in Recurrent Neural Networks , 2004, NIPS.

[57]  Nils Bertschinger,et al.  Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks , 2004, Neural Computation.

[58]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[59]  R. Stanley Williams,et al.  Molecule-Independent Electrical Switching in Pt/Organic Monolayer/Ti Devices , 2004 .

[60]  Chrisantha Fernando,et al.  Pattern Recognition in a Bucket , 2003, ECAL.

[61]  James M. Tour,et al.  Logic and memory with nanocell circuits , 2003 .

[62]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[63]  Paul D. Franzon,et al.  Nanocell logic gates for molecular computing , 2002 .

[64]  Henry Markram,et al.  The "Liquid Computer": A Novel Strategy for Real-Time Computing on Time Series , 2002 .

[65]  M. Reed,et al.  Molecular random access memory cell , 2001 .

[66]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[67]  Chen,et al.  Large On-Off Ratios and Negative Differential Resistance in a Molecular Electronic Device. , 1999, Science.

[68]  R.H. Dennard,et al.  Design Of Ion-implanted MOSFET's with Very Small Physical Dimensions , 1974, Proceedings of the IEEE.

[69]  Christopher G. Langton,et al.  Computation at the edge of chaos: Phase transitions and emergent computation , 1990 .

[70]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[71]  E. Gardner,et al.  Optimal storage properties of neural network models , 1988 .

[72]  B. Derrida Dynamical phase transition in nonsymmetric spin glasses , 1987 .

[73]  B. Derrida,et al.  Random networks of automata: a simple annealed approximation , 1986 .

[74]  L.O. Chua,et al.  Memristive devices and systems , 1976, Proceedings of the IEEE.

[75]  L. Chua Memristor-The missing circuit element , 1971 .