Evolutionary aspects of reservoir computing

Reservoir computing (RC) is a powerful computational paradigm that allows high versatility with cheap learning. While other artificial intelligence approaches need exhaustive resources to specify their inner workings, RC is based on a reservoir with highly nonlinear dynamics that does not require a fine tuning of its parts. These dynamics project input signals into high-dimensional spaces, where training linear readouts to extract input features is vastly simplified. Thus, inexpensive learning provides very powerful tools for decision-making, controlling dynamical systems, classification, etc. RC also facilitates solving multiple tasks in parallel, resulting in a high throughput. Existing literature focuses on applications in artificial intelligence and neuroscience. We review this literature from an evolutionary perspective. RC’s versatility makes it a great candidate to solve outstanding problems in biology, which raises relevant questions. Is RC as abundant in nature as its advantages should imply? Has it evolved? Once evolved, can it be easily sustained? Under what circumstances? (In other words, is RC an evolutionarily stable computing paradigm?) To tackle these issues, we introduce a conceptual morphospace that would map computational selective pressures that could select for or against RC and other computing paradigms. This guides a speculative discussion about the questions above and allows us to propose a solid research line that brings together computation and evolution with RC as test model of the proposed hypotheses. This article is part of the theme issue ‘Liquid brains, solid brains: How distributed cognitive architectures process information’.

[1]  X. R. Wang,et al.  Relating Fisher information to order parameters. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[2]  David M. Raup,et al.  Geometric analysis of shell coiling; general problems , 1966 .

[3]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[4]  Peter Ford Dominey,et al.  Real-Time Parallel Processing of Grammatical Structure in the Fronto-Striatal System: A Recurrent Network Simulation Study Using Reservoir Computing , 2013, PloS one.

[5]  H. S. Meyer,et al.  Cell Type–Specific Three-Dimensional Structure of Thalamocortical Circuits in a Column of Rat Vibrissal Cortex , 2011, Cerebral cortex.

[6]  Michael Levin,et al.  On Having No Head: Cognition throughout Biological Systems , 2016, Front. Psychol..

[7]  J. Hopfield Physics, Computation, and Why Biology Looks so Different , 1994 .

[8]  W. Bialek,et al.  Are Biological Systems Poised at Criticality? , 2010, 1012.2242.

[9]  Tad McGeer,et al.  Passive Dynamic Walking , 1990, Int. J. Robotics Res..

[10]  Minoru Asada,et al.  Information processing in echo state networks at the edge of chaos , 2011, Theory in Biosciences.

[11]  Chrisantha Fernando,et al.  Pattern Recognition in a Bucket , 2003, ECAL.

[12]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[13]  Pablo Balenzuela,et al.  Criticality in Large-Scale Brain fMRI Dynamics Unveiled by a Novel Point Process Analysis , 2012, Front. Physio..

[14]  Uri Alon,et al.  The Mass-Longevity Triangle: Pareto Optimality and the Geometry of Life-History Trait Space , 2015, PLoS Comput. Biol..

[15]  Thomas L Russell,et al.  Retinal synaptic pathways underlying the response of the rabbit local edge detector. , 2010, Journal of neurophysiology.

[16]  Auke Jan Ijspeert,et al.  Central pattern generators for locomotion control in animals and robots: A review , 2008, Neural Networks.

[17]  Matej Hoffmann,et al.  What Is Morphological Computation? On How the Body Contributes to Cognition and Control , 2017, Artificial Life.

[18]  Uri Alon,et al.  Inferring biological tasks using Pareto analysis of high-dimensional data , 2015, Nature Methods.

[19]  W. Maass,et al.  What makes a dynamical system computationally powerful ? , 2022 .

[20]  H. Ritter,et al.  Self-organizing maps for internal representations , 1990, Psychological research.

[21]  Stefano Fusi,et al.  Why neurons mix: high dimensionality for higher cognition , 2016, Current Opinion in Neurobiology.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  Martijn Wisse,et al.  Design and Construction of MIKE; a 2-D Autonomous Biped Based on Passive Dynamic Walking , 2006 .

[24]  G. R. Mcghee,et al.  Theoretical Morphology: The Concept and Its Applications , 1998 .

[25]  K. Niklas COMPUTER MODELS OF EARLY LAND PLANT EVOLUTION , 2004 .

[26]  Helmut Hauser,et al.  Computing with a muscular-hydrostat system , 2013, 2013 IEEE International Conference on Robotics and Automation.

[27]  Phil Husbands,et al.  Feathered Flyer: Integrating Morphological Computation and Sensory Reflexes into a Physically Simulated Flapping-Wing Robot for Robust Flight Manoeuvre , 2007, ECAL.

[28]  Wolfgang Maass,et al.  Searching for principles of brain computation , 2016, Current Opinion in Behavioral Sciences.

[29]  W. Wildman,et al.  Theoretical Neuroscience , 2014 .

[30]  F. Helmchen,et al.  Behaviour-dependent recruitment of long-range projection neurons in somatosensory cortex , 2013, Nature.

[31]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[32]  Jordi Garcia-Ojalvo,et al.  Recurrence-based information processing in gene regulatory networks. , 2018, Chaos.

[33]  D. Chialvo Emergent complex neural dynamics , 2010, 1010.2530.

[34]  Han Ju,et al.  Spatiotemporal Memory Is an Intrinsic Property of Networks of Dissociated Cortical Neurons , 2015, The Journal of Neuroscience.

[35]  Ricard V Solé,et al.  Information theory, predictability and the emergence of complex life , 2017, Royal Society Open Science.

[36]  Wolfgang Maass,et al.  Movement Generation and Control with Generic Neural Microcircuits , 2004, BioADIT.

[37]  Nils Bertschinger,et al.  Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks , 2004, Neural Computation.

[38]  Henry Markram,et al.  Theory of the Computational Function of Microcircuit Dynamics , 2004 .

[39]  Seung Hwan Lee,et al.  Reservoir computing using dynamic memristors for temporal information processing , 2017, Nature Communications.

[40]  Helmut Hauser,et al.  A soft body as a reservoir: case studies in a dynamic model of octopus-inspired soft robotic arm , 2013, Front. Comput. Neurosci..

[41]  Helmut Hauser,et al.  Towards a theoretical foundation for morphological computation with compliant bodies , 2011, Biological Cybernetics.

[42]  Rolf Pfeifer,et al.  How the body shapes the way we think - a new view on intelligence , 2006 .

[43]  José Carlos Príncipe,et al.  Special issue on echo state networks and liquid state machines , 2007, Neural Networks.

[44]  Robert A. Legenstein,et al.  2007 Special Issue: Edge of chaos and prediction of computational performance for neural circuit models , 2007 .

[45]  O. Sporns,et al.  Network morphospace , 2015, Journal of The Royal Society Interface.

[46]  Wolfgang Maass,et al.  Cerebral Cortex Advance Access published February 15, 2006 A Statistical Analysis of Information- Processing Properties of Lamina-Specific , 2022 .

[47]  Alexander Rivkind,et al.  Local Dynamics in Trained Recurrent Neural Networks. , 2015, Physical review letters.

[48]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[49]  Tao Li,et al.  Information processing via physical soft body , 2015, Scientific Reports.

[50]  Stefan Habenschuss,et al.  Stochastic Computations in Cortical Microcircuit Models , 2013, PLoS Comput. Biol..

[51]  Michael J. Berry,et al.  High Accuracy Decoding of Dynamical Motion from a Large Retinal Population , 2014, PLoS Comput. Biol..

[52]  R. Weale Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[53]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[54]  R. Thomas,et al.  Boolean formalization of genetic control circuits. , 1973, Journal of theoretical biology.

[55]  H. Seo,et al.  A reservoir of time constants for memory traces in cortical neurons , 2011, Nature Neuroscience.

[56]  Joaquín Goñi,et al.  On the origins of hierarchy in complex networks , 2013, Proceedings of the National Academy of Sciences.

[57]  Eric Smith,et al.  The Origin and Nature of Life on Earth: The Emergence of the Fourth Geosphere , 2016 .

[58]  W. Levick Receptive fields and trigger features of ganglion cells in the visual streak of the rabbit's retina , 1967, The Journal of physiology.

[59]  Gerald F. Joyce,et al.  Molecular evolution: Booting up life , 2002, Nature.

[60]  A. Clark Being There: Putting Brain, Body, and World Together Again , 1996 .

[61]  David M. Raup,et al.  How Nature Works: The Science of Self-Organized Criticality , 1997 .

[62]  Benjamin Schrauwen,et al.  Reservoir-based techniques for speech recognition , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[63]  Benjamin Schrauwen,et al.  Locomotion Without a Brain: Physical Reservoir Computing in Tensegrity Structures , 2013, Artificial Life.

[64]  Giulio Ruffini,et al.  Detection of Generalized Synchronization using Echo State Networks , 2017, Chaos.

[65]  W. Singer Cortical dynamics revisited , 2013, Trends in Cognitive Sciences.

[66]  Geert Morthier,et al.  Experimental demonstration of reservoir computing on a silicon photonics chip , 2014, Nature Communications.

[67]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[68]  Per Bak,et al.  How Nature Works , 1996 .

[69]  John M. Beggs,et al.  Neuronal Avalanches in Neocortical Circuits , 2003, The Journal of Neuroscience.

[70]  Lorenzo Livi,et al.  Investigating echo state networks dynamics by means of recurrence analysis , 2016, IEEE Trans. Neural Networks Learn. Syst..

[71]  Lorenzo Livi,et al.  Determination of the Edge of Criticality in Echo State Networks Through Fisher Information Maximization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[72]  Rolf Pfeifer,et al.  On the influence of morphology of tactile sensors for behavior and control , 2006, Robotics Auton. Syst..

[73]  W. Singer,et al.  Distributed Fading Memory for Stimulus Properties in the Primary Visual Cortex , 2009, PLoS biology.

[74]  Vinita Silaparasetty Perceptrons , 2021, Deep Learning Projects Using TensorFlow 2.

[75]  Eduardo D. Sontag,et al.  Computational Aspects of Feedback in Neural Circuits , 2006, PLoS Comput. Biol..

[76]  B. Corominas-Murtra,et al.  On the basic computational structure of gene regulatory networks. , 2009, Molecular bioSystems.

[77]  Wolfgang Maass,et al.  Emergence of Dynamic Memory Traces in Cortical Microcircuit Models through STDP , 2013, The Journal of Neuroscience.

[78]  Chandana Paul,et al.  The Tendon Network of the Fingers Performs Anatomical Computation at a Macroscopic Scale , 2007, IEEE Transactions on Biomedical Engineering.

[79]  Henry Markram,et al.  Computational models for generic cortical microcircuits , 2004 .

[80]  Sumaiyah,et al.  Other Minds : The Octopus , the Sea , and the Deep Origins of Consciousness , 2018 .

[81]  Lorenzo Livi,et al.  Interpreting RNN behaviour via excitable network attractors , 2018, ArXiv.

[82]  The Concept of Information in Biology , 2000, Philosophy of Science.

[83]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[84]  Stephen Wolfram,et al.  Universality and complexity in cellular automata , 1983 .

[85]  Benjamin Schrauwen,et al.  Stable Output Feedback in Reservoir Computing Using Ridge Regression , 2008, ICANN.

[86]  Eva Jablonka,et al.  The evolution of information in the major transitions. , 2006, Journal of theoretical biology.

[87]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[88]  Benjamin Schrauwen,et al.  On Computational Power and the Order-Chaos Phase Transition in Reservoir Computing , 2008, NIPS.

[89]  L. Abbott,et al.  Beyond the edge of chaos: amplification and temporal integration by recurrent networks in the chaotic regime. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[90]  T. Mione,et al.  The evolutionary biology of plants , 1998, Economic Botany.

[91]  Paul Nurse,et al.  Life, logic and information , 2008, Nature.

[92]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[93]  W. Maass,et al.  A quantitative analysis of information about past and present stimuli encoded by spikes of A1 neurons. , 2012, Journal of neurophysiology.

[94]  W. Bialek,et al.  Statistical thermodynamics of natural images. , 2008, Physical review letters.

[95]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[96]  Antonius M J VanDongen,et al.  Short-Term Memory in Networks of Dissociated Cortical Neurons , 2013, The Journal of Neuroscience.

[97]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[98]  O Shoval,et al.  Evolutionary Trade-Offs, Pareto Optimality, and the Geometry of Phenotype Space , 2012, Science.

[99]  Stefano Nichele,et al.  Reservoir Computing Using Nonuniform Binary Cellular Automata , 2017, Complex Syst..

[100]  Ricard V. Sol'e,et al.  Systems poised to criticality through Pareto selective forces , 2015, 1510.08697.

[101]  Benjamin Schrauwen,et al.  The body as a reservoir: locomotion and sensing with linear feedback , 2011 .

[102]  Benjamin Schrauwen,et al.  Reservoir Computing Trends , 2012, KI - Künstliche Intelligenz.

[103]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[104]  Ricard V. Sol'e,et al.  A multiobjective optimization approach to statistical mechanics , 2013, 1310.6372.

[105]  Gerald F. Joyce,et al.  Bit by Bit: The Darwinian Basis of Life , 2012, PLoS biology.

[106]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[107]  Anthony J. Dellureficio A Crack in Creation: Gene Editing and the Unthinkable Power to Control Evolution. By Jennifer A. Doudna and Samuel H. Sternberg. Boston (Massachusetts): Houghton Mifflin Harcourt Publishers. $28.00. xx + 281 p.; ill.; index. ISBN: 9780544716940 (hc); 9780544716964 (eb). 2017. , 2019 .

[108]  Jorge Wagensberg,et al.  Complexity versus Uncertainty: The Question of Staying Alive , 2000 .

[109]  Uri Alon,et al.  Evolutionary tradeoffs, Pareto optimality and the morphology of ammonite shells , 2015, BMC Systems Biology.

[110]  Lorenzo Livi,et al.  On the Interpretation and Characterization of Echo State Networks Dynamics: A Complex Systems Perspective , 2018, Advances in Data Analysis with Computational Intelligence Methods.

[111]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[112]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[113]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[114]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[115]  Helmut Hauser,et al.  The role of feedback in morphological computation with compliant bodies , 2012, Biological Cybernetics.

[116]  A. Lansner,et al.  The cortex as a central pattern generator , 2005, Nature Reviews Neuroscience.

[117]  Udo Kischka,et al.  A neural network model of phantom limbs , 1995, Biological Cybernetics.

[118]  Garrison W. Cottrell,et al.  2007 Special Issue: Learning grammatical structure with Echo State Networks , 2007 .

[119]  Yun Wang,et al.  Synaptic connections and small circuits involving excitatory and inhibitory neurons in layers 2-5 of adult rat and cat neocortex: triple intracellular recordings and biocytin labelling in vitro. , 2002, Cerebral cortex.

[120]  Jan Danckaert,et al.  Delay-Based Reservoir Computing: Noise Effects in a Combined Analog and Digital Implementation , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[121]  John Maynard Smith,et al.  From replicators to reproducers: the first major transitions leading to life. , 1997, Journal of theoretical biology.

[122]  Ricard Solé,et al.  Phase transitions in Pareto optimal complex networks. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[123]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[124]  Paul-Gerhard Plöger,et al.  Echo State Networks used for Motor Control , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[125]  R. Pfeifer,et al.  Self-Organization, Embodiment, and Biologically Inspired Robotics , 2007, Science.

[126]  Javier Macia,et al.  How to Make a Synthetic Multicellular Computer , 2014, PloS one.

[127]  M. A. Muñoz Colloquium: Criticality and dynamical scaling in living systems , 2017, Reviews of Modern Physics.

[128]  Benjamin Schrauwen,et al.  Phoneme Recognition with Large Hierarchical Reservoirs , 2010, NIPS.

[129]  James P. Crutchfield,et al.  Revisiting the Edge of Chaos: Evolving Cellular Automata to Perform Computations , 1993, Complex Syst..

[130]  J. García-Ojalvo,et al.  Hypoarousal non-stationary ADHD biomarker based on echo-state networks , 2018, bioRxiv.

[131]  Mantas Lukosevicius,et al.  A Practical Guide to Applying Echo State Networks , 2012, Neural Networks: Tricks of the Trade.

[132]  Nikolaus Kriegeskorte,et al.  Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation , 2014, PLoS Comput. Biol..

[133]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[134]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[135]  Ronald G. Harley,et al.  An Introduction to the Echo State Network and its Applications in Power System , 2009, 2009 15th International Conference on Intelligent System Applications to Power Systems.

[136]  S. Kauffman At Home in the Universe: The Search for the Laws of Self-Organization and Complexity , 1995 .

[137]  Andrew R. McKinstry-Wu,et al.  Connectome: How the Brain’s Wiring Makes Us Who We Are , 2013 .

[138]  Satoshi Murata,et al.  Self-reconfigurable robots , 2007, IEEE Robotics & Automation Magazine.

[139]  Benjamin Schrauwen,et al.  Optoelectronic Reservoir Computing , 2011, Scientific Reports.

[140]  Ricard V. Solé,et al.  The morphospace of language networks , 2018, Scientific Reports.

[141]  Henry Markram,et al.  Fading memory and kernel properties of generic cortical microcircuit models , 2004, Journal of Physiology-Paris.

[142]  Benjamin Schrauwen,et al.  An experimental unification of reservoir computing methods , 2007, Neural Networks.

[143]  P. Schuster How does complexity arise in evolution: Nature's recipe for mastering scarcity, abundance, and unpredictability , 1996 .

[144]  Helmut Hauser,et al.  Exploiting short-term memory in soft body dynamics as a computational resource , 2014, Journal of The Royal Society Interface.

[145]  Carlos A. Coello Coello,et al.  Evolutionary multi-objective optimization: a historical view of the field , 2006, IEEE Comput. Intell. Mag..

[146]  Christoph Adami,et al.  Annals of the New York Academy of Sciences the Use of Information Theory in Evolutionary Biology , 2022 .

[147]  Ricard Sol'e,et al.  Multiobjective Optimization and Phase Transitions , 2015, ECCS.

[148]  Kunihiko Fukushima,et al.  Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition , 1982 .

[149]  D. Kleinfeld,et al.  'Where' and 'what' in the whisker sensorimotor system , 2008, Nature Reviews Neuroscience.

[150]  Jorge Hidalgo,et al.  Information-based fitness and the emergence of criticality in living systems , 2013, Proceedings of the National Academy of Sciences.

[151]  Gheorghe Paun,et al.  DNA Computing: New Computing Paradigms , 1998 .

[152]  Helmut Hauser,et al.  Computation with mechanically coupled springs for compliant robots , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[153]  B. Schrauwen,et al.  Isolated word recognition with the Liquid State Machine: a case study , 2005, Inf. Process. Lett..

[154]  M. A. Muñoz,et al.  Griffiths phases and the stretching of criticality in brain networks , 2013, Nature Communications.

[155]  Luís F. Seoane Multiobjetive optimization in models of synthetic and natural living systems , 2016 .

[156]  Peter Schuster Optimization of multiple criteria: Pareto efficiency and fast heuristics should be more popular than they are , 2012, Complex..

[157]  E. Marder,et al.  Principles of rhythmic motor pattern generation. , 1996, Physiological reviews.

[158]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[159]  Robert A. Legenstein,et al.  Methods for Estimating the Computational Power and Generalization Capability of Neural Microcircuits , 2004, NIPS.

[160]  Christopher G. Langton,et al.  Computation at the edge of chaos: Phase transitions and emergent computation , 1990 .

[161]  Sara Imari Walker,et al.  The algorithmic origins of life , 2012, Journal of The Royal Society Interface.