On Reservoir Computing: From Mathematical Foundations to Unconventional Applications

In a typical unconventional computation setup the goal is to exploit a given dynamical system, which cannot be easily adjusted or programmed, for information processing applications. While one has some intuition of how to use the system, it is often the case that it is not entirely clear how to achieve this in practice. Reservoir computing represents a set of approaches that could be useful in such situations. As a paradigm, reservoir computing harbours enormous technological potential which can be naturally released in the context of unconventional computation. In this chapter several key concepts of reservoir computing are reviewed, re-interpreted, and synthesized to aid in realizing the unconventional computation agenda, and to illustrate what unconventional computation might be. Some philosophical approaches are discussed too, e.g. the strongly related implementation problem. The focus is on understanding reservoir computing in the classical setup, where a single fixed dynamical system is used: To this end, mathematical foundations of reservoir computing are presented, in particular the Stone-Weierstrass approximation theorem, with a mixture of rigor, and intuitive explanations. To make the synthesis it was crucial to thoroughly analyze both the differences and similarities between Liquid State Machines and Echo State Networks, and find a common context insensitive base. The result of the synthesis is the suggested Reservoir Machine model. The model could be used to analyze how to build unconventional information processing devices and to understand their computing capacity.

[1]  Alireza Goudarzi,et al.  Towards a Calculus of Echo State Networks , 2014, BICA.

[2]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[3]  Henry Markram,et al.  A Model for Real-Time Computation in Generic Neural Microcircuits , 2002, NIPS.

[4]  Benjamin Schrauwen,et al.  Information Processing Capacity of Dynamical Systems , 2012, Scientific Reports.

[5]  Miguel C. Soriano,et al.  Minimal approach to neuro-inspired information processing , 2015, Front. Comput. Neurosci..

[6]  H. Putnam.  Representation and Reality , 1993 .

[7]  Adonis Bogris,et al.  High-speed all-optical pattern recognition of dispersive Fourier images through a photonic reservoir computing subsystem. , 2015, Optics letters.

[8]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[9]  Zoran Konkoli,et al.  On Information Processing with Networks of Nano-Scale Switching Elements , 2014, Int. J. Unconv. Comput..

[10]  Herbert Jaeger,et al.  Echo state network , 2007, Scholarpedia.

[11]  W. Rudin.  Principles of mathematical analysis , 1964 .

[12]  Eduardo D. Sontag,et al.  Principles of real-time computing with feedback applied to cortical microcircuit models , 2005, NIPS.

[13]  S. Massar,et al.  Mean Field Theory of Dynamical Systems Driven by External Signals , 2012, ArXiv.

[14]  David J. Chalmers,et al.  A Computational Foundation for the Study of Cognition , 2011 .

[15]  Matthias Scheutz,et al.  When Physical Systems Realize Functions... , 1999, Minds and Machines.

[16]  Christof Teuscher,et al.  Memristor-based reservoir computing , 2012, 2012 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[17]  J. Dieudonne.  Foundations of Modern Analysis , 1969 .

[18]  Leon O. Chua,et al.  Fading memory and the problem of approximating nonlinear operators with volterra series , 1985 .

[19]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[20]  Eduardo D. Sontag,et al.  Computational Aspects of Feedback in Neural Circuits , 2006, PLoS Comput. Biol..

[21]  Benjamin Schrauwen,et al.  Memristor Models for Machine Learning , 2014, Neural Computation.

[22]  Zoran Konkoli,et al.  On the Inverse Pattern Recognition Problem in the Context of the Time-Series Data Processing with Memristor Networks , 2017 .

[23]  David Joslin.  Real realization: Dennett’s real patterns versus Putnam’s ubiquitous automata , 2006, Minds and Machines.

[24]  Stefan J. Kiebel,et al.  Re-visiting the echo state property , 2012, Neural Networks.

[25]  Henry Markram,et al.  On the computational power of circuits of spiking neurons , 2004, J. Comput. Syst. Sci..

[26]  Benjamin Schrauwen,et al.  Reservoir Computing Trends , 2012, KI - Künstliche Intelligenz.

[27]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[28]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[29]  David J. Chalmers,et al.  Does a rock implement every finite-state automaton? , 1996, Synthese.

[30]  Zoran Konkoli,et al.  A Perspective on Putnam's Realizability Theorem in the Context of Unconventional Computation , 2015, Int. J. Unconv. Comput..

[31]  Susan Stepney,et al.  Reservoir Computing with Computational Matter , 2018 .