Network Structure Effects in Reservoir Computers

A reservoir computer is a complex nonlinear dynamical system that has been shown to be useful for solving certain problems, such as prediction of chaotic signals, speech recognition, or control of robotic systems. Typically, a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network, driving the nodes with an input signal and using the node outputs to fit a training signal. In this work, we set up reservoirs where the edges (or connections) between all the network nodes are either +1 or 0 and proceed to alter the network structure by flipping some of these edges from +1 to -1. We use this simple network because it turns out to be easy to characterize; we may use the fraction of edges flipped as a measure of how much we have altered the network. In some cases, the network can be rearranged in a finite number of ways without changing its structure; these rearrangements are symmetries of the network, and the number of symmetries is also useful for characterizing the network. We find that changing the number of edges flipped in the network changes the rank of the covariance of a matrix consisting of the time series from the different nodes in the network and speculate that this rank is important for understanding the reservoir computer performance.

[1]  Herbert Jaeger,et al.  Computing optimal discrete readout weights in reservoir computing is NP-hard , 2019, Neurocomputing.

[2]  R. Brockett,et al.  Reservoir observers: Model-free inference of unmeasured variables in chaotic systems. , 2017, Chaos.

[3]  Masanobu Inubushi,et al.  Reservoir Computing Beyond Memory-Nonlinearity Trade-off , 2017, Scientific Reports.

[4]  Herbert Jaeger,et al.  Echo State Property Linked to an Input: Exploring a Fundamental Characteristic of Recurrent Neural Networks , 2013, Neural Computation.

[5]  R. Penrose.  A Generalized inverse for matrices , 1955 .

[6]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.

[7]  Salim Mejaouri,et al.  Reservoir computing with a single delay-coupled non-linear mechanical oscillator , 2018, Journal of Applied Physics.

[8]  M. Golubitsky,et al.  Singularities and groups in bifurcation theory , 1985 .

[9]  Adilson E Motter,et al.  Stable Chimeras and Independently Synchronizable Clusters. , 2017, Physical review letters.

[10]  Edward Ott,et al.  Attractor reconstruction by machine learning. , 2018, Chaos.

[11]  Serge Massar,et al.  Using a reservoir computer to learn chaotic attractors, with applications to chaos synchronisation and cryptography , 2018, Physical review. E.

[12]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[13]  Wesley De Neve,et al.  On the application of reservoir computing networks for noisy image recognition , 2018, Neurocomputing.

[14]  Michael Small,et al.  Consistency in echo-state networks. , 2019, Chaos.

[15]  Louis Pecora,et al.  Approximate cluster synchronization in networks with symmetries and parameter mismatches. , 2016, Chaos.

[16]  K. Aberer,et al.  German National Research Center for Information Technology , 2007 .

[17]  Ulrich Parlitz,et al.  Observing spatio-temporal dynamics of excitable media using reservoir computing. , 2018, Chaos.

[18]  V. Buchstaber,et al.  Mathematical Proceedings of the Cambridge Philosophical Society , 1979 .

[19]  William Stein,et al.  SAGE: Software for Algebra and Geometry Experimentation , 2006 .

[20]  Sarah Marzen,et al.  The difference between memory and prediction in linear recurrent networks , 2017, Physical review. E.

[21]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[22]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[23]  Daniel J. Gauthier,et al.  Rapid Time Series Prediction with a Hardware-Based Reservoir Computer , 2018, Chaos.

[24]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[25]  M. C. Soriano,et al.  Advances in photonic reservoir computing , 2017 .

[26]  W. Marsden.  I and J , 2012 .

[27]  Benjamin Schrauwen,et al.  Reservoir Computing Trends , 2012, KI - Künstliche Intelligenz.

[28]  Francesco Sorrentino,et al.  Cluster synchronization and isolated desynchronization in complex networks with symmetries , 2013, Nature Communications.