Master memory function for delay-based reservoir computers with single-variable dynamics

We show that many delay-based reservoir computers considered in the literature can be characterized by a universal master memory function (MMF). Once computed for two independent parameters, this function provides linear memory capacity for any delay-based singlevariable reservoir with small inputs. Moreover, we propose an analytical description of the MMF that enables its efficient and fast computation. Our approach can be applied not only to reservoirs governed by known dynamical rules such as Mackey-Glass or Ikeda-like systems, but also to reservoirs whose dynamical model is not available. We also present results comparing the performance of the reservoir computer and the memory capacity given by the MMF.

[1]  Juan-Pablo Ortega,et al.  Reservoir Computing Universality With Stochastic Inputs , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[2]  François Horlin,et al.  Equalization of the Non-Linear Satellite Communication Channel with an Echo State Network , 2015, 2015 IEEE 81st Vehicular Technology Conference (VTC Spring).

[3]  Dan Wang,et al.  Prediction performance of reservoir computing system based on a semiconductor laser subject to double optical feedback and optical injection. , 2018, Optics express.

[4]  Conrad Sanderson,et al.  Armadillo: a template-based C++ library for linear algebra , 2016, J. Open Source Softw..

[5]  Toshiyuki Yamane,et al.  Recent Advances in Physical Reservoir Computing: A Review , 2018, Neural Networks.

[6]  Germany,et al.  Deep Learning with a Single Neuron: Folding a Deep Neural Network in Time using Feedback-Modulated Delay Loops , 2020, ArXiv.

[7]  Miguel C. Soriano,et al.  Comparison of Photonic Reservoir Computing Systems for Fiber Transmission Equalization , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[8]  Daniel Brunner,et al.  Parallel photonic information processing at gigabyte per second data rates using transient states , 2013, Nature Communications.

[9]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[10]  Benjamin Schrauwen,et al.  Optoelectronic Reservoir Computing , 2011, Scientific Reports.

[11]  Laurent Larger,et al.  Optimal nonlinear information processing capacity in delay-based reservoir computers , 2014, Scientific Reports.

[12]  Erik Bollt,et al.  On explaining the surprising success of reservoir computing forecaster of chaos? The universal machine learning dynamical system with contrast to VAR and DMD. , 2020, Chaos.

[13]  Laurent Larger,et al.  Tutorial: Photonic Neural Networks in Delay Systems , 2018, Journal of Applied Physics.

[14]  Atsushi Uchida,et al.  Impact of input mask signals on delay-based photonic reservoir computing with semiconductor lasers. , 2018, Optics express.

[15]  André Röhm,et al.  Performance boost of time-delay reservoir computing by non-resonant clock cycle , 2019, Neural Networks.

[16]  José Carlos Príncipe,et al.  Liquid state machines and cultured cortical networks: The separation property , 2009, Biosyst..

[17]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[18]  Atsushi Uchida,et al.  Laser dynamical reservoir computing with consistency: an approach of a chaos mask signal. , 2016, Optics express.

[19]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[20]  Luis Pesquera,et al.  Reservoir Computing with an Ensemble of Time-Delay Reservoirs , 2017, Cognitive Computation.

[21]  Michiel Hermans,et al.  Online Training of an Opto-Electronic Reservoir Computer Applied to Real-Time Channel Equalization , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Simone Scardapane,et al.  Semi-supervised Echo State Networks for Audio Classification , 2017, Cognitive Computation.

[23]  Jan Danckaert,et al.  Real-time Audio Processing with a Cascade of Discrete-Time Delay Line-Based Reservoir Computers , 2017, Cognitive Computation.

[24]  M. C. Soriano,et al.  Advances in photonic reservoir computing , 2017 .

[25]  Romain Modeste Nguimdo,et al.  Prediction performance of reservoir computing systems based on a diode-pumped erbium-doped microchip laser subject to optical feedback. , 2017, Optics letters.

[26]  Miguel C. Soriano,et al.  Improving detection in optical communications using all-optical reservoir computing , 2017, 2017 Conference on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC).

[27]  Serge Massar,et al.  High performance photonic reservoir computer based on a coherently driven passive cavity , 2015, ArXiv.

[28]  Cristina Masoller,et al.  Machine learning algorithms for predicting the amplitude of chaotic laser pulses. , 2019, Chaos.

[29]  Laurent Larger,et al.  Reinforcement Learning in a large scale photonic Recurrent Neural Network , 2017, Optica.

[30]  Laurent Larger,et al.  Stochastic Nonlinear Time Series Forecasting Using Time-Delay Reservoir Computers: Performance and Universality , 2013, Neural Networks.

[31]  Michelle Girvan,et al.  Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model , 2018, Chaos.

[32]  Germany,et al.  Emulating complex networks with a single delay differential equation , 2021 .

[33]  Claudio Gallicchio,et al.  Richness of Deep Echo State Network Dynamics , 2019, IWANN.

[34]  Kathy Lüdge,et al.  Insight into delay based reservoir computing via eigenvalue analysis , 2021, Journal of Physics: Photonics.

[35]  Aniel,et al.  Conditions for reservoir computing performance using semiconductor lasers with delayed optical feedback , 2017 .

[36]  Benjamin Schrauwen,et al.  Information Processing Capacity of Dynamical Systems , 2012, Scientific Reports.

[37]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[38]  Felix Köster,et al.  Limitations of the recall capabilities in delay based reservoir computing systems , 2020, Cognitive Computation.

[39]  Erik Bollt,et al.  Next generation reservoir computing , 2021, Nature Communications.

[40]  Zhao Yang,et al.  Reservoir computing system with double optoelectronic feedback loops. , 2019, Optics express.

[41]  Ingo Fischer,et al.  Photonic machine learning implementation for signal recovery in optical communications , 2018, Scientific Reports.

[42]  Laurent Larger,et al.  High-Speed Photonic Reservoir Computing Using a Time-Delay-Based Architecture: Million Words per Second Classification , 2017 .

[43]  M. C. Soriano,et al.  Cross-predicting the dynamics of an optically injected single-mode semiconductor laser using reservoir computing. , 2019, Chaos.

[44]  Claudio Gallicchio,et al.  Local Lyapunov exponents of deep echo state networks , 2018, Neurocomputing.

[45]  Joseph D. Hart,et al.  Experiments with arbitrary networks in time-multiplexed delay systems. , 2017, Chaos.

[46]  Geert Morthier,et al.  Experimental demonstration of reservoir computing on a silicon photonics chip , 2014, Nature Communications.

[47]  Daniel Brunner,et al.  Conditions for reservoir computing performance using semiconductor lasers with delayed optical feedback. , 2017, Optics express.

[48]  André Röhm,et al.  Multiplexed networks: reservoir computing with virtual and real nodes , 2018, Journal of Physics Communications.

[49]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[50]  Salim Mejaouri,et al.  Reservoir computing with a single delay-coupled non-linear mechanical oscillator , 2018, Journal of Applied Physics.

[51]  Chrisantha Fernando,et al.  Pattern Recognition in a Bucket , 2003, ECAL.

[52]  Atsushi Uchida,et al.  Reservoir Computing Using Multiple Lasers With Feedback on a Photonic Integrated Circuit , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[53]  Kathy Lüdge,et al.  Reservoir Computing Using Laser Networks , 2020, IEEE Journal of Selected Topics in Quantum Electronics.