Optoelectronic Reservoir Computing

Reservoir computing is a recently introduced, highly efficient bio-inspired approach for processing time dependent data. The basic scheme of reservoir computing consists of a non linear recurrent dynamical system coupled to a single input layer and a single output layer. Within these constraints many implementations are possible. Here we report an optoelectronic implementation of reservoir computing based on a recently proposed architecture consisting of a single non linear node and a delay line. Our implementation is sufficiently fast for real time information processing. We illustrate its performance on tasks of practical importance such as nonlinear channel equalization and speech recognition, and obtain results comparable to state of the art digital implementations.

[1]  Junghsi Lee,et al.  Adaptive algorithms for bilinear filtering , 1994, Optics & Photonics.

[2]  Benjamin Schrauwen,et al.  Memory versus non-linearity in reservoirs , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[3]  Amir F. Atiya,et al.  New results on recurrent network training: unifying the algorithms and accelerating convergence , 2000, IEEE Trans. Neural Networks Learn. Syst..

[4]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[5]  W. Maass,et al.  What makes a dynamical system computationally powerful ? , 2022 .

[6]  Danko Nikolic,et al.  Temporal dynamics of information content carried by neurons in the primary visual cortex , 2006, NIPS.

[7]  Benjamin Schrauwen,et al.  Memory in linear recurrent neural networks in continuous time , 2010, Neural Networks.

[8]  Dharmendra S. Modha,et al.  The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[9]  Jochen J. Steil,et al.  Recent advances in efficient learning of recurrent networks , 2009, ESANN.

[10]  Benjamin Schrauwen,et al.  Phoneme Recognition with Large Hierarchical Reservoirs , 2010, NIPS.

[11]  Paul Lamere,et al.  Sphinx-4: a flexible open source framework for speech recognition , 2004 .

[12]  Terrence J. Sejnowski,et al.  New Directions in Statistical Signal Processing: From Systems to Brains (Neural Information Processing) , 2006 .

[13]  T. Erneux.  Applied Delay Differential Equations , 2009 .

[14]  Laurent Larger,et al.  Routes to chaos and multiple time scale dynamics in broadband bandpass nonlinear delay electro-optic oscillators. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  J.J. Steil,et al.  Backpropagation-decorrelation: online recurrent learning with O(N) complexity , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[16]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[17]  H. John Caulfield,et al.  Why future supercomputing requires optics , 2010 .

[18]  Laurent Larger,et al.  From flow to map in an experimental high-dimensional electro-optic nonlinear delay oscillator. , 2005, Physical review letters.

[19]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[20]  Jochen J. Steil,et al.  Improving reservoirs using intrinsic plasticity , 2008, Neurocomputing.

[21]  Richard F. Lyon,et al.  A computational model of filtering, detection, and compression in the cochlea , 1982, ICASSP.

[22]  Benjamin Schrauwen,et al.  Toward optical signal processing using photonic reservoir computing. , 2008, Optics express.

[23]  Benjamin Schrauwen,et al.  Reservoir-based techniques for speech recognition , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[24]  Benjamin Schrauwen,et al.  Isolated word recognition using a Liquid State Machine , 2005, ESANN.

[25]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[26]  Benjamin Schrauwen,et al.  An experimental unification of reservoir computing methods , 2007, Neural Networks.

[27]  Laurent Larger,et al.  Chaotic breathers in delayed electro-optical systems. , 2005, Physical review letters.

[28]  Acknowledgments , 2006, Molecular and Cellular Endocrinology.

[29]  Pawel Bujnowski,et al.  Aspiration and Cooperation in Multiperson Prisoner's Dilemma , 2009 .

[30]  Peter Tiño,et al.  Simple Deterministically Constructed Recurrent Neural Networks , 2010, IDEAL.

[31]  Herbert Jaeger,et al.  Adaptive Nonlinear System Identification with Echo State Networks , 2002, NIPS.

[32]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[33]  L. Gould.  The Cat is Out of the Bag , 1996 .

[34]  I. Bizjak,et al.  Measurement of the wrong-sign decays D0 --> K+ pi- pi0 and D0 --> K+ pi- pi+ pi-, and search for CP violation. , 2005, Physical review letters.

[35]  Terrence J. Sejnowski,et al.  What Makes a Dynamical System Computationally Powerful , 2007 .

[36]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[37]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[38]  Johannes Schemmel,et al.  Edge of Chaos Computation in Mixed-Mode VLSI - A Hard Liquid , 2004, NIPS.

[39]  Chrisantha Fernando,et al.  Pattern Recognition in a Bucket , 2003, ECAL.

[40]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.

[41]  J. van Leeuwen,et al.  Intelligent Data Engineering and Automated Learning , 2003, Lecture Notes in Computer Science.