Online Training for High-Performance Analogue Readout Layers in Photonic Reservoir Computers

Reservoir computing is a bio-inspired computing paradigm for processing time-dependent signals. The performance of its hardware implementation is comparable to state-of-the-art digital algorithms on a series of benchmark tasks. The major bottleneck of its implementations is the readout layer, based on slow offline post-processing. Few analogue solutions have been proposed, but all suffered from noticeable decrease in performance due to added complexity of the setup. Here, we propose the use of online training to solve these issues. We study the applicability of this method using numerical simulations of an experimentally feasible reservoir computer with an analogue readout layer. We also consider a nonlinear output layer, which would be very difficult to train with traditional methods. We show numerically that online learning allows to circumvent the added complexity of the analogue layer and obtain the same level of performance as with a digital layer. This work paves the way to high-performance fully analogue reservoir computers through the use of online training of the output layers.

[1]  L. Larger,et al.  Optoelectronic reservoir computing: tackling noise-induced performance degradation. , 2013, Optics express.

[2]  François Horlin,et al.  High performance bio-inspired analog equalizer for DVB-S2 non-linear communication channel , 2016, 2016 IEEE 17th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[3]  Zach DeVito,et al.  Opt , 2017 .

[4]  Serge Massar,et al.  Fully analogue photonic reservoir computer , 2016, Scientific Reports.

[5]  Jan Danckaert,et al.  Delay-Based Reservoir Computing: Noise Effects in a Combined Analog and Digital Implementation , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Gunnar Rätsch,et al.  Advanced Lectures on Machine Learning , 2004, Lecture Notes in Computer Science.

[7]  Geert Morthier,et al.  Experimental demonstration of reservoir computing on a silicon photonics chip , 2014, Nature Communications.

[8]  G. Arfken.  Mathematical Methods for Physicists , 1967 .

[9]  Benjamin Schrauwen,et al.  Optoelectronic Reservoir Computing , 2011, Scientific Reports.

[10]  Laurent Larger,et al.  Photonic nonlinear transient computing with multiple-delay wavelength dynamics. , 2012, Physical review letters.

[11]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[12]  Amir Hussain,et al.  Novel Biologically Inspired Approaches to Extracting Online Information from Temporal Data , 2014, Cognitive Computation.

[13]  Daniel Brunner,et al.  Parallel photonic information processing at gigabyte per second data rates using transient states , 2013, Nature Communications.

[14]  W. Graham Richards,et al.  Art of electronics , 1983, Nature.

[15]  L. Christophorou.  Science , 2018, Emerging Dynamics: Science, Energy, Society and Values.

[16]  Abdelkader Benyettou,et al.  Novel Approach Using Echo State Networks for Microscopic Cellular Image Segmentation , 2016, Cognitive Computation.

[17]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[18]  Henri H. Arsenault,et al.  Optical Processing and Computing , 1989 .

[19]  Serge Massar,et al.  All-optical Reservoir Computing , 2012, Optics express.

[20]  Donald N. B. Hall,et al.  SPIE's 1994 International Symposium on Optics, Imaging, and Instrumentation , 1994 .

[21]  J. E. Glynn,et al.  Numerical Recipes: The Art of Scientific Computing , 1989 .

[22]  Michiel Hermans,et al.  Online Training of an Opto-Electronic Reservoir Computer Applied to Real-Time Channel Equalization , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Thomas J. Naughton,et al.  Photonic neural networks , 2012, Nature Physics.

[24]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[25]  Serge Massar,et al.  High performance photonic reservoir computer based on a coherently driven passive cavity , 2015, ArXiv.

[26]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.

[27]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[28]  A. Tikhonov,et al.  Numerical Methods for the Solution of Ill-Posed Problems , 1995 .

[29]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[30]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.