Efficient design of hardware-enabled reservoir computing in FPGAs

In this work, we propose a new approach towards the efficient optimization and implementation of reservoir computing hardware reducing the required domain expert knowledge and optimization effort. First, we adapt the reservoir input mask to the structure of the data via linear autoencoders. We therefore incorporate the advantages of dimensionality reduction and dimensionality expansion achieved by conventional algorithmically efficient linear algebra procedures of principal component analysis. Second, we employ evolutionary-inspired genetic algorithm techniques resulting in a highly efficient optimization of reservoir dynamics with dramatically reduced number of evaluations comparing to exhaustive search. We illustrate the method on the so-called single-node reservoir computing architecture, especially suitable for implementation in ultrahigh-speed hardware. The combination of both methods and the resulting reduction of time required for performance optimization of a hardware system establish a strategy towards machine learning hardware capable of self-adaption to optimally solve specific problems. We confirm the validity of those principles building reservoir computing hardware based on a field-programmable gate array.

[1]  Laurent Larger,et al.  Virtual chimera states for delayed-feedback systems. , 2013, Physical review letters.

[2]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[3]  Ingo Fischer,et al.  Photonic machine learning implementation for signal recovery in optical communications , 2018, Scientific Reports.

[4]  Daniel Brunner,et al.  Parallel photonic information processing at gigabyte per second data rates using transient states , 2013, Nature Communications.

[5]  Jan Danckaert,et al.  Delay-Based Reservoir Computing: Noise Effects in a Combined Analog and Digital Implementation , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[8]  Zachary Cope,et al.  The London , 1963 .

[9]  Michiel Hermans,et al.  Optoelectronic Systems Trained With Backpropagation Through Time , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[11]  A. Roli Artificial Neural Networks , 2012, Lecture Notes in Computer Science.

[12]  H. Hotelling Analysis of a complex of statistical variables into principal components. , 1933 .

[13]  Benjamin Schrauwen,et al.  Recurrent Kernel Machines: Computing with Infinite Echo State Networks , 2012, Neural Computation.

[14]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[15]  Bogdan Penkovsky Theory and Modeling of Complex Nonlinear Delay Dynamics Applied to Neuromorphic Computing , 2017 .

[16]  Laurent Larger,et al.  Laser chimeras as a paradigm for multistable patterns in complex systems , 2014, Nature Communications.

[17]  Michiel Hermans,et al.  Online Training of an Opto-Electronic Reservoir Computer Applied to Real-Time Channel Equalization , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[19]  Kalyanmoy Deb,et al.  An introduction to genetic algorithms , 1999 .

[20]  Serge Massar,et al.  All-optical Reservoir Computing , 2012, Optics express.

[21]  Karl Pearson F.R.S. LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[22]  Damien Querlioz,et al.  Neuromorphic computing with nanoscale spintronic oscillators , 2017, Nature.

[23]  Mahadev Satyanarayanan,et al.  OpenFace: A general-purpose face recognition library with mobile applications , 2016 .

[24]  Michael J. Watts,et al.  IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS Publication Information , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[25]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[26]  Laurent Larger,et al.  High-Speed Photonic Reservoir Computing Using a Time-Delay-Based Architecture: Million Words per Second Classification , 2017 .

[27]  M. C. Soriano,et al.  Complex photonics: Dynamics and applications of delay-coupled semiconductors lasers , 2013 .

[28]  R. Miikkulainen,et al.  Self-Organization, Plasticity, and Low-Level Visual Phenomena in a Laterally Connected Map Model of the Primary Visual Cortex , 1997 .

[29]  Gordon Pipa,et al.  An Introduction to Delay-Coupled Reservoir Computing , 2015 .

[30]  Geert Morthier,et al.  Experimental demonstration of reservoir computing on a silicon photonics chip , 2014, Nature Communications.

[31]  Julian F. Miller,et al.  Evolvable Systems: From Biology to Hardware - 9th International Conference, ICES 2010, York, UK, September 6-8, 2010. Proceedings , 2010, ICES.

[32]  Laurent Larger,et al.  Photonic nonlinear transient computing with multiple-delay wavelength dynamics. , 2012, Physical review letters.

[33]  Marlien Herselman,et al.  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 2015 .

[34]  Reiner W. Hartenstein,et al.  Field-Programmable Logic Architectures, Synthesis and Applications , 1994, Lecture Notes in Computer Science.

[35]  Adrian Thompson,et al.  An Evolved Circuit, Intrinsic in Silicon, Entwined with Physics , 1996, ICES.

[36]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.