Machine learning based on reservoir computing with time-delayed optoelectronic and photonic systems.

The concept of reservoir computing emerged from a specific machine learning paradigm characterized by a three-layered architecture (input, reservoir, and output), where only the output layer is trained and optimized for a particular task. In recent years, this approach has been successfully implemented using various hardware platforms based on optoelectronic and photonic systems with time-delayed feedback. In this review, we provide a survey of the latest advances in this field, with some perspectives related to the relationship between reservoir computing, nonlinear dynamics, and network theory.

[1]  Laurent Larger,et al.  Photonic nonlinear transient computing with multiple-delay wavelength dynamics. , 2012, Physical review letters.

[2]  Jan Danckaert,et al.  Strongly asymmetric square waves in a time-delayed system. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  Laurent Larger,et al.  Mixed-mode oscillations in slow-fast delayed optoelectronic systems. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  Dan Wang,et al.  Prediction performance of reservoir computing system based on a semiconductor laser subject to double optical feedback and optical injection. , 2018, Optics express.

[5]  Paul Woafo,et al.  Cluster synchronization in coupled chaotic semiconductor lasers and application to switching in chaos-secured communication networks , 2003 .

[6]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[7]  Laurent Larger,et al.  Effect of parameter mismatch on the synchronization of chaotic semiconductor lasers with electro-optical feedback. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[8]  Benjamin Schrauwen,et al.  Toward optical signal processing using photonic reservoir computing. , 2008, Optics express.

[9]  Satoshi Sunada,et al.  Compact reservoir computing with a photonic integrated circuit. , 2018, Optics express.

[10]  Luis Pesquera,et al.  Reservoir Computing with an Ensemble of Time-Delay Reservoirs , 2017, Cognitive Computation.

[11]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[12]  L. Larger,et al.  Mismatch-induced bit error rate in optical chaos communications using semiconductor lasers with electrooptical feedback , 2005, IEEE Journal of Quantum Electronics.

[13]  Rajarshi Roy,et al.  Using synchronization for prediction of high-dimensional chaotic dynamics. , 2008, Physical review letters.

[14]  Zheng-Mao Wu,et al.  Effects of Some Operation Parameters on the Performance of a Reservoir Computing System Based on a Delay Feedback Semiconductor Laser With Information Injection by Current Modulation , 2019, IEEE Access.

[15]  F. Bretenaker,et al.  A Model for Designing Ultralow Noise Single- and Dual-Loop 10-GHz Optoelectronic Oscillators , 2017, Journal of Lightwave Technology.

[16]  Atsushi Uchida,et al.  Consistency of nonlinear system response to complex drive signals. , 2004 .

[17]  Eckehard Schöll,et al.  Experimental observations of group synchrony in a system of chaotic optoelectronic oscillators. , 2013, Physical review letters.

[18]  Lute Maleki,et al.  Ultralow-noise mode-locked laser with coupled optoelectronic oscillator configuration. , 2005, Optics letters.

[19]  Miguel C. Soriano,et al.  Reservoir computing with a single time-delay autonomous Boolean node , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[20]  M. C. Soriano,et al.  Constructive Role of Noise for High-Quality Replication of Chaotic Attractor Dynamics Using a Hardware-Based Reservoir Computer , 2019, Physical Review Applied.

[21]  Laurent Larger,et al.  Delayed dynamical systems: networks, chimeras and reservoir computing , 2018, Philosophical Transactions of the Royal Society A.

[22]  J. Oden,et al.  Multi-Gbit/s optical phase chaos communications using a time-delayed optoelectronic oscillator with a three-wave interferometer nonlinearity. , 2017, Chaos.

[23]  Laurent Larger,et al.  Wideband chaos generation using a delayed oscillator and a two-dimensional nonlinearity induced by a quadrature phase-shift-keying electro-optic modulator. , 2011, Optics letters.

[24]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[25]  Laurent Larger,et al.  Complexity in electro-optic delay dynamics: modelling, design and applications , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[26]  Paul Devgan,et al.  A Review of Optoelectronic Oscillators for High Speed Signal Processing Applications , 2013 .

[27]  Giovanni Giacomelli,et al.  Spatio-temporal phenomena in complex systems with time delays , 2017, 2206.03120.

[28]  Y. Hao,et al.  Four-channels reservoir computing based on polarization dynamics in mutually coupled VCSELs system. , 2019, Optics express.

[29]  Yanne K Chembo,et al.  Phase noise performance comparison between optoelectronic oscillators based on optical delay lines and whispering gallery mode resonators. , 2014, Optics express.

[30]  Weimin Zhou,et al.  Injection-locked dual opto-electronic oscillator with ultra-low phase noise and ultra-low spurious level , 2005, IEEE Transactions on Microwave Theory and Techniques.

[31]  R. Brockett,et al.  Reservoir observers: Model-free inference of unmeasured variables in chaotic systems. , 2017, Chaos.

[32]  Jianping Yao,et al.  Optoelectronic Oscillators for High Speed and High Resolution Optical Sensing , 2017, Journal of Lightwave Technology.

[33]  T. Erneux,et al.  Effect of Time Delay on the Stability of Optoelectronic Oscillators Based on Whispering-Gallery Mode Resonators , 2016, IEEE Journal of Quantum Electronics.

[34]  Miguel C. Soriano,et al.  Comparison of Photonic Reservoir Computing Systems for Fiber Transmission Equalization , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[35]  M. Riou,et al.  Temporal pattern recognition with delayed feedback spin-torque nano-oscillators. , 2019, Physical review applied.

[36]  Serge Massar,et al.  Brain-Inspired Photonic Signal Processor for Generating Periodic Patterns and Emulating Chaotic Systems , 2017 .

[37]  Caitlin R. S. Williams,et al.  Synchronization states and multistability in a ring of periodic oscillators: experimentally variable coupling delays. , 2013, Chaos.

[38]  Laurent Larger,et al.  Crenelated fast oscillatory outputs of a two-delay electro-optic oscillator. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[39]  Toshiyuki Yamane,et al.  Recent Advances in Physical Reservoir Computing: A Review , 2018, Neural Networks.

[40]  Laurent Larger,et al.  Stochastic Nonlinear Time Series Forecasting Using Time-Delay Reservoir Computers: Performance and Universality , 2013, Neural Networks.

[41]  Francesco Sorrentino,et al.  Cluster synchronization and isolated desynchronization in complex networks with symmetries , 2013, Nature Communications.

[42]  Adonis Bogris,et al.  High-speed all-optical pattern recognition of dispersive Fourier images through a photonic reservoir computing subsystem. , 2015, Optics letters.

[43]  Benjamin Schrauwen,et al.  Information Processing Capacity of Dynamical Systems , 2012, Scientific Reports.

[44]  Wei Pan,et al.  Fast physical and pseudo random number generation based on a nonlinear optoelectronic oscillator , 2015 .

[45]  Romain Modeste Nguimdo,et al.  Prediction performance of reservoir computing systems based on a diode-pumped erbium-doped microchip laser subject to optical feedback. , 2017, Optics letters.

[46]  Lucas Illing,et al.  Isochronal chaos synchronization of delay-coupled optoelectronic oscillators. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[47]  Yanne K Chembo,et al.  Laser-based optoelectronic generation of narrowband microwave chaos for radars and radio-communication scrambling. , 2017, Optics letters.

[48]  Atsushi Uchida,et al.  Reservoir Computing Using Multiple Lasers With Feedback on a Photonic Integrated Circuit , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[49]  M. C. Soriano,et al.  Advances in photonic reservoir computing , 2017 .

[51]  L. Larger,et al.  Generation of Ultralow Jitter Optical Pulses Using Optoelectronic Oscillators With Time-Lens Soliton-Assisted Compression , 2009, Journal of Lightwave Technology.

[52]  Benjamin Schrauwen,et al.  An experimental unification of reservoir computing methods , 2007, Neural Networks.

[53]  Adilson E Motter,et al.  Robustness of optimal synchronization in real networks. , 2011, Physical review letters.

[54]  Zhao Yang,et al.  Reservoir computing system with double optoelectronic feedback loops. , 2019, Optics express.

[55]  Lin Lin,et al.  Polarization Multiplexing Reservoir Computing Based on a VCSEL With Polarized Optical Feedback , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[56]  R. Roy,et al.  Experimental observation of chimeras in coupled-map lattices , 2012, Nature Physics.

[57]  Damien Querlioz,et al.  Neuromorphic computing with nanoscale spintronic oscillators , 2017, Nature.

[58]  Jan Danckaert,et al.  Slow–fast dynamics of a time-delayed electro-optic oscillator , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[59]  Laurent Larger,et al.  Effects of gain and bandwidth on the multimode behavior of optoelectronic microwave oscillators. , 2008, Optics express.

[60]  Joseph D. Hart,et al.  Adding connections can hinder network synchronization of time-delayed oscillators. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[61]  Laurent Larger,et al.  Theoretical and experimental study of slow-scale Hopf limit-cycles in laser-based wideband optoelectronic oscillators , 2014 .

[62]  Laurent Larger,et al.  Chaos-based communications at high bit rates using commercial fibre-optic links , 2005, Nature.

[63]  Mohammad Reza Salehi,et al.  Optical signal processing using photonic reservoir computing , 2014 .

[64]  Laurent Larger,et al.  High-Speed Photonic Reservoir Computing Using a Time-Delay-Based Architecture: Million Words per Second Classification , 2017 .

[65]  Jacques M. Bahi,et al.  Noise and Chaos Contributions in Fast Random Bit Sequence Generated From Broadband Optoelectronic Entropy Sources , 2014, IEEE Transactions on Circuits and Systems I: Regular Papers.

[66]  Khaldoun Saleh,et al.  Effect of Laser Coupling and Active Stabilization on the Phase Noise Performance of Optoelectronic Microwave Oscillators Based on Whispering-Gallery-Mode Resonators , 2015, IEEE Photonics Journal.

[67]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[68]  Rajarshi Roy,et al.  Chaotic lasers: The world's fastest dice , 2008 .

[69]  Adilson E Motter,et al.  Topological Control of Synchronization Patterns: Trading Symmetry for Stability. , 2019, Physical review letters.

[70]  Li Fang,et al.  All-optical reservoir computer based on saturation of absorption. , 2014, Optics express.

[71]  Serge Massar,et al.  All-optical Reservoir Computing , 2012, Optics express.

[72]  Geert Morthier,et al.  Experimental demonstration of reservoir computing on a silicon photonics chip , 2014, Nature Communications.

[73]  Michiel Hermans,et al.  Embodiment of Learning in Electro-Optical Signal Processors , 2016, Physical review letters.

[74]  Salim Mejaouri,et al.  Reservoir computing with a single delay-coupled non-linear mechanical oscillator , 2018, Journal of Applied Physics.

[75]  Daniel Brunner,et al.  Conditions for reservoir computing performance using semiconductor lasers with delayed optical feedback. , 2017, Optics express.

[76]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[77]  VerstraetenD.,et al.  2007 Special Issue , 2007 .

[78]  Atsushi Uchida,et al.  Laser dynamical reservoir computing with consistency: an approach of a chaos mask signal. , 2016, Optics express.

[79]  David J. Schwab,et al.  A high-bias, low-variance introduction to Machine Learning for physicists , 2018, Physics reports.

[80]  Jian Wang,et al.  Secure communications of CAP-4 and OOK signals over MMF based on electro-optic chaos. , 2017, Optics letters.

[81]  Yu Jin,et al.  Numerical Simulation and Experiment on Optical Packet Header Recognition Utilizing Reservoir Computing Based on Optoelectronic Feedback , 2017, IEEE Photonics Journal.

[82]  P. Woafo,et al.  Transitions from spatiotemporal chaos to cluster and complete synchronization states in a shift-invariant set of coupled nonlinear oscillators. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[83]  Damien Rontani,et al.  Experimental reservoir computing using VCSEL polarization dynamics. , 2019, Optics express.

[84]  Serge Massar,et al.  High performance photonic reservoir computer based on a coherently driven passive cavity , 2015, ArXiv.

[85]  Michiel Hermans,et al.  Online Training of an Opto-Electronic Reservoir Computer Applied to Real-Time Channel Equalization , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[86]  Romain Modeste Nguimdo,et al.  Effect of Fiber Dispersion on Broadband Chaos Communications Implemented by Electro-Optic Nonlinear Delay Phase Dynamics , 2010, Journal of Lightwave Technology.

[87]  Paul Woafo,et al.  Analysis of Phase-Locking in Narrow-Band Optoelectronic Oscillators With Intermediate Frequency , 2015, IEEE Journal of Quantum Electronics.

[88]  Wei Pan,et al.  Optoelectronic Oscillators (OEOs) to Sensing, Measurement, and Detection , 2016, IEEE Journal of Quantum Electronics.

[89]  Romain Modeste Nguimdo,et al.  Phase Noise Performance of Optoelectronic Oscillators Based on Whispering-Gallery Mode Resonators , 2015, IEEE Journal of Quantum Electronics.

[90]  Yu-Shuang Hou,et al.  Prediction and classification performance of reservoir computing system using mutually delay-coupled semiconductor lasers , 2019, Optics Communications.

[91]  Demis Hassabis,et al.  A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play , 2018, Science.

[92]  Damien Rontani,et al.  All-Optical Reservoir Computing on a Photonic Chip Using Silicon-Based Ring Resonators , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[93]  Benjamin Schrauwen,et al.  Optoelectronic Reservoir Computing , 2011, Scientific Reports.

[94]  Ingo Fischer,et al.  PAM-4 Transmission at 1550 nm Using Photonic Reservoir Computing Post-Processing , 2019, IEEE Access.

[96]  Hegui Zhu,et al.  Simultaneous recognition of two channels of optical packet headers utilizing reservoir computing subject to mutual-coupling optoelectronic feedback , 2018 .

[97]  Atsushi Uchida,et al.  Impact of input mask signals on delay-based photonic reservoir computing with semiconductor lasers. , 2018, Optics express.

[98]  Masatoshi Bunsen,et al.  Reservoir computing with a slowly modulated mask signal for preprocessing using a mutually coupled optoelectronic system , 2016 .

[99]  Marc Haelterman,et al.  Virtualization of a Photonic Reservoir Computer , 2016, Journal of Lightwave Technology.

[100]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[101]  Xin Yin,et al.  A Neuromorphic Silicon Photonics Nonlinear Equalizer For Optical Communications With Intensity Modulation and Direct Detection , 2019, Journal of Lightwave Technology.

[102]  M. C. Soriano,et al.  Complex photonics: Dynamics and applications of delay-coupled semiconductors lasers , 2013 .

[103]  Laurent Larger,et al.  Optimal nonlinear information processing capacity in delay-based reservoir computers , 2014, Scientific Reports.

[104]  Andrew Katumba,et al.  Neuromorphic Computing Based on Silicon Photonics and Reservoir Computing , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[105]  Daniel J. Gauthier,et al.  Rapid Time Series Prediction with a Hardware-Based Reservoir Computer , 2018, Chaos.

[106]  Laurent Larger,et al.  Optoelectronic oscillators with time-delayed feedback , 2019, Reviews of Modern Physics.

[107]  YuShuang Hou,et al.  Parallel information processing by a reservoir computing system based on a VCSEL subject to double optical feedback and optical injection. , 2019, Optics express.

[108]  Ying Chang,et al.  Optical packet header identification utilizing an all-optical feedback chaotic reservoir computing , 2016 .

[109]  Zheng-Mao Wu,et al.  Performance optimization research of reservoir computing system based on an optical feedback semiconductor laser under electrical information injection. , 2019, Optics express.

[110]  L. Larger,et al.  On the Phase Noise Performance of Nonlinear Double-Loop Optoelectronic Microwave Oscillators , 2012, IEEE Journal of Quantum Electronics.

[111]  Guy Verschaffelt,et al.  A poor man’s coherent Ising machine based on opto-electronic feedback systems for solving optimization problems , 2019, Nature Communications.

[112]  M. C. Soriano,et al.  Information Processing Using Transient Dynamics of Semiconductor Lasers Subject to Delayed Feedback , 2013, IEEE Journal of Selected Topics in Quantum Electronics.

[113]  A. Uchida,et al.  Fast physical random bit generation with chaotic semiconductor lasers , 2008 .

[114]  Paul Woafo,et al.  Breather and Pulse-Package Dynamics in Multinonlinear Electrooptical Systems With Delayed Feedback , 2016, IEEE Photonics Journal.

[115]  Hongxi Yin,et al.  Efficient optoelectronic reservoir computing with three-route input based on optical delay lines. , 2019, Applied optics.

[116]  Laurent Larger,et al.  Tutorial: Photonic Neural Networks in Delay Systems , 2018, Journal of Applied Physics.

[117]  Louis M. Pecora,et al.  Network Structure Effects in Reservoir Computers , 2019, Chaos.

[118]  Dimitris Syvridis,et al.  Micro ring resonators as building blocks for an all-optical high-speed reservoir-computing bit-pattern-recognition system , 2013 .

[119]  Sarah Uvin,et al.  Low-Loss Photonic Reservoir Computing with Multimode Photonic Integrated Circuits , 2018, Scientific Reports.

[120]  Zhang,et al.  From low-dimensional synchronous chaos to high-dimensional desynchronous spatiotemporal chaos in coupled systems , 2000, Physical review letters.

[121]  Ingo Fischer,et al.  Photonic machine learning implementation for signal recovery in optical communications , 2018, Scientific Reports.

[122]  Daniel Brunner,et al.  Parallel photonic information processing at gigabyte per second data rates using transient states , 2013, Nature Communications.

[123]  Meucci,et al.  Two-dimensional representation of a delayed dynamical system. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[124]  Chii-Chang Chen,et al.  Optical neural networks based on optical fiber-communication system , 2019, Neurocomputing.

[125]  Serge Massar,et al.  Fully analogue photonic reservoir computer , 2016, Scientific Reports.

[126]  L. Larger,et al.  Optoelectronic reservoir computing: tackling noise-induced performance degradation. , 2013, Optics express.

[127]  M. C. Soriano,et al.  A Unified Framework for Reservoir Computing and Extreme Learning Machines based on a Single Time-delayed Neuron , 2015, Scientific Reports.

[128]  Daniel J. Gauthier,et al.  Hopf bifurcations in time-delay systems with band-limited feedback , 2005 .

[129]  Edward Ott,et al.  Complex dynamics and synchronization of delayed-feedback nonlinear oscillators , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[130]  Lute Maleki,et al.  High frequency optical subcarrier generator , 1994 .

[131]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[132]  Joseph D. Hart,et al.  Experiments with arbitrary networks in time-multiplexed delay systems. , 2017, Chaos.

[133]  Damien Rontani,et al.  Enhanced performance of a reservoir computer using polarization dynamics in VCSELs. , 2018, Optics letters.

[134]  Weisheng Hu,et al.  Chaotic optical communications over 100-km fiber transmission at 30-Gb/s bit rate. , 2018, Optics letters.

[135]  K. Alan Shore,et al.  Physics and applications of laser diode chaos , 2015 .