Training a Multi-Layer Photonic Spiking Neural Network With Modified Supervised Learning Algorithm Based on Photonic STDP

We propose a framework for hardware architecture and learning algorithm co-design of multi-layer photonic spiking neural network (SNN). The vertical-cavity surface-emitting laser with an embedded saturable absorber (VCSEL-SA) which contains two polarization-resolved modes is employed as a spiking neuron. The connection between two identical polarization modes is considered as the excitatory synapse, whereas the connection between two orthogonal polarization modes is regarded as the inhibitory synapse. The physical model of the photonic spiking neuron is derived based on the combination of spin-flip model and Yamada model. The photonic spike timing dependent plasticity (STDP) is applied to design a hardware-friendly biologically plausible supervised learning algorithm for a multi-layer photonic SNN. Thanks to the polarization mode competition effect in the VCSEL-SA, the proposed neuromorphic network is capable of solving the classical XOR problem. The effect of physical parameters of photonic neuron on the training convergence is also considered. We further extend the multi-layer photonic SNN to realize other logic learning tasks. To the best of our knowledge, such a modified supervised learning algorithm dedicated for a multi-layer photonic SNN has not yet been reported, which is interesting for spiking learning of neuromorphic photonics.

[1]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[2]  Antonio Hurtado,et al.  Optical neuron using polarisation switching in a 1550nm-VCSEL. , 2010, Optics express.

[3]  Wei Pan,et al.  Cascadable Neuron-Like Spiking Dynamics in Coupled VCSELs Subject to Orthogonally Polarized Optical Pulse Injection , 2017, IEEE Journal of Selected Topics in Quantum Electronics.

[4]  Antonio Hurtado,et al.  Investigation of vertical cavity surface emitting laser dynamics for neuromorphic photonic systems , 2012 .

[5]  Xiao-Dong Lin,et al.  Stable Propagation of Inhibited Spiking Dynamics in Vertical-Cavity Surface-Emitting Lasers for Neuromorphic Photonic Networks , 2018, IEEE Access.

[6]  Paul R. Prucnal,et al.  Recent progress in semiconductor excitable lasers for photonic spike processing , 2016 .

[7]  J. Feldmann,et al.  All-optical spiking neurosynaptic networks with self-learning capabilities , 2019, Nature.

[8]  Michael Pfeiffer,et al.  Deep Learning With Spiking Neurons: Opportunities and Challenges , 2018, Front. Neurosci..

[9]  Paul R. Prucnal,et al.  Photonic Implementation of Spike-Timing-Dependent Plasticity and Learning Algorithms of Biological Neural Systems , 2016, Journal of Lightwave Technology.

[10]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[11]  Ioana Sporea,et al.  Supervised Learning in Multilayer Spiking Neural Networks , 2012, Neural Computation.

[12]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[13]  Matěj Hejda,et al.  Ultrafast optical integration and pattern classification for neuromorphic photonics based on spiking VCSEL neurons , 2020, Scientific Reports.

[14]  Qing Wu,et al.  Efficient and self-adaptive in-situ learning in multilayer memristor neural networks , 2018, Nature Communications.

[15]  Paul R. Prucnal,et al.  Silicon Photonic Modulator Neuron , 2018, Physical Review Applied.

[16]  Yukihiro Kaneko,et al.  Supervised Learning Using Spike-Timing-Dependent Plasticity of Memristive Synapses , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[17]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[18]  Mingguo Zhao,et al.  Towards artificial general intelligence with hybrid Tianjic chip architecture , 2019, Nature.

[19]  Jianye Zhao,et al.  Optical spike-timing-dependent plasticity with weight-dependent learning window and reward modulation. , 2015, Optics express.

[20]  Haizhou Li,et al.  A Spiking Neural Network System for Robust Sequence Recognition , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Indranil Chakraborty,et al.  Photonic In-Memory Computing Primitive for Spiking Neural Networks Using Phase-Change Materials , 2019, Physical Review Applied.

[22]  Tarek El-Ghazawi,et al.  Neuromorphic photonics with electro-absorption modulators. , 2018, Optics express.

[23]  Genquan Han,et al.  Spike Sequence Learning in a Photonic Spiking Neural Network Consisting of VCSELs-SA With Supervised Training , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[24]  Paul R. Prucnal,et al.  Temporal Information Processing With an Integrated Laser Neuron , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[25]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[26]  J. Yang,et al.  Memristive crossbar arrays for brain-inspired computing , 2019, Nature Materials.

[27]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[28]  Catherine D. Schuman,et al.  A Survey of Neuromorphic Computing and Neural Networks in Hardware , 2017, ArXiv.

[29]  Joshua Robertson,et al.  Controlled Propagation of Spiking Dynamics in Vertical-Cavity Surface-Emitting Lasers: Towards Neuromorphic Photonic Networks , 2017, IEEE Journal of Selected Topics in Quantum Electronics.

[30]  Robert A. Nawrocki,et al.  A Mini Review of Neuromorphic Architectures and Implementations , 2016, IEEE Transactions on Electron Devices.

[31]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[32]  Shuiying Xiang,et al.  STDP-Based Unsupervised Spike Pattern Learning in a Photonic Spiking Neural Network With VCSELs and VCSOAs , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[33]  Stefan Schliebs,et al.  Span: Spike Pattern Association Neuron for Learning Spatio-Temporal Spike Patterns , 2012, Int. J. Neural Syst..

[34]  Gang Pan,et al.  Darwin: a neuromorphic hardware co-processor based on Spiking Neural Networks , 2015, Science China Information Sciences.

[35]  T. Sejnowski,et al.  Reliability of spike timing in neocortical neurons. , 1995, Science.

[36]  Yue Hao,et al.  Numerical Implementation of Wavelength-Dependent Photonic Spike Timing Dependent Plasticity Based on VCSOA , 2018, IEEE Journal of Quantum Electronics.

[37]  San Miguel M,et al.  Light-polarization dynamics in surface-emitting semiconductor lasers. , 1995, Physical review. A, Atomic, molecular, and optical physics.

[38]  Tim Gollisch,et al.  Rapid Neural Coding in the Retina with Relative Spike Latencies , 2008, Science.

[39]  Joshua Robertson,et al.  Controlled inhibition of spiking dynamics in VCSELs for neuromorphic photonics: theory and experiments. , 2017, Optics letters.

[40]  Kaushik Roy,et al.  Towards spike-based machine intelligence with neuromorphic computing , 2019, Nature.

[41]  Mable P. Fok,et al.  Photonic implementation of a neuronal algorithm applicable towards angle of arrival detection and localization. , 2015, Optics express.

[42]  Joshua Robertson,et al.  Toward Neuromorphic Photonic Networks of Ultrafast Spiking Laser Neurons , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[43]  Ammar Belatreche,et al.  A review of learning in biologically plausible spiking neural networks , 2019, Neural Networks.

[44]  Paul R. Prucnal,et al.  Machine Learning With Neuromorphic Photonics , 2019, Journal of Lightwave Technology.

[45]  Andrzej J. Kasinski,et al.  Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting , 2010, Neural Computation.

[46]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[47]  Claudio R Mirasso,et al.  Intensity and polarization self-pulsations in vertical-cavity surface-emitting lasers. , 2002, Optics letters.

[48]  Yue Hao,et al.  Polarization-resolved and polarization- multiplexed spike encoding properties in photonic neuron based on VCSEL-SA , 2018, Scientific Reports.

[49]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.

[50]  R Kuszelewicz,et al.  Relative refractory period in an excitable semiconductor laser. , 2014, Physical review letters.

[51]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[52]  Yusuf Leblebici,et al.  Neuromorphic computing with multi-memristive synapses , 2017, Nature Communications.

[53]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[54]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[55]  Timothée Masquelier,et al.  Deep Learning in Spiking Neural Networks , 2018, Neural Networks.

[56]  Rufin van Rullen,et al.  Neurons Tune to the Earliest Spikes Through STDP , 2005, Neural Computation.

[57]  Matěj Hejda,et al.  VCSELs for Fast Neuromorphic Photonic Systems Operating at GHz Rates , 2020, 2020 Optical Fiber Communications Conference and Exhibition (OFC).

[58]  Mable P. Fok.  A Review: Neural-Inspired Photonic Functional Systems for Dynamic RF Signal Processing , 2020, Journal of Lightwave Technology.

[59]  S. Thorpe,et al.  Spike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike Trains , 2008, PloS one.

[60]  S. Xiang,et al.  All-optical inhibitory dynamics in photonic neuron based on polarization mode competition in a VCSEL with an embedded saturable absorber. , 2019, Optics letters.

[61]  Harish Bhaskaran,et al.  On-chip photonic synapse , 2017, Science Advances.

[62]  J Gautrais,et al.  Rate coding versus temporal order coding: a theoretical approach. , 1998, Bio Systems.

[63]  P. R. Prucnal,et al.  A Leaky Integrate-and-Fire Laser Neuron for Ultrafast Cognitive Computing , 2013, IEEE Journal of Selected Topics in Quantum Electronics.

[64]  Yue Tian,et al.  Pulse lead/lag timing detection for adaptive feedback and control based on optical spike-timing-dependent plasticity. , 2013, Optics letters.

[65]  Yahui Zhang,et al.  Hardware Architecture and Algorithm Co-Design for Multi-Layer Photonic Neuromorphic Network with Excitable VCSELs-SA , 2020, 2020 Optical Fiber Communications Conference and Exhibition (OFC).

[66]  Yue Hao,et al.  All-optical neuromorphic XOR operation with inhibitory dynamics of a single photonic spiking neuron based on a VCSEL-SA. , 2020, Optics letters.

[67]  Sylvain Barbay,et al.  Photonic Computing With Single and Coupled Spiking Micropillar Lasers , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[68]  Minoru Yamada,et al.  A theoretical analysis of self-sustained pulsation phenomena in narrow-stripe semiconductor lasers , 1993 .

[69]  Peng Lin,et al.  Fully memristive neural networks for pattern classification with unsupervised learning , 2018 .

[70]  Yue Hao,et al.  Photonic Generation of Neuron-Like Dynamics Using VCSELs Subject to Double Polarized Optical Injection , 2018, Journal of Lightwave Technology.

[71]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[72]  C. David Wright,et al.  In-memory computing on a photonic platform , 2018, Science Advances.