Machine Learning With Neuromorphic Photonics

Neuromorphic photonics has experienced a recent surge of interest over the last few years, promising orders of magnitude improvements in both speed and energy efficiency over digital electronics. This paper provides a tutorial overview of neuromorphic photonic systems and their application to optimization and machine learning problems. We discuss the physical advantages of photonic processing systems, and we describe underlying device models that allow practical systems to be constructed. We also describe several real-world applications for control and deep learning inference. Finally, we discuss scalability in the context of designing a full-scale neuromorphic photonic processing system, considering aspects such as signal integrity, noise, and hardware fabrication platforms. The paper is intended for a wide audience and teaches how theory, research, and device concepts from neuromorphic photonics could be applied in practical machine learning systems.

[1]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[2]  Yongqiang Wang,et al.  An investigation of deep neural networks for noise robust speech recognition , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  M. Mitchell Waldrop,et al.  The chips are down for Moore’s law , 2016, Nature.

[4]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[5]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[6]  Di Liang,et al.  Recent progress in lasers on silicon , 2010 .

[7]  M. Pickett,et al.  A scalable neuristor built with Mott memristors. , 2013, Nature materials.

[8]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[9]  G. Stein,et al.  Respect the unstable , 2003 .

[10]  Yoshua Bengio,et al.  BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 , 2016, ArXiv.

[11]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  B. Romeira,et al.  Regenerative memory in time-delayed neuromorphic photonic resonators , 2016, Scientific Reports.

[13]  Trevor Bekolay,et al.  Nengo: a Python tool for building large-scale functional brain models , 2014, Front. Neuroinform..

[14]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[15]  Chen Sun,et al.  Monolithic silicon-photonic platforms in state-of-the-art CMOS SOI processes [Invited]. , 2018, Optics express.

[16]  Bhavin J. Shastri,et al.  Neuromorphic Photonic Integrated Circuits , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[17]  Paul R. Prucnal,et al.  Recent progress in semiconductor excitable lasers for photonic spike processing , 2016 .

[18]  François Pêcheux,et al.  VHDL-AMS and Verilog-AMS as alternative hardware description languages for efficient modeling of multidiscipline systems , 2005, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[19]  Quoc V. Le,et al.  Neural Architecture Search with Reinforcement Learning , 2016, ICLR.

[20]  M Radziunas,et al.  Excitability of a semiconductor laser by a two-mode homoclinic bifurcation. , 2001, Physical review letters.

[21]  Joshua Robertson,et al.  Controlled Propagation of Spiking Dynamics in Vertical-Cavity Surface-Emitting Lasers: Towards Neuromorphic Photonic Networks , 2017, IEEE Journal of Selected Topics in Quantum Electronics.

[22]  Sae Woo Nam,et al.  Superconducting optoelectronic circuits for neuromorphic computing , 2016, ArXiv.

[23]  Joel Emer,et al.  Eyeriss: an Energy-efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks Accessed Terms of Use , 2022 .

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Paul R. Prucnal,et al.  A TeraMAC Neuromorphic Photonic Processor , 2018, 2018 IEEE Photonics Conference (IPC).

[26]  Paul R. Prucnal,et al.  An integrated analog O/E/O link for multi-channel laser neurons , 2016 .

[27]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[28]  P. Prucnal,et al.  NEUROMORPHIC PHOTONICS , 2017 .

[29]  Paul R. Prucnal,et al.  Progress in neuromorphic photonics , 2017 .

[30]  Bowen Wang,et al.  Schematic driven simulation and layout of complex photonic IC's , 2016, 2016 IEEE 13th International Conference on Group IV Photonics (GFP).

[31]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[32]  Dharmendra S. Modha,et al.  Backpropagation for Energy-Efficient Neuromorphic Computing , 2015, NIPS.

[33]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[34]  Bernard Brezzo,et al.  TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[35]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[36]  Paul R. Prucnal,et al.  Microring Weight Banks , 2016, IEEE Journal of Selected Topics in Quantum Electronics.

[37]  Serge Massar,et al.  Fully analogue photonic reservoir computer , 2016, Scientific Reports.

[38]  A. Biberman,et al.  An ultralow power athermal silicon modulator , 2014, Nature Communications.

[39]  Geert Morthier,et al.  Experimental demonstration of reservoir computing on a silicon photonics chip , 2014, Nature Communications.

[40]  Lukas Chrostowski,et al.  Silicon Photonics Circuit Design: Methods, Tools and Challenges , 2018 .

[41]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[42]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[43]  B. Schrauwen,et al.  Cascadable excitability in microrings. , 2012, Optics express.

[44]  Ian O'Connor,et al.  Design and behavioral modeling tools for optical network-on-chip , 2004, Proceedings Design, Automation and Test in Europe Conference and Exhibition.

[45]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[46]  Jun Wang,et al.  Two neural network approaches to model predictive control , 2008, 2008 American Control Conference.

[47]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[48]  Indranil Chakraborty,et al.  Photonic In-Memory Computing Primitive for Spiking Neural Networks Using Phase-Change Materials , 2019, Physical Review Applied.

[49]  John E. Bowers,et al.  Quantum dot lasers for silicon photonics [Invited] , 2015 .

[50]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[51]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[52]  John Bowers,et al.  Photonic Integration With Epitaxial III–V on Silicon , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[53]  John R McDonnell,et al.  Training Neural Networks with Weight Constraints , 1993 .

[54]  Paul R. Prucnal,et al.  Temporal Dynamics of an Integrated Laser Neuron , 2018, 2018 IEEE Photonics Conference (IPC).

[55]  Yasuhiko Arakawa,et al.  Quantum dot lasers for silicon photonics , 2016, 2016 21st OptoElectronics and Communications Conference (OECC) held jointly with 2016 International Conference on Photonics in Switching (PS).

[56]  Terrence C. Stewart,et al.  Large-Scale Synthesis of Functional Spiking Neural Circuits , 2014, Proceedings of the IEEE.

[57]  Chen Sun,et al.  Addressing link-level design tradeoffs for integrated photonic interconnects , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[58]  Isabel De Sousa,et al.  The future of packaging with silicon photonics , 2016 .

[59]  Andrew S. Cassidy,et al.  Convolutional networks for fast, energy-efficient neuromorphic computing , 2016, Proceedings of the National Academy of Sciences.

[60]  Dirk Englund,et al.  Deep learning with coherent nanophotonic circuits , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[61]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[62]  Stephen P. Boyd,et al.  Fast Model Predictive Control Using Online Optimization , 2010, IEEE Transactions on Control Systems Technology.

[63]  G. Lo,et al.  Novel integration technique for silicon/III-V hybrid laser. , 2014, Optics express.

[64]  Thomas Ferreira de Lima,et al.  Multi-channel control for microring weight banks. , 2016, Optics express.

[65]  Sylvain Barbay,et al.  Excitability in a semiconductor laser with saturable absorber. , 2011, Optics letters.

[66]  Taghi M. Khoshgoftaar,et al.  Deep learning applications and challenges in big data analytics , 2015, Journal of Big Data.

[67]  Paul R. Prucnal,et al.  Spike processing with a graphene excitable laser , 2016, Scientific Reports.

[68]  Alessandro Aimar,et al.  NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[69]  Paul R Prucnal,et al.  Two-pole microring weight banks. , 2018, Optics letters.

[70]  Paul R. Prucnal,et al.  Broadcast and Weight: An Integrated Network For Scalable Photonic Spike Processing , 2014, Journal of Lightwave Technology.

[71]  Huug de Waardt,et al.  All fiber-optic neural network using coupled SOA based ring lasers , 2002, IEEE Trans. Neural Networks.

[72]  P. R. Prucnal,et al.  A Leaky Integrate-and-Fire Laser Neuron for Ultrafast Cognitive Computing , 2013, IEEE Journal of Selected Topics in Quantum Electronics.

[73]  Chong Wang,et al.  Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin , 2015, ICML.

[74]  Eugene M. Izhikevich,et al.  Neural excitability, Spiking and bursting , 2000, Int. J. Bifurc. Chaos.

[75]  David A. B. Miller.  Attojoule Optoelectronics for Low-Energy Information Processing and Communications , 2017, Journal of Lightwave Technology.

[76]  David A. B. Miller,et al.  Device Requirements for Optical Interconnects to Silicon Chips , 2009, Proceedings of the IEEE.

[77]  Paul R Prucnal,et al.  SIMPEL: circuit model for photonic spike processing laser neurons. , 2014, Optics express.

[78]  Daniel Brunner,et al.  Parallel photonic information processing at gigabyte per second data rates using transient states , 2013, Nature Communications.

[79]  Jan M. Maciejowski,et al.  Predictive control : with constraints , 2002 .

[80]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[81]  Philip Y. Ma,et al.  Feedback control for microring weight banks. , 2018, Optics express.

[82]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[83]  Shanhui Fan,et al.  Training of Photonic Neural Networks through In Situ Backpropagation , 2018, 2019 Conference on Lasers and Electro-Optics (CLEO).

[84]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[85]  Cristina Masoller,et al.  Unveiling the complex organization of recurrent patterns in spiking dynamical systems , 2014, Scientific Reports.

[86]  Paul R Prucnal,et al.  A high performance photonic pulse processing device. , 2009, Optics express.

[87]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[88]  Rajeev J Ram,et al.  Integrating photonics with silicon nanoelectronics for the next generation of systems on a chip , 2018, Nature.

[89]  J. Alspector,et al.  A Cascadable Neural Network Chip Set With On-chip Learning Using Noise And Gain Annealing , 1992, 1992 Proceedings of the IEEE Custom Integrated Circuits Conference.

[90]  M. Lipson,et al.  Battery-operated integrated frequency comb generator , 2018, Nature.

[91]  Yue Hao,et al.  Polarization-resolved and polarization- multiplexed spike encoding properties in photonic neuron based on VCSEL-SA , 2018, Scientific Reports.

[92]  Masaya Notomi,et al.  Ultracompact O-E-O converter based on fF-capacitance nanophotonic integration , 2018, 2018 Conference on Lasers and Electro-Optics (CLEO).

[93]  Paul R. Prucnal,et al.  Silicon Photonic Modulator Neuron , 2018, Physical Review Applied.

[94]  John J. Paulos,et al.  The Effects of Precision Constraints in a Backpropagation Learning Network , 1990, Neural Computation.

[95]  Ellen Zhou,et al.  Neuromorphic photonic networks using silicon photonic weight banks , 2017, Scientific Reports.

[96]  Qianfan Xu,et al.  Silicon microring resonators with 1.5-μm radius , 2008 .

[97]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[98]  Volker J. Sorger,et al.  Sub 1-volt graphene-based plasmonic electroabsorption modulator on silicon , 2017 .

[99]  R Kuszelewicz,et al.  Relative refractory period in an excitable semiconductor laser. , 2014, Physical review letters.

[100]  Jens H. Schmid,et al.  Roadmap on silicon photonics , 2016 .

[101]  S. Koehl,et al.  Development of CMOS-Compatible Integrated Silicon Photonics Devices , 2006, IEEE Journal of Selected Topics in Quantum Electronics.

[102]  Robert W. Keyes,et al.  Optical Logic-in the Light of Computer Technology , 1985 .

[103]  Eric C. Kerrigan,et al.  An FPGA implementation of a sparse quadratic programming solver for constrained predictive control , 2011, FPGA '11.

[104]  Sae Woo Nam,et al.  All-silicon light-emitting diodes waveguide-integrated with superconducting single-photon detectors. , 2017, Applied physics letters.

[105]  Terrence C. Stewart,et al.  A Technical Overview of the Neural Engineering Framework , 2012 .

[106]  D. Thomson,et al.  Hybrid III--V on Silicon Lasers for Photonic Integrated Circuits on Silicon , 2014 .

[107]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[108]  Joseph W. Goodman,et al.  Fan-in and Fan-out with Optical Interconnections , 1985 .