Photonics for artificial intelligence and neuromorphic computing

Research in photonic computing has flourished due to the proliferation of optoelectronic components on photonic integration platforms. Photonic integrated circuits have enabled ultrafast artificial neural networks, providing a framework for a new class of information processing machines. Algorithms running on such hardware have the potential to address the growing demand for machine learning and artificial intelligence, in areas such as medical diagnosis, telecommunications, and high-performance and scientific computing. In parallel, the development of neuromorphic electronics has highlighted challenges in that domain, in particular, related to processor latency. Neuromorphic photonics offers sub-nanosecond latencies, providing a complementary opportunity to extend the domain of artificial intelligence. Here, we review recent advances in integrated photonic neuromorphic systems, discuss current and future challenges, and outline the advances in science and technology needed to meet those challenges.

[1]  Xingyuan Xu,et al.  Photonic perceptron based on a Kerr microcomb for high-speed, scalable, optical neural networks , 2020, 2020 International Topical Meeting on Microwave Photonics (MWP).

[2]  F.J. Leonberger,et al.  Optical interconnections for VLSI systems , 1984, Proceedings of the IEEE.

[3]  Matěj Hejda,et al.  Ultrafast optical integration and pattern classification for neuromorphic photonics based on spiking VCSEL neurons , 2020, Scientific Reports.

[4]  Paul R. Prucnal,et al.  Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs) , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[5]  C. Huyghebaert,et al.  Graphene–silicon phase modulators with gigahertz bandwidth , 2017, Nature Photonics.

[6]  Tarek El-Ghazawi,et al.  ITO-based electro-absorption modulator for photonic neural activation function , 2019, APL Materials.

[7]  Vladimir Liberman,et al.  Broadband transparent optical phase change materials for high-performance nonvolatile photonics , 2018, Nature Communications.

[8]  Harish Bhaskaran,et al.  On-chip photonic synapse , 2017, Science Advances.

[9]  J. O'Brien,et al.  Universal linear optics , 2015, Science.

[10]  P. Dumon,et al.  Silicon microring resonators , 2012 .

[11]  Michal Lipson,et al.  Nanophotonic lithium niobate electro-optic modulators. , 2017, Optics express.

[12]  Harish Bhaskaran,et al.  Integrated all-photonic non-volatile multi-level memory , 2015, Nature Photonics.

[13]  R. Soref,et al.  Electrooptical effects in silicon , 1987 .

[14]  Paul R. Prucnal,et al.  Temporal Information Processing With an Integrated Laser Neuron , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[15]  Qing Wu,et al.  Long short-term memory networks in memristor crossbar arrays , 2018, Nature Machine Intelligence.

[16]  Kazuhiro Terao,et al.  Machine learning at the energy and intensity frontiers of particle physics , 2018, Nature.

[17]  Yue Jiang,et al.  All-optical neural network with nonlinear activation functions , 2019, Optica.

[18]  Shuiying Xiang,et al.  STDP-Based Unsupervised Spike Pattern Learning in a Photonic Spiking Neural Network With VCSELs and VCSOAs , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[19]  Michal Lipson,et al.  Photonic-chip-based frequency combs , 2019, Nature Photonics.

[20]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[21]  Mark Peterson,et al.  Luxtera's Silicon Photonics Platform for Transceiver Manufacturing , 2014 .

[22]  Volker J. Sorger,et al.  Electronic Bottleneck Suppression in Next-generation Networks with Integrated Photonic Digital-to-analog Converters , 2019 .

[23]  Zhiping Zhou,et al.  On-chip light sources for silicon photonics , 2015, Light: Science & Applications.

[24]  Chaoran Huang,et al.  Lateral bipolar junction transistor on a silicon photonics platform. , 2020, Optics express.

[25]  P. Prucnal,et al.  NEUROMORPHIC PHOTONICS , 2017 .

[26]  J. Feldmann,et al.  All-optical spiking neurosynaptic networks with self-learning capabilities , 2019, Nature.

[27]  T. Kippenberg,et al.  Microresonator based optical frequency combs , 2012, 2012 Conference on Lasers and Electro-Optics (CLEO).

[28]  D Psaltis,et al.  Optical information processing based on an associative-memory model of neural nets with thresholding and feedback. , 1985, Optics letters.

[29]  Song Han,et al.  Fast inference of deep neural networks in FPGAs for particle physics , 2018, Journal of Instrumentation.

[30]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[31]  Yi Luo,et al.  All-optical machine learning using diffractive deep neural networks , 2018, Science.

[32]  Kazuyuki Aihara,et al.  A fully programmable 100-spin coherent Ising machine with all-to-all connections , 2016, Science.

[33]  Daniel Brunner,et al.  Parallel photonic information processing at gigabyte per second data rates using transient states , 2013, Nature Communications.

[34]  Paul R. Prucnal,et al.  Demonstration of Multivariate Photonics: Blind Dimensionality Reduction With Integrated Photonics , 2019, Journal of Lightwave Technology.

[35]  Dirk Englund,et al.  Heuristic recurrent algorithms for photonic Ising machines , 2020, Nature Communications.

[36]  Javier Ayala,et al.  300-mm Monolithic Silicon Photonics Foundry Technology , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[37]  Paul R Prucnal,et al.  Ultrafast all-optical implementation of a leaky integrate-and-fire neuron. , 2011, Optics express.

[38]  K Vyrsokinos,et al.  An all-optical neuron with sigmoid activation function. , 2019, Optics express.

[39]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[40]  H. Rong,et al.  A 128 Gb/s PAM4 Silicon Microring Modulator With Integrated Thermo-Optic Resonance Tuning , 2019, Journal of Lightwave Technology.

[41]  Joshua Robertson,et al.  Controlled inhibition of spiking dynamics in VCSELs for neuromorphic photonics: theory and experiments. , 2017, Optics letters.

[42]  Damien Rontani,et al.  Human action recognition with a large-scale brain-inspired photonic computer , 2019, Nat. Mach. Intell..

[43]  Terrence C. Stewart,et al.  Large-Scale Synthesis of Functional Spiking Neural Circuits , 2014, Proceedings of the IEEE.

[44]  Salvador Balle,et al.  Excitability and optical pulse generation in semiconductor lasers driven by resonant tunneling diode photo-detectors. , 2013, Optics express.

[45]  B. Romeira,et al.  Regenerative memory in time-delayed neuromorphic photonic resonators , 2016, Scientific Reports.

[46]  E. Turner,et al.  HIGH‐FREQUENCY ELECTRO‐OPTIC COEFFICIENTS OF LITHIUM NIOBATE , 1966 .

[47]  Chen Sun,et al.  Monolithic silicon-photonic platforms in state-of-the-art CMOS SOI processes [Invited]. , 2018, Optics express.

[48]  Nathan Youngblood,et al.  Device‐Level Photonic Memories and Logic Applications Using Phase‐Change Materials , 2018, Advanced materials.

[49]  D.A.B. Miller,et al.  Rationale and challenges for optical interconnects to electronic chips , 2000, Proceedings of the IEEE.

[50]  L. Liu,et al.  High-performance hybrid silicon and lithium niobate Mach–Zehnder modulators for 100 Gbit s−1 and beyond , 2018, Nature Photonics.

[51]  Jingdong Luo,et al.  Trimming of high-Q-factor silicon ring resonators by electron beam bleaching. , 2012, Optics letters.

[52]  Till Acker,et al.  DNA methylation-based classification of central nervous system tumours , 2018, Nature.

[53]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[54]  N. Harris,et al.  Efficient, compact and low loss thermo-optic phase shifter in silicon. , 2014, Optics express.

[55]  Paul R. Prucnal,et al.  Photonic Implementation of Spike-Timing-Dependent Plasticity and Learning Algorithms of Biological Neural Systems , 2016, Journal of Lightwave Technology.

[56]  Alireza Samani,et al.  Silicon Photonic Segmented Modulator-Based Electro-Optic DAC for 100 Gb/s PAM-4 Generation , 2015, IEEE Photonics Technology Letters.

[57]  Paul R. Prucnal,et al.  Spike processing with a graphene excitable laser , 2016, Scientific Reports.

[58]  Reck,et al.  Experimental realization of any discrete unitary operator. , 1994, Physical review letters.

[59]  L. Chrostowski,et al.  Wavelength tuning and stabilization of microring-based filters using silicon in-resonator photoconductive heaters. , 2015, Optics express.

[60]  J. Danckaert,et al.  Solitary and coupled semiconductor ring lasers as optical spiking neurons. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[61]  Indranil Chakraborty,et al.  Toward Fast Neural Computing using All-Photonic Phase Change Spiking Neurons , 2018, Scientific Reports.

[62]  Bhavin J. Shastri,et al.  Neuromorphic Photonic Integrated Circuits , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[63]  Ping Tak Peter Tang,et al.  Sparse Coding by Spiking Neural Networks: Convergence Theory and Computational Results , 2017, ArXiv.

[64]  A. N. Tait,et al.  A superconducting thermal switch with ultrahigh impedance for interfacing superconductors to semiconductors , 2019, Nature Electronics.

[65]  Paul R. Prucnal,et al.  Continuous Calibration of Microring Weights for Analog Optical Networks , 2016, IEEE Photonics Technology Letters.

[66]  P. Winzer,et al.  Integrated lithium niobate electro-optic modulators operating at CMOS-compatible voltages , 2018, Nature.

[67]  Mike E. Davies,et al.  Benchmarks for progress in neuromorphic computing , 2019, Nature Machine Intelligence.

[68]  Masaya Notomi,et al.  Femtofarad optoelectronic integration demonstrating energy-saving signal conversion and nonlinear functions , 2019, Nature Photonics.

[69]  Paul R. Prucnal,et al.  Demonstration of Photonic Neural Network for Fiber Nonlinearity Compensation in Long-Haul Transmission Systems , 2020, 2020 Optical Fiber Communications Conference and Exhibition (OFC).

[70]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[71]  R Kuszelewicz,et al.  Relative refractory period in an excitable semiconductor laser. , 2014, Physical review letters.

[72]  Martin Rinard,et al.  Noise-Aware Dynamical System Compilation for Analog Devices with Legno , 2020, ASPLOS.

[73]  Lukas Chrostowski,et al.  Silicon Photonics Circuit Design: Methods, Tools and Challenges , 2018 .

[74]  Sae Woo Nam,et al.  Superconducting optoelectronic circuits for neuromorphic computing , 2016, ArXiv.

[75]  Vijay Vusirikala,et al.  Field and lab experimental demonstration of nonlinear impairment compensation using neural networks , 2019, Nature Communications.

[76]  Paul R. Prucnal,et al.  An integrated analog O/E/O link for multi-channel laser neurons , 2016 .

[77]  Vivienne Sze,et al.  Efficient Processing of Deep Neural Networks: A Tutorial and Survey , 2017, Proceedings of the IEEE.

[78]  Alan Y. Liu,et al.  Heterogeneous Silicon Photonic Integrated Circuits , 2016, Journal of Lightwave Technology.

[79]  Aleksandar Nesic,et al.  Hybrid integration of silicon photonics circuits and InP lasers by photonic wire bonding , 2018, Optica.

[80]  Robert W. Keyes,et al.  Optical Logic-in the Light of Computer Technology , 1985 .

[81]  Nicola Calabretta,et al.  Deep Neural Network Through an InP SOA-Based Photonic Integrated Cross-Connect , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[82]  R. Baets,et al.  Trimming of silicon ring resonator by electron beam induced compaction and strain. , 2008, Optics express.

[83]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[84]  Sae Woo Nam,et al.  All-silicon light-emitting diodes waveguide-integrated with superconducting single-photon detectors. , 2017, Applied physics letters.

[85]  Paul R. Prucnal,et al.  Photonic Multiply-Accumulate Operations for Neural Networks , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[86]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[87]  Kaushik Roy,et al.  Roadmap on emerging hardware and technology for machine learning , 2020, Nanotechnology.

[88]  C. David Wright,et al.  In-memory computing on a photonic platform , 2018, Science Advances.

[89]  Philip Y. Ma,et al.  Feedback control for microring weight banks. , 2018, Optics express.

[90]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[91]  Paul R Prucnal,et al.  A high performance photonic pulse processing device. , 2009, Optics express.

[92]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[93]  S. Pinna,et al.  3D integrated hybrid silicon laser , 2015, 2015 European Conference on Optical Communication (ECOC).

[94]  Ian A. D. Williamson,et al.  Training of Photonic Neural Networks through In Situ Backpropagation , 2019 .

[95]  Weiwen Zou,et al.  High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays. , 2019, Optics express.

[96]  Julian Kates-Harbeck,et al.  Predicting disruptive instabilities in controlled fusion plasmas through deep learning , 2019, Nature.

[97]  P. R. Prucnal,et al.  A Leaky Integrate-and-Fire Laser Neuron for Ultrafast Cognitive Computing , 2013, IEEE Journal of Selected Topics in Quantum Electronics.

[98]  Ivana Gasulla,et al.  Principles, fundamentals, and applications of programmable integrated photonics , 2020 .

[99]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[100]  Sae Woo Nam,et al.  Design, fabrication, and metrology of 10 × 100 multi-planar integrated photonic routing manifolds for neural networks , 2018, APL Photonics.

[101]  Dennis W Prather,et al.  110 GHz CMOS compatible thin film LiNbO3 modulator on silicon. , 2016, Optics express.

[102]  Behrad Gholipour,et al.  Amorphous Metal‐Sulphide Microfibers Enable Photonic Synapses for Brain‐Like Computing , 2015 .

[103]  Shanhui Fan,et al.  Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[104]  Paul R. Prucnal,et al.  Microring Weight Banks , 2016, IEEE Journal of Selected Topics in Quantum Electronics.

[105]  Geert Morthier,et al.  Experimental demonstration of reservoir computing on a silicon photonics chip , 2014, Nature Communications.

[106]  Rajeev J. Ram,et al.  Single-chip microprocessor that communicates directly using light , 2015, Nature.

[107]  Paul R. Prucnal,et al.  Machine Learning With Neuromorphic Photonics , 2019, Journal of Lightwave Technology.

[108]  Easwar Magesan,et al.  Machine Learning for Discriminating Quantum Measurement Trajectories and Improving Readout. , 2014, Physical review letters.

[109]  Milan M. Milosevic,et al.  Ion Implantation in Silicon for Trimming the Operating Wavelength of Ring Resonators , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[110]  Abu Sebastian,et al.  Tutorial: Brain-inspired computing using phase-change memory devices , 2018, Journal of Applied Physics.

[111]  Wim Bogaerts,et al.  Programmable Photonics: An Opportunity for an Accessible Large-Volume PIC Ecosystem , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[112]  Benjamin Schrauwen,et al.  Optoelectronic Reservoir Computing , 2011, Scientific Reports.

[113]  Dirk Englund,et al.  Deep learning with coherent nanophotonic circuits , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[114]  J W Goodman,et al.  Fully parallel, high-speed incoherent optical method for performing discrete Fourier transforms. , 1978, Optics letters.

[115]  Huug de Waardt,et al.  All fiber-optic neural network using coupled SOA based ring lasers , 2002, IEEE Trans. Neural Networks.

[116]  Christopher C. Tison,et al.  Linear programmable nanophotonic processors , 2018, Optica.

[117]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[118]  Paul R. Prucnal,et al.  Broadcast and Weight: An Integrated Network For Scalable Photonic Spike Processing , 2014, Journal of Lightwave Technology.

[119]  Paul R. Prucnal,et al.  Silicon Photonic Modulator Neuron , 2018, Physical Review Applied.

[120]  Tarek El-Ghazawi,et al.  Neuromorphic photonics with electro-absorption modulators. , 2018, Optics express.

[121]  Ellen Zhou,et al.  Neuromorphic photonic networks using silicon photonic weight banks , 2017, Scientific Reports.

[122]  Laurent Larger,et al.  Reinforcement Learning in a large scale photonic Recurrent Neural Network , 2017, Optica.

[123]  T. Kippenberg,et al.  Optical frequency comb generation from a monolithic microresonator , 2007, Nature.

[124]  Laurent Larger,et al.  Tutorial: Photonic Neural Networks in Delay Systems , 2018, Journal of Applied Physics.

[125]  M. Chagnon,et al.  Design, analysis, and transmission system performance of a 41 GHz silicon photonic modulator. , 2015, Optics express.

[126]  Serge Massar,et al.  All-optical Reservoir Computing , 2012, Optics express.

[127]  Sae Woo Nam,et al.  Superconducting optoelectronic loop neurons , 2019, Journal of Applied Physics.

[128]  Di Liang,et al.  Recent progress in lasers on silicon , 2010 .

[129]  Yue Tian,et al.  Pulse lead/lag timing detection for adaptive feedback and control based on optical spike-timing-dependent plasticity. , 2013, Optics letters.

[130]  Pritish Narayanan,et al.  Equivalent-accuracy accelerated neural-network training using analogue memory , 2018, Nature.

[131]  Mario Miscuglio,et al.  A Winograd-Based Integrated Photonics Accelerator for Convolutional Neural Networks , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[132]  I. Sagnes,et al.  Excitability and self-pulsing in a photonic crystal nanocavity , 2012 .

[133]  Joseph W. Goodman,et al.  Fan-in and Fan-out with Optical Interconnections , 1985 .

[134]  Wei Li,et al.  Electrically pumped continuous-wave III–V quantum dot lasers on silicon , 2016, Nature Photonics.