Efficient On-Chip Training of Optical Neural Networks Using Genetic Algorithm

Recent advances in silicon photonic chips have made huge progress in optical computing owing to their flexibility in the reconfiguration of various tasks. Its deployment of neural networks serves as an alternative for mitigating the rapidly increased demand for computing resources in electronic platforms. However, it remains a formidable challenge to train the online programmable optical neural networks efficiently, being restricted by the difficulty in obtaining gradient information on a physical device when executing a gradient descent algorithm. Here, we experimentally demonstrate an efficient, physics-agnostic, and closed-loop protocol for training optical neural networks on chip. A gradient-free algorithm, that is, the genetic algorithm, is adopted. The protocol is on-chip implementable, physical agnostic (no need to rely on characterization and offline modeling), and gradientfree. The protocol works for various types of chip structures and is especially helpful to those that cannot be analytically decomposed and characterized. We confirm its viability using several practical tasks, including the crossbar switch and the Iris classification. Finally, by comparing our physics-agonistic and gradient-free method to the off-chip and gradient-based training methods, we demonstrate the robustness of our system to perturbations such as imperfect phase implementation and photodetection noise. Optical processors with gradient-free genetic algorithms have broad application potentials in pattern recognition, reinforcement learning, quantum computing, and realistic applications (such as facial recognition, natural language processing, and autonomous vehicles).

[1]  Anthony Laing,et al.  Generation and sampling of quantum states of light in a silicon chip , 2018, Nature Physics.

[2]  Humphreys,et al.  An Optimal Design for Universal Multiport Interferometers , 2016, 1603.08788.

[3]  Barak A. Pearlmutter,et al.  Automatic differentiation in machine learning: a survey , 2015, J. Mach. Learn. Res..

[4]  Philip Walther,et al.  Experimental boson sampling , 2012, Nature Photonics.

[5]  Ke Li,et al.  Multipurpose silicon photonics signal processor core , 2017, Nature Communications.

[6]  Mario Miscuglio,et al.  A Winograd-Based Integrated Photonics Accelerator for Convolutional Neural Networks , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[7]  Gregory R. Steinbrecher,et al.  Quantum transport simulations in a programmable nanophotonic processor , 2015, Nature Photonics.

[8]  J. O'Brien,et al.  Universal linear optics , 2015, Science.

[9]  Xu Han,et al.  Efficient training and design of photonic neural network through neuroevolution , 2019, Optics express.

[10]  M. Thompson,et al.  Generating, manipulating and measuring entanglement and mixture with a reconfigurable photonic circuit , 2012 .

[11]  Wim Bogaerts,et al.  Demonstration of a 4 × 4-port universal linear circuit , 2016 .

[12]  Xu Wang,et al.  Self-learning photonic signal processor with an optical neural network chip , 2019, 1902.07318.

[13]  Alejandro Perdomo-Ortiz,et al.  A generative modeling approach for benchmarking and training shallow quantum circuits , 2018, npj Quantum Information.

[14]  Benchmarking integrated photonic architectures , 2017, 1705.09211.

[15]  Sasikanth Manipatruni,et al.  Design of optical neural networks with component imprecisions , 2019, Optics express.

[16]  David A. B. Miller,et al.  Self-configuring universal linear optical component [Invited] , 2013, 1303.4602.

[17]  G. Lo,et al.  An optical neural chip for implementing complex-valued neural network , 2021, Nature Communications.

[18]  D Zhu,et al.  Training of quantum circuits on a hybrid quantum computer , 2018, Science Advances.

[19]  Patricia Yang Liu,et al.  Microring resonator-assisted Fourier transform spectrometer with enhanced resolution and large bandwidth in single chip solution , 2019, Nature Communications.

[20]  Harish Bhaskaran,et al.  On-chip photonic synapse , 2017, Science Advances.

[21]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[22]  Shanhui Fan,et al.  Training of Photonic Neural Networks through In Situ Backpropagation , 2018, 2019 Conference on Lasers and Electro-Optics (CLEO).

[23]  Tobias J. Osborne,et al.  Training deep quantum neural networks , 2020, Nature Communications.

[24]  Andrea Melloni,et al.  Canceling Thermal Cross-Talk Effects in Photonic Integrated Circuits , 2019, Journal of Lightwave Technology.

[25]  David A. B. Miller,et al.  Meshing optics with applications , 2017, Nature Photonics.

[26]  Reck,et al.  Experimental realization of any discrete unitary operator. , 1994, Physical review letters.

[27]  Hariharan Narayanan,et al.  Efficient Sampling from Time-Varying Log-Concave Distributions , 2013, J. Mach. Learn. Res..

[28]  Nicolò Spagnolo,et al.  Benchmarking integrated linear-optical architectures for quantum information processing , 2017, Scientific Reports.

[29]  Ivana Gasulla,et al.  Programmable multifunctional integrated nanophotonics , 2018, Nanophotonics.

[30]  Guo-Qiang Lo,et al.  50-Gb/s silicon optical modulator with traveling-wave electrodes. , 2013, Optics express.

[31]  J L O'Brien,et al.  60  dB high-extinction auto-configured Mach-Zehnder interferometer. , 2016, Optics letters.

[32]  Risto Miikkulainen,et al.  Designing neural networks through neuroevolution , 2019, Nat. Mach. Intell..

[33]  Daniel Pérez,et al.  Silicon Photonics Rectangular Universal Interferometer , 2017 .

[34]  Bhavin J. Shastri,et al.  Neuromorphic photonic networks using silicon photonic weight banks , 2016, Scientific Reports.

[35]  Graham D. Marshall,et al.  Large-scale silicon quantum photonics implementing arbitrary two-qubit processing , 2018, Nature Photonics.

[36]  Dirk Englund,et al.  Deep learning with coherent nanophotonic circuits , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[37]  Nicolò Spagnolo,et al.  Learning an unknown transformation via a genetic approach , 2016, Scientific Reports.

[38]  Chris G. H. Roeloffzen,et al.  Programmable photonic signal processor chip for radiofrequency applications , 2015, 1505.00094.

[39]  Kenneth O. Stanley,et al.  Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning , 2017, ArXiv.