Hybrid training of optical neural networks

Optical neural networks are emerging as a promising type of machine learning hardware capable of energy-efficient, parallel computation. Today’s optical neural networks are mainly developed to perform optical inference after in silico training on digital simulators. However, various physical imperfections that cannot be accurately modelled may lead to the notorious “reality gap” between the digital simulator and the physical system. To address this challenge, we demonstrate hybrid training of optical neural networks where the weight matrix is trained with neuron activation functions computed optically via forward propagation through the network. We examine the efficacy of hybrid training with three different networks: an optical linear classifier, a hybrid opto-electronic network, and a complex-valued optical network. We perform a comparative study to in silico training, and our results show that hybrid training is robust against different kinds of static noise. Our platform-agnostic hybrid training scheme can be applied to a wide variety of optical neural networks, and this work paves the way towards advanced all-optical training in machine intelligence.

[1]  Johannes Schemmel,et al.  Surrogate gradients for analog neuromorphic computing , 2020, Proceedings of the National Academy of Sciences.

[2]  D. Englund,et al.  Hardware error correction for programmable photonics , 2021, ArXiv.

[3]  A. Ozcan,et al.  Spectrally encoded single-pixel machine vision using diffractive networks , 2021, Science Advances.

[4]  A. Lvovsky,et al.  Backpropagation through nonlinear units for the all-optical training of neural networks , 2019, Photonics Research.

[5]  David A. B. Miller,et al.  Setting up meshes of interferometers - reversed local light interference method , 2017 .

[6]  Danna Zhou,et al.  d. , 1840, Microbial pathogenesis.

[7]  Thomas D. Barrett,et al.  Fully reconfigurable coherent optical vector-matrix multiplication. , 2020, Optics letters.

[8]  A. Boes,et al.  11 TOPS photonic convolutional accelerator for optical neural networks , 2021, Nature.

[9]  D Psaltis,et al.  Optical implementation of the Hopfield model. , 1985, Applied optics.

[10]  Demetri Psaltis,et al.  Optical Neural Computers , 1987, Topical Meeting on Optical Computing.

[11]  Logan G. Wright,et al.  An optical neural network using less than 1 photon per multiplication , 2021, Nature Communications.

[12]  Yue Jiang,et al.  All-optical neural network with nonlinear activation functions , 2019, Optica.

[13]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[14]  Qionghai Dai,et al.  Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit , 2020, Nature Photonics.

[15]  Qing Wu,et al.  Efficient and self-adaptive in-situ learning in multilayer memristor neural networks , 2018, Nature Communications.

[16]  Qing Wu,et al.  In situ training of feed-forward and recurrent convolutional memristor networks , 2019, Nature Machine Intelligence.

[17]  Dmitry K. Polyushkin,et al.  Ultrafast machine vision with 2D material neural network image sensors , 2020, Nature.

[18]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[19]  R. Gerchberg.  A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[20]  V. Sorger,et al.  Massively parallel amplitude-only Fourier neural network , 2020, AI and Optical Data Sciences II.

[21]  P. Alam.  ‘T’ , 2021, Composites Engineering: An A–Z Guide.

[22]  Yi Luo,et al.  All-optical machine learning using diffractive deep neural networks , 2018, Science.

[23]  Bhavin J. Shastri,et al.  Photonics for artificial intelligence and neuromorphic computing , 2020, Nature Photonics.

[24]  Gordon Wetzstein,et al.  Inference in artificial intelligence with deep optics and photonics , 2020, Nature.

[25]  David A. B. Miller,et al.  Parallel Programming of an Arbitrary Feedforward Photonic Network , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[26]  B. Gupta,et al.  Learning on an analog VLSI neural network chip , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[27]  P. Alam.  ‘Z’ , 2021, Composites Engineering: An A–Z Guide.

[28]  Dirk Englund,et al.  Deep learning with coherent nanophotonic circuits , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[29]  David R. So,et al.  Carbon Emissions and Large Neural Network Training , 2021, ArXiv.

[30]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[31]  L. A. González,et al.  Pixelated phase computer holograms for the accurate encoding of scalar complex fields. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[32]  Laurent Larger,et al.  Reinforcement Learning in a large scale photonic Recurrent Neural Network , 2017, Optica.

[33]  W H Lee,et al.  Binary computer-generated holograms. , 1979, Applied optics.

[34]  Ryan Hamerly,et al.  Large-Scale Optical Neural Networks based on Photoelectric Multiplication , 2018, Physical Review X.

[35]  G. Lo,et al.  An optical neural chip for implementing complex-valued neural network , 2021, Nature Communications.

[36]  David A. B. Miller,et al.  Perfect optics with imperfect components , 2015 .

[37]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[38]  Martin M. Stein,et al.  Deep physical neural networks trained with backpropagation , 2022, Nature.

[39]  Xuan Li,et al.  Parallel convolutional processing using an integrated photonic tensor core , 2021, Nature.

[40]  Demetri Psaltis,et al.  Competitive photonic neural networks , 2021, Nature Photonics.

[41]  Steven R. Skinner,et al.  Reinforcement and backpropagation training for an optical neural network using self-lensing effects , 2000, IEEE Trans. Neural Networks Learn. Syst..

[42]  David A. B. Miller.  Attojoule Optoelectronics for Low-Energy Information Processing and Communications , 2017, Journal of Lightwave Technology.

[43]  P. Alam.  ‘A’ , 2021, Composites Engineering: An A–Z Guide.

[44]  P. Alam,et al.  R , 1823, The Herodotus Encyclopedia.