Optical coherent dot-product chip for sophisticated deep learning regression

Optical implementations of neural networks (ONNs) herald the next-generation high-speed and energy-efficient deep learning computing by harnessing the technical advantages of large bandwidth and high parallelism of optics. However, due to the problems of the incomplete numerical domain, limited hardware scale, or inadequate numerical accuracy, the majority of existing ONNs were studied for basic classification tasks. Given that regression is a fundamental form of deep learning and accounts for a large part of current artificial intelligence applications, it is necessary to master deep learning regression for further development and deployment of ONNs. Here, we demonstrate a silicon-based optical coherent dot-product chip (OCDC) capable of completing deep learning regression tasks. The OCDC adopts optical fields to carry out operations in the complete real-value domain instead of in only the positive domain. Via reusing, a single chip conducts matrix multiplications and convolutions in neural networks of any complexity. Also, hardware deviations are compensated via in-situ backpropagation control provided the simplicity of chip architecture. Therefore, the OCDC meets the requirements for sophisticated regression tasks and we successfully demonstrate a representative neural network, the AUTOMAP (a cutting-edge neural network model for image reconstruction). The quality of reconstructed images by the OCDC and a 32-bit digital computer is comparable. To the best of our knowledge, there is no precedent of performing such state-of-the-art regression tasks on ONN chips. It is anticipated that the OCDC can promote the novel accomplishment of ONNs in modern AI applications including autonomous driving, natural language processing, and scientific study. An optical coherent chip completes state-of-the-art image reconstruction tasks with 32-bit computer comparable image quality, showing potential in conquering sophisticated deep learning regression tasks.

[1]  G. Lo,et al.  An optical neural chip for implementing complex-valued neural network , 2021, Nature Communications.

[2]  Shanhui Fan,et al.  Training of Photonic Neural Networks through In Situ Backpropagation , 2018, 2019 Conference on Lasers and Electro-Optics (CLEO).

[3]  Xuan Li,et al.  Parallel convolutional processing using an integrated photonic tensor core , 2021, Nature.

[4]  Ryan Hamerly,et al.  Large-Scale Optical Neural Networks based on Photoelectric Multiplication , 2018, Physical Review X.

[5]  Ellen Zhou,et al.  Neuromorphic photonic networks using silicon photonic weight banks , 2017, Scientific Reports.

[6]  L. Liu,et al.  High-performance hybrid silicon and lithium niobate Mach–Zehnder modulators for 100 Gbit s−1 and beyond , 2018, Nature Photonics.

[7]  Zuyuan He,et al.  Programmable matrix operation with reconfigurable time-wavelength plane manipulation and dispersed time delay. , 2019, Optics express.

[8]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[9]  H. Tang,et al.  Broadband nanoelectromechanical phase shifting of light on a chip , 2013, 1312.2454.

[10]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[11]  Rajeev J. Ram,et al.  Single-chip microprocessor that communicates directly using light , 2015, Nature.

[12]  Yi Yang,et al.  Nanophotonic particle simulation and inverse design using artificial neural networks , 2018, Science Advances.

[13]  Weiwen Zou,et al.  Optical patching scheme for optical convolutional neural networks based on wavelength-division multiplexing and optical delay lines. , 2020, Optics letters.

[14]  M. Mitchell Waldrop,et al.  The chips are down for Moore’s law , 2016, Nature.

[15]  John Tran,et al.  cuDNN: Efficient Primitives for Deep Learning , 2014, ArXiv.

[16]  Sorin Grigorescu,et al.  A Survey of Deep Learning Techniques for Autonomous Driving , 2020, J. Field Robotics.

[17]  Qionghai Dai,et al.  Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit , 2020, Nature Photonics.

[18]  Guy Lever,et al.  Human-level performance in 3D multiplayer games with population-based reinforcement learning , 2018, Science.

[19]  Anastasios Tefas,et al.  Femtojoule per MAC Neuromorphic Photonics: An Energy and Technology Roadmap , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[20]  P. Winzer,et al.  Integrated lithium niobate electro-optic modulators operating at CMOS-compatible voltages , 2018, Nature.

[21]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[22]  C. Wright,et al.  Photonics for artificial intelligence and neuromorphic computing , 2020, ArXiv.

[23]  Sae Woo Nam,et al.  Design, fabrication, and metrology of 10 × 100 multi-planar integrated photonic routing manifolds for neural networks , 2018, APL Photonics.

[24]  Sasikanth Manipatruni,et al.  Design of optical neural networks with component imprecisions , 2019, Optics express.

[25]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.

[26]  Mike Preuss,et al.  Planning chemical syntheses with deep neural networks and symbolic AI , 2017, Nature.

[27]  Zongfu Yu,et al.  Training Deep Neural Networks for the Inverse Design of Nanophotonic Structures , 2017, 2019 Conference on Lasers and Electro-Optics (CLEO).

[28]  Sae Woo Nam,et al.  Multi-planar amorphous silicon photonics with compact interplanar couplers, cross talk mitigation, and low crossing loss , 2017 .

[29]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[30]  P. Prucnal,et al.  Primer on silicon neuromorphic photonic processors: architecture and compiler , 2020 .

[31]  Demis Hassabis,et al.  Improved protein structure prediction using potentials from deep learning , 2020, Nature.

[32]  Bhavin J. Shastri,et al.  Neuromorphic photonic networks using silicon photonic weight banks , 2016, Scientific Reports.

[33]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[34]  Volker L. Deringer,et al.  Origins of structural and electronic transitions in disordered silicon , 2021, Nature.

[35]  Bruce R. Rosen,et al.  MGH–USC Human Connectome Project datasets with ultra-high b-value diffusion MRI , 2016, NeuroImage.

[36]  Allen G. Lindgren,et al.  The Inverse Discrete Radon Transform with Applications to Tomographic Imaging Using Projection Data , 1981 .

[37]  Lei Yu,et al.  Deep-learning-powered photonic analog-to-digital conversion , 2019, Light, science & applications.

[38]  Xuanjing Huang,et al.  Recurrent Neural Network for Text Classification with Multi-Task Learning , 2016, IJCAI.

[39]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[40]  Paul Strauss,et al.  Magnetic Resonance Imaging Physical Principles And Sequence Design , 2016 .

[41]  Yibo Zhang,et al.  Phase recovery and holographic image reconstruction using deep learning in neural networks , 2017, Light: Science & Applications.

[42]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[43]  Paul R. Prucnal,et al.  Photonic Multiply-Accumulate Operations for Neural Networks , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[44]  Weiwen Zou,et al.  High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays. , 2019, Optics express.

[45]  P. Alam.  ‘T’ , 2021, Composites Engineering: An A–Z Guide.

[46]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[47]  Shaofu Xu,et al.  Optical Convolutional Neural Network With WDM-Based Optical Patching and Microring Weighting Banks , 2021, IEEE Photonics Technology Letters.

[48]  Anastasios Tefas,et al.  Neuromorphic Photonics With Coherent Linear Neurons Using Dual-IQ Modulation Cells , 2020, Journal of Lightwave Technology.

[49]  Gabriel M. Rebeiz,et al.  Bonded thin film lithium niobate modulator on a silicon photonics platform exceeding 100 GHz 3-dB electrical modulation bandwidth. , 2018, Optics express.

[50]  Mordechai Segev,et al.  Highlighting photonics: looking into the next decade , 2021, eLight.

[51]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[52]  Rajeev J Ram,et al.  Integrating photonics with silicon nanoelectronics for the next generation of systems on a chip , 2018, Nature.

[53]  Dirk Englund,et al.  Deep learning with coherent nanophotonic circuits , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[54]  Dave Bergeron,et al.  More than Moore , 2008, CICC.

[55]  A. Boes,et al.  11 TOPS photonic convolutional accelerator for optical neural networks , 2021, Nature.