Analysis of Diffractive Optical Neural Networks and Their Integration With Electronic Neural Networks

Optical machine learning offers advantages in terms of power efficiency, scalability, and computation speed. Recently, an optical machine learning method based on diffractive deep neural networks (D2NNs) has been introduced to execute a function as the input light diffracts through passive layers, designed by deep learning using a computer. Here, we introduce improvements to D2NNs by changing the training loss function and reducing the impact of vanishing gradients in the error back-propagation step. Using five phase-only diffractive layers, we numerically achieved a classification accuracy of 97.18% and 89.13% for optical recognition of handwritten digits and fashion products, respectively; using both phase and amplitude modulation (complex-valued) at each layer, our inference performance improved to 97.81% and 89.32%, respectively. Furthermore, we report the integration of D2NNs with electronic neural networks to create hybrid classifiers that significantly reduce the number of input pixels into an electronic network using an ultra-compact front-end D2NN with a layer-to-layer distance of a few wavelengths, also reducing the complexity of the successive electronic network. Using a five-layer phase-only D2NN jointly optimized with a single fully connected electronic layer, we achieved a classification accuracy of 98.71% and 90.04% for the recognition of handwritten digits and fashion products, respectively. Moreover, the input to the electronic network was compressed by >7.8 times down to 10 × 10 pixels. Beyond creating low-power and high-frame rate machine learning platforms, D2NN-based hybrid neural networks will find applications in smart optical imager and sensor design.

[1]  Paul R. Prucnal,et al.  Principles of Neuromorphic Photonics , 2017, ArXiv.

[2]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[3]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[4]  Yichuan Tang,et al.  Deep Learning using Linear Support Vector Machines , 2013, 1306.0239.

[5]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[6]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[7]  Andrei Faraon,et al.  A review of dielectric optical metasurfaces for wavefront control , 2018, Nanophotonics.

[8]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[9]  Hermann Ney,et al.  Cross-entropy vs. squared error training: a theoretical and experimental comparison , 2013, INTERSPEECH.

[10]  Nicholas Soures,et al.  Neuro-MMI: A Hybrid Photonic-Electronic Machine Learning Platform , 2018, 2018 IEEE Photonics Society Summer Topical Meeting Series (SUM).

[11]  Pu Jian,et al.  Programmable unitary spatial mode manipulation. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[12]  Anschrift Dr. Cornelia Denz.  Optical Neural Networks , 1998, Vieweg+Teubner Verlag.

[13]  Francis T. S. Yu.  Optical neural network: architecture, design, and models , 1991, Optics & Photonics.

[14]  J. Goodman.  Introduction to Fourier optics , 1969 .

[15]  Andrei Faraon,et al.  Wavefront shaping with disorder-engineered metasurfaces , 2017, Nature Photonics.

[16]  Ellen Zhou,et al.  Neuromorphic photonic networks using silicon photonic weight banks , 2017, Scientific Reports.

[17]  Bhiksha Raj,et al.  Be Careful What You Backpropagate: A Case For Linear Output Activations & Gradient Boosting , 2017, ArXiv.

[18]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[19]  K. Wagner,et al.  Fully interconnected, two-dimensional neural arrays using wavelength-multiplexed volume holograms. , 1991, Optics letters.

[20]  Mark Horowitz,et al.  1.1 Computing's energy problem (and what we can do about it) , 2014, 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC).

[21]  Laurent Larger,et al.  Reinforcement Learning in a large scale photonic Recurrent Neural Network , 2017, Optica.

[22]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[23]  B Javidi,et al.  Optical implementation of neural networks for face recognition by the use of nonlinear joint transform correlators. , 1995, Applied optics.

[24]  Andrew L. Maas.  Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[25]  Indranil Chakraborty,et al.  Toward Fast Neural Computing using All-Photonic Phase Change Spiking Neurons , 2018, Scientific Reports.

[26]  D Psaltis,et al.  Optical information processing based on an associative-memory model of neural nets with thresholding and feedback. , 1985, Optics letters.

[27]  R. Piestun,et al.  Aperiodic Volume Optics , 2010 .

[28]  Shanhui Fan,et al.  Training of Photonic Neural Networks through In Situ Backpropagation , 2018, 2019 Conference on Lasers and Electro-Optics (CLEO).

[29]  D. Psaltis,et al.  Holography in artificial neural networks , 1990, Nature.

[30]  Francis Yu.  Optical neural network - Architecture, design and models , 1991 .

[31]  Gordon Wetzstein,et al.  Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification , 2018, Scientific Reports.

[32]  K Wagner,et al.  Multilayer optical learning networks. , 1987, Applied optics.

[33]  Yi Luo,et al.  All-optical machine learning using diffractive deep neural networks , 2018, Science.

[34]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[35]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[36]  Volker J. Sorger,et al.  Electrooptic nonlinear activation functions for vector matrix multiplications in optical neural networks , 2018 .

[37]  Vladimir Ceperic,et al.  On-Chip Optical Convolutional Neural Networks , 2018, ArXiv.

[38]  Boris N. Chichkov,et al.  Two-photon polymerization technique with sub-50 nm resolution by sub-10 fs laser pulses , 2012 .

[39]  Dirk Englund,et al.  Deep learning with coherent nanophotonic circuits , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[40]  D. Brady,et al.  Adaptive optical networks using photorefractive crystals. , 1988, Applied optics.

[41]  D Psaltis,et al.  Optical implementation of the Hopfield model. , 1985, Applied optics.

[42]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[43]  Armin Mehrabian,et al.  PCNNA: A Photonic Convolutional Neural Network Accelerator , 2018, 2018 31st IEEE International System-on-Chip Conference (SOCC).