Design of optical neural networks with component imprecisions

For the benefit of designing scalable, fault resistant optical neural networks (ONNs), we investigate the effects architectural designs have on the ONNs' robustness to imprecise components. We train two ONNs - one with a more tunable design (GridNet) and one with better fault tolerance (FFTNet) - to classify handwritten digits. When simulated without any imperfections, GridNet yields a better accuracy (∼98%) than FFTNet (∼95%). However, under a small amount of error in their photonic components, the more fault tolerant FFTNet overtakes GridNet. We further provide thorough quantitative and qualitative analyses of ONNs' sensitivity to varying levels and types of imprecisions. Our results offer guidelines for the principled design of fault-tolerant ONNs as well as a foundation for further research.

[1]  Ran El-Yaniv,et al.  Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations , 2016, J. Mach. Learn. Res..

[2]  Bahram Jalali,et al.  Analog optical computing primitives in silicon photonics. , 2016, Optics letters.

[3]  Zhenhua Ni,et al.  Monolayer graphene as a saturable absorber in a mode-locked laser , 2010, 1007.2243.

[4]  J. O'Brien,et al.  Universal linear optics , 2015, Science.

[5]  Anthony Laing,et al.  Direct dialling of Haar random unitary matrices , 2015, 1506.06220.

[6]  David A. B. Miller,et al.  Perfect optics with imperfect components , 2015 .

[7]  Bhavin J. Shastri,et al.  Neuromorphic photonic networks using silicon photonic weight banks , 2016, Scientific Reports.

[8]  Seyed-Mohsen Moosavi-Dezfooli,et al.  Robustness of classifiers: from adversarial to random noise , 2016, NIPS.

[9]  Ames,et al.  Using an imperfect photonic network to implement random unitaries , 2017 .

[10]  Yacob Ben-Aryeh,et al.  Quantum fast fourier transform and quantum computation by linear optics , 2007 .

[11]  Paul R. Prucnal,et al.  Broadcast and Weight: An Integrated Network For Scalable Photonic Spike Processing , 2014, Journal of Lightwave Technology.

[12]  K. Kikuchi.  Characterization of semiconductor-laser phase noise and estimation of bit-error rate performance with low-speed offline digital coherent receivers. , 2012, Optics express.

[13]  Fabio Sciarrino,et al.  Thermally reconfigurable quantum photonic circuits at telecom wavelength by femtosecond laser micromachining , 2015, Light: Science & Applications.

[14]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[15]  David A. B. Miller,et al.  Matrix optimization on universal unitary photonic devices , 2018, Physical Review Applied.

[16]  Reck,et al.  Experimental realization of any discrete unitary operator. , 1994, Physical review letters.

[17]  Yoshua Bengio,et al.  Unitary Evolution Recurrent Neural Networks , 2015, ICML.

[18]  G. Lo,et al.  Ultralow loss single layer submicron silicon waveguide crossing for SOI optical interconnect. , 2013, Optics express.

[19]  M. Moewe,et al.  Narrow linewidth high power thermally tuned sampled-grating distributed Bragg reflector laser , 2013, 2013 Optical Fiber Communication Conference and Exposition and the National Fiber Optic Engineers Conference (OFC/NFOEC).

[20]  J L O'Brien,et al.  60  dB high-extinction auto-configured Mach-Zehnder interferometer. , 2016, Optics letters.

[21]  Humphreys,et al.  An Optimal Design for Universal Multiport Interferometers , 2016, 1603.08788.

[22]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[23]  Nicolò Spagnolo,et al.  Benchmarking integrated linear-optical architectures for quantum information processing , 2017, Scientific Reports.

[24]  Photoinitiator-free multi-photon fabrication of compact optical waveguides in polydimethylsiloxane , 2018, Optical Materials Express.

[25]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[26]  Benjamin Schrauwen,et al.  Optoelectronic Reservoir Computing , 2011, Scientific Reports.

[27]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .

[28]  J. Tukey,et al.  An algorithm for the machine calculation of complex Fourier series , 1965 .

[29]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[30]  Michael J. Connelly,et al.  Semiconductor Optical Amplifiers , 2002 .

[31]  Optical Bistability Based on the Carrier Dispersion Effect in SOI Ring Resonators , 2006 .

[32]  Gregory R. Steinbrecher,et al.  Quantum transport simulations in a programmable nanophotonic processor , 2015, Nature Photonics.

[33]  David A. B. Miller,et al.  Meshing optics with applications , 2017, Nature Photonics.

[34]  A C Selden,et al.  Pulse transmission through a saturable absorber , 1967 .

[35]  Ellen Zhou,et al.  Neuromorphic photonic networks using silicon photonic weight banks , 2017, Scientific Reports.

[36]  Yann LeCun,et al.  Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs , 2016, ICML.

[37]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[38]  Sandeep Subramanian,et al.  Deep Complex Networks , 2017, ICLR.

[39]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[40]  Yoshua Bengio,et al.  Série Scientifique Scientific Series Incorporating Second-order Functional Knowledge for Better Option Pricing Incorporating Second-order Functional Knowledge for Better Option Pricing , 2022 .

[41]  Gordon Wetzstein,et al.  Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification , 2018, Scientific Reports.

[42]  Robert A Norwood,et al.  Nonlinear optical components for all-optical probabilistic graphical model , 2018, Nature Communications.

[43]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[44]  Eric Mazur,et al.  Femtosecond laser micromachining in transparent materials , 2008 .

[45]  D Psaltis,et al.  Optical implementation of the Hopfield model. , 1985, Applied optics.