Photonic Convolution Neural Network Based on Interleaved Time-Wavelength Modulation

Convolution neural network (CNN), as one of the most powerful and popular technologies, has achieved remarkable progress for image and video classification since its invention. However, with the high definition video-data explosion, convolution layers in the CNN architecture will occupy a great amount of computing time and memory resources due to high computation complexity of matrix multiply accumulate operation. In this paper, a novel integrated photonic CNN is proposed based on double correlation operations through interleaved time-wavelength modulation. Micro-ring based multi-wavelength manipulation and single dispersion medium are utilized to realize convolution operation and replace the conventional optical delay lines. 200 images are tested in MNIST datasets with accuracy of 85.5<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> in our photonic CNN versus 86.5<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> in 64-bit computer. We also analyze the computing error of photonic CNN caused by various micro-ring parameters, operation baud rates and the characteristics of micro-ring weighting bank. Furthermore, a tensor processing unit based on 4×4 mesh with 1.2 Tera operation per second (TOPS) computing capability at 20 G baud rate is proposed and analyzed to form a paralleled photonic CNN.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Zuyuan He,et al.  Programmable matrix operation with reconfigurable time-wavelength plane manipulation and dispersed time delay. , 2019, Optics express.

[3]  Akihiko Kasagi,et al.  Efficient cuDNN-Compatible Convolution-Pooling on the GPU , 2019, PPAM.

[4]  Philip Y. Ma,et al.  Feedback control for microring weight banks. , 2018, Optics express.

[5]  Vladimir Ceperic,et al.  On-Chip Optical Convolutional Neural Networks , 2018, ArXiv.

[6]  Andrew Lavin,et al.  Fast Algorithms for Convolutional Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Thomas Ferreira de Lima,et al.  Photonic independent component analysis using an on-chip microring weight bank. , 2020, Optics express.

[8]  H. John,et al.  Why future supercomputing requires optics , 2010 .

[9]  Matthew Mattina,et al.  Systolic Tensor Array: An Efficient Structured-Sparse GEMM Accelerator for Mobile CNN Inference , 2020, IEEE Computer Architecture Letters.

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Akihiko Kasagi,et al.  Efficient convolution pooling on the GPU , 2020, J. Parallel Distributed Comput..

[12]  Byoung-Joon Seo,et al.  Electrooptic Polymer Ring Resonator Modulation up to 165 GHz , 2007, IEEE Journal of Selected Topics in Quantum Electronics.

[13]  Luca P. Carloni,et al.  Silicon Photonics Codesign for Deep Learning , 2020, Proceedings of the IEEE.

[14]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Weiwen Zou,et al.  Optical patching scheme for optical convolutional neural networks based on wavelength-division multiplexing and optical delay lines. , 2020, Optics letters.

[16]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[18]  Daniel Brand,et al.  MEC: Memory-efficient Convolution for Deep Neural Network , 2017, ICML.

[19]  Marat Dukhan,et al.  The Indirect Convolution Algorithm , 2019, ArXiv.

[20]  Jingdong Luo,et al.  Ring resonator-based electrooptic polymer traveling-wave modulator , 2006, Journal of Lightwave Technology.

[21]  Dirk Englund,et al.  Deep learning with coherent nanophotonic circuits , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[22]  Yangqing Jia,et al.  Learning Semantic Image Representations at a Large Scale , 2014 .

[23]  Tianshi Chen,et al.  DaDianNao: A Neural Network Supercomputer , 2017, IEEE Transactions on Computers.