Dynamic Precision Analog Computing for Neural Networks

[1]  Bhavin J. Shastri,et al.  Neuromorphic Photonic Integrated Circuits , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[2]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[3]  Lin Yang,et al.  On-chip CMOS-compatible optical signal processor , 2012, 2012 Asia Communications and Photonics Conference (ACP).

[4]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[5]  Bahaa E. A. Saleh,et al.  Shot-noise-limited performance of optical neural networks , 1996, IEEE Trans. Neural Networks.

[6]  Paul R. Prucnal,et al.  A Laser Spiking Neuron in a Photonic Integrated Circuit. , 2020, 2012.08516.

[7]  Raghuraman Krishnamoorthi,et al.  Quantizing deep convolutional networks for efficient inference: A whitepaper , 2018, ArXiv.

[8]  Miao Hu,et al.  ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[9]  Yuandong Tian,et al.  Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search , 2018, ArXiv.

[10]  Omer Levy,et al.  GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding , 2018, BlackboxNLP@EMNLP.

[11]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[12]  Kurt Keutzer,et al.  HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks , 2020, NeurIPS.

[13]  C. Wright,et al.  Photonics for artificial intelligence and neuromorphic computing , 2020, ArXiv.

[14]  Paul R. Prucnal,et al.  Broadcast and Weight: An Integrated Network For Scalable Photonic Spike Processing , 2014, Journal of Lightwave Technology.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  A. Sripad,et al.  A necessary and sufficient condition for quantization errors to be uniform and white , 1977 .

[17]  Catherine Graves,et al.  Dot-product engine for neuromorphic computing: Programming 1T1M crossbar to accelerate matrix-vector multiplication , 2016, 2016 53nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[18]  Theodore Antonakopoulos,et al.  Mixed-Precision Deep Learning Based on Computational Memory , 2020, Frontiers in Neuroscience.

[19]  Chen Feng,et al.  A Quantization-Friendly Separable Convolution for MobileNets , 2018, 2018 1st Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2).

[20]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Robert M. Gray,et al.  Quantization noise spectra , 1990, IEEE Trans. Inf. Theory.

[22]  David Blaauw,et al.  Analog in-memory subthreshold deep neural network accelerator , 2017, 2017 IEEE Custom Integrated Circuits Conference (CICC).

[23]  Daniel Soudry,et al.  Post training 4-bit quantization of convolutional networks for rapid-deployment , 2018, NeurIPS.

[24]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Kurt Keutzer,et al.  ZeroQ: A Novel Zero Shot Quantization Framework , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  David A. B. Miller,et al.  Perfect optics with imperfect components , 2015 .

[27]  Marco Cococcioni,et al.  Photonic Neural Networks: A Survey , 2019, IEEE Access.

[28]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[29]  Dirk Englund,et al.  Deep learning with coherent nanophotonic circuits , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[30]  Paul R. Prucnal,et al.  Silicon Photonic Modulator Neuron , 2018, Physical Review Applied.

[31]  Avi Mendelson,et al.  NICE: Noise Injection and Clamping Estimation for Neural Network Quantization , 2018, Mathematics.

[32]  Paul R. Prucnal,et al.  Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs) , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[33]  Evangelos Eleftheriou,et al.  Accurate deep neural network inference using computational phase-change memory , 2019, Nature Communications.

[34]  Sachin S. Talathi,et al.  Fixed Point Quantization of Deep Convolutional Networks , 2015, ICML.

[35]  Ryan Hamerly,et al.  Large-Scale Optical Neural Networks based on Photoelectric Multiplication , 2018, Physical Review X.

[36]  Yusuf Leblebici,et al.  Neuromorphic computing with multi-memristive synapses , 2017, Nature Communications.

[37]  Jongeun Lee,et al.  DPS: Dynamic Precision Scaling for Stochastic Computing-based Deep Neural Networks* , 2018, 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[38]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[39]  Peng Lin,et al.  Reinforcement learning with analogue memristor arrays , 2019, Nature Electronics.

[40]  Yoshua Bengio,et al.  Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation , 2013, ArXiv.

[41]  Kurt Keutzer,et al.  HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Evangelos Eleftheriou,et al.  Mixed-precision architecture based on computational memory for training deep neural networks , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[43]  Paul R. Prucnal,et al.  Machine Learning With Neuromorphic Photonics , 2019, Journal of Lightwave Technology.

[44]  Ojas Parekh,et al.  Energy Scaling Advantages of Resistive Memory Crossbar Based Computation and Its Application to Sparse Coding , 2016, Front. Neurosci..

[45]  Ellen Zhou,et al.  Neuromorphic photonic networks using silicon photonic weight banks , 2017, Scientific Reports.

[46]  Steven K. Esser,et al.  Learned Step Size Quantization , 2019, ICLR.

[47]  Yan Wang,et al.  Fully Quantized Network for Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Dharmendra S. Modha,et al.  Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference , 2018, ArXiv.

[49]  P. Narayanan,et al.  Recent progress in analog memory-based accelerators for deep learning , 2018, Journal of Physics D: Applied Physics.

[50]  Paul R. Prucnal,et al.  Microring Weight Banks , 2016, IEEE Journal of Selected Topics in Quantum Electronics.

[51]  Yu-Li You,et al.  Audio Coding: Theory and Applications , 2010 .

[52]  Pritish Narayanan,et al.  Neuromorphic computing using non-volatile memory , 2017 .

[53]  Heiner Giefers,et al.  Mixed-precision in-memory computing , 2017, Nature Electronics.

[54]  Nikolas Ioannou,et al.  Deep learning acceleration based on in-memory computing , 2019, IBM J. Res. Dev..

[55]  Engin Ipek,et al.  Making Memristive Neural Network Accelerators Reliable , 2018, 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[56]  Ron Banner,et al.  Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming , 2020, ArXiv.

[57]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Steven J. Plimpton,et al.  Resistive memory device requirements for a neural algorithm accelerator , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[59]  John Tran,et al.  cuDNN: Efficient Primitives for Deep Learning , 2014, ArXiv.

[60]  Bo Chen,et al.  Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[61]  Fabien Cardinaux,et al.  Mixed Precision DNNs: All you need is a good parametrization , 2019, ICLR.

[62]  Xiangyu Zhang,et al.  ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design , 2018, ECCV.

[63]  Paul R. Prucnal,et al.  Photonic Multiply-Accumulate Operations for Neural Networks , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[64]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[65]  Qing Wu,et al.  Long short-term memory networks in memristor crossbar arrays , 2018, Nature Machine Intelligence.

[66]  Zhijian Liu,et al.  HAQ: Hardware-Aware Automated Quantization With Mixed Precision , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).