Binary Separable Convolutional: An Efficient Fast Image Classification Method

The training and running of neural network require large computational space and memory space, which makes it difficult to deploy on a resource-constrained embedded sys-tems. To address this limitation, we introduce a two-stage pipeline: Depth separable, local binary. Our method is divided into two steps. Firstly, a deep separable convolution is applied to each input channel. Then, point-by-point convolution is applied to the feature map obtained by the filter. In the second step, we use local binarization method to initialize the filter corresponding to the input channel into sparse binary. In network training, the sparse binary filter remains fixed and only needs to train convolution of 1×1 size. Our experimental results show that, on the basis of approximate accuracy with the original network, we have reduced the number of convolution parameters by 9x to 10x, and reduced the training time and testing time to by 2x. Our compression method helps to deploy complex neural networks on resource-constrained embedded platform.

[1]  B. Noble,et al.  On certain integrals of Lipschitz-Hankel type involving products of bessel functions , 1955, Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences.

[2]  Ran El-Yaniv,et al.  Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations , 2016, J. Mach. Learn. Res..

[3]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[4]  Le Song,et al.  Deep Fried Convnets , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  H. E. Chandler,et al.  Technical writer's handbook , 1982, IEEE Transactions on Professional Communication.

[6]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Ross B. Girshick,et al.  Reducing Overfitting in Deep Networks by Decorrelating Representations , 2015, ICLR.

[8]  Tara N. Sainath,et al.  Structured Transforms for Small-Footprint Deep Learning , 2015, NIPS.

[9]  T. Yorozu,et al.  Electron Spectroscopy Studies on Magneto-Optical Media and Plastic Substrate Interface , 1987, IEEE Translation Journal on Magnetics in Japan.