论文信息 - Pruning convolution neural network (squeezenet) using taylor expansion-based criterion

Pruning convolution neural network (squeezenet) using taylor expansion-based criterion

Recent research in the field of deep learning focuses on reducing the model size of the Convolution Neural Network (CNN) by various compression techniques like Pruning, Quantization and Encoding (eg. Huffman encoding). This paper proposes a way to prune the CNN based on Taylor expansion of change in cost function ΔC of the model. The proposed algorithm uses greedy criteria based pruning with fine-tuning by backpropagation on SqueezeNet architecture. Transfer learning technique is used to train the SqueezeNet on the CIFAR-10 dataset. The proposed algorithm achieves 70% model reduction on SqueezeNet architecture with only 1% drop in accuracy.

Mohamed El-Sharkawy | Akash Sunil Gaikwad

[1] Forrest N. Iandola,et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[2] Timo Aila,et al. Pruning Convolutional Neural Networks for Resource Efficient Transfer Learning , 2016, ArXiv.

[3] Jacek M. Zurada,et al. Building Efficient ConvNets using Redundant Feature Pruning , 2018, ArXiv.

[4] Hanan Samet,et al. Pruning Filters for Efficient ConvNets , 2016, ICLR.

[5] H. Howie Huang,et al. Performance Analysis of GPU-Based Convolutional Neural Networks , 2016, 2016 45th International Conference on Parallel Processing (ICPP).

[6] Wonyong Sung,et al. Structured Pruning of Deep Convolutional Neural Networks , 2015, ACM J. Emerg. Technol. Comput. Syst..

[7] Michael I. Jordan,et al. Advances in Neural Information Processing Systems 30 , 1995 .

[8] Yann LeCun,et al. Optimal Brain Damage , 1989, NIPS.