DRFNet: A lightweight and high accuracy network for resource-limited implementation

With the increased demand of recognition accuracy in image classification, the convolutional neural networks (CNNs) grow rapidly in scale, which makes CNNs more difficult to be deployed on resource-limited platforms. In this paper, we present a lightweight Deep Residual FireNet (DRFNet). The new DRFNet has 26 weight layers and uses the fire module as its basic building module. In its architecture, we use three max-pooling operations, and additionally two average-pooling layers to preserve a higher resolution and larger receptive field. In addition, four bypass connections are used to further improve the accuracy rate. With this new design, the proposed DRFNet has successfully reduced the number of weight parameters to be around 1.85 million, 3% of AlexNet and 1.3% of VGG16. Meanwhile, the new DRFNet has achieved 91.44% top-5 accuracy rate on the ImageNet validation dataset, which is higher than AlexNet, SqueezeNet, VGG16 and GoogLeNet.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Joan Bruna,et al.  Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation , 2014, NIPS.

[3]  Thomas A. Funkhouser,et al.  Dilated Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Song Han,et al.  Learning both Weights and Connections for Efficient Neural Network , 2015, NIPS.

[5]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[6]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[11]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[12]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).