Multipath feature recalibration DenseNet for image classification

Recently, deep neural networks have demonstrated their efficiency in image classification tasks, which are commonly achieved by an extended depth and width of network architecture. However, poor convergence, over-fitting and gradient disappearance might be generated with such comprehensive architectures. Therefore, DenseNet is developed to address these problems. Although DenseNet adopts bottleneck technique in DenseBlocks to avoid relearning feature-maps and decrease parameters, this operation may lead to the skip and loss of important features. Besides, it still takes oversized computational power when the depth and width of the network architecture are increased for better classification. In this paper, we propose a variate of DenseNet, named Multipath Feature Recalibration DenseNet (MFR-DenseNet), to stack convolution layers instead of adopting bottleneck for improving feature extraction. Meanwhile, we build multipath DenseBlocks with Squeeze-Excitation (SE) module to represent the interdependencies of useful feature-maps among different DenseBlocks. Experiments in CIFAR-10, CIFAR-100, MNIST and SVHN reveal the efficiency of our network, with further reduced redundancy whilst maintaining the high accuracy of DenseNet.

[1]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[3]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[6]  Yaoqin Xie,et al.  A Sparse-View CT Reconstruction Method Based on Combination of DenseNet and Deconvolution , 2018, IEEE Transactions on Medical Imaging.

[7]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Qing Zhou,et al.  Grading of hepatocellular carcinoma using 3D SE-DenseNet in dynamic enhanced MR images , 2019, Comput. Biol. Medicine.

[9]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[10]  Ke Zhang,et al.  Multiple Feature Reweight DenseNet for Image Classification , 2019, IEEE Access.

[11]  Jürgen Schmidhuber,et al.  Training Very Deep Networks , 2015, NIPS.

[12]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[13]  Kilian Q. Weinberger,et al.  CondenseNet: An Efficient DenseNet Using Learned Group Convolutions , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[15]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[16]  Shuicheng Yan,et al.  Dual Path Networks , 2017, NIPS.

[17]  Jaewoo Kang,et al.  Multipath-DenseNet: A Supervised ensemble architecture of densely connected convolutional networks , 2019, Inf. Sci..

[18]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[20]  Sepp Hochreiter,et al.  The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions , 1998, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[21]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[22]  L. Deng,et al.  The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web] , 2012, IEEE Signal Processing Magazine.

[23]  Gregory Shakhnarovich,et al.  FractalNet: Ultra-Deep Neural Networks without Residuals , 2016, ICLR.

[24]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Wenqi Liu,et al.  SparseNet: A Sparse DenseNet for Image Classification , 2018, ArXiv.