Influence of the receptive field size on accuracy and performance of a convolutional neural network

Convolutional neural networks (CNNs) have been successfully applied to many tasks such as digit and object recognition. In this paper we study the size of the receptive field of deep convolutional neural networks, in particular, we check the idea of a "redundant" receptive field. We run a set of experiments on two common CNN models — VGG16 and ResNet18 — in order to explore the influence of receptive field size on CNN's training time, accuracy, and performance. We run experiments using the MakiFlow framework on the CALTECH256 dataset. The experiments' results show that the optimization of neural networks (NNs) by reducing the size of the receptive field allows to reduce the NN’s training time by 5-7% while maintaining the accuracy of the network.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[3]  A. V. Gaidel,et al.  Efficiency of machine learning algorithms and convolutional neural network for detection of pathological changes in MR images of the brain , 2020 .

[4]  Masakazu Matsugu,et al.  Subject independent facial expression recognition with robust face detection using a convolutional neural network , 2003, Neural Networks.

[5]  R. Bohush,et al.  Person tracking algorithm based on convolutional neural network for indoor video surveillance , 2020 .

[6]  A. V. Mingalev,et al.  Test-object recognition in thermal images , 2019 .

[7]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[10]  Francesco Visin,et al.  A guide to convolution arithmetic for deep learning , 2016, ArXiv.

[11]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[12]  Ali Borji,et al.  What are the Receptive, Effective Receptive, and Projective Fields of Neurons in Convolutional Neural Networks? , 2017, ArXiv.

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).