Coupled convolution layer for convolutional neural network

We propose a coupled convolution layer comprising multiple parallel convolutions with mutually constrained filters. Inspired by biological human vision mechanism, we constrain the convolution filters such that one set of filter weights should be geometrically rotated, mirrored, or be the negative of the other. Our analysis suggests that the coupled convolution layer is more effective for lower layer where feature maps preserve geometric properties. Experimental comparisons demonstrate that the proposed coupled convolution layer performs slightly better than the original layer while decreasing the number of parameters. We evaluate its effect compared to non-constrained convolution layer using the CIFAR-10, CIFAR-100, and PlanktonSet 1.0 datasets.

[1]  Zhigang Zeng,et al.  Global asymptotical stability analysis for a kind of discrete-time recurrent neural network with discontinuous activation functions , 2016, Neurocomputing.

[2]  Gianluca Vinti,et al.  Max-product neural network and quasi-interpolation operators activated by sigmoidal functions , 2016, J. Approx. Theory.

[3]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[4]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[5]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[6]  B. Wandell.  Foundations of vision , 1995 .

[7]  Zhigang Zeng,et al.  Region stability analysis for switched discrete-time recurrent neural network with multiple equilibria , 2017, Neurocomputing.

[8]  Renato Spigler,et al.  Solving Volterra integral equations of the second kind by sigmoidal functions approximation , 2013 .

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Minho Lee,et al.  Convolutional Neural Network with Biologically Inspired ON/OFF ReLU , 2015, ICONIP.

[11]  Renato Spigler,et al.  A collocation method for solving nonlinear Volterra integro-differential equations of neutral type by sigmoidal functions , 2014 .

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Feilong Cao,et al.  The approximation operators with sigmoidal functions , 2009, Comput. Math. Appl..

[15]  Feilong Cao,et al.  Scattered data approximation by neural networks operators , 2016, Neurocomputing.

[16]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[17]  Bum Il Hong,et al.  A note on neural network approximation with a sigmoidal function , 2016 .

[18]  Qiang Chen,et al.  Network In Network , 2013, ICLR.