On-Off Center-Surround Receptive Fields for Accurate and Robust Image Classification

Robustness to variations in lighting conditions is a key objective for any deep vision system. To this end, our paper extends the receptive field of convolutional neural networks with two residual components, ubiquitous in the visual processing system of vertebrates: On-center and off-center pathways, with excitatory center and inhibitory surround; OOCS for short. The on-center pathway is excited by the presence of a light stimulus in its center, but not in its surround, whereas the off-center one is excited by the absence of a light stimulus in its center, but not in its surround. We design OOCS pathways via a difference of Gaussians, with their variance computed analytically from the size of the receptive fields. OOCS pathways complement each other in their response to light stimuli, ensuring this way a strong edge-detection capability, and as a result an accurate and robust inference under challenging lighting conditions. We provide extensive empirical evidence showing that networks supplied with the OOCS edge representation gain accuracy and illumination-robustness compared to standard deep models.

[1]  Kunihiko Fukushima,et al.  Neocognitron for handwritten digit recognition , 2003, Neurocomputing.

[2]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[3]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[5]  Radu Grosu,et al.  Designing Worm-inspired Neural Networks for Interpretable Robotic Control , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[6]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Nicolai Petkov,et al.  Modifications of center-surround, spot detection and dot-pattern selective operators , 2005 .

[8]  M. R. Blackburn,et al.  A Simple Computational Model of Center-Surround Receptive Fields in the Retina , 1993 .

[9]  Radu Grosu,et al.  Liquid Time-constant Networks , 2020, AAAI.

[10]  Radu Grosu,et al.  A Natural Lottery Ticket Winner: Reinforcement Learning with Ordinary Neural Circuits , 2020, ICML.

[11]  E. Callaway.  Structure and function of parallel pathways in the primate early visual system , 2005, The Journal of physiology.

[12]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  Raquel Urtasun,et al.  Understanding the Effective Receptive Field in Deep Convolutional Neural Networks , 2016, NIPS.

[14]  Jian Yang,et al.  Selective Kernel Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  R. W. Rodieck.  Quantitative analysis of cat retinal ganglion cell response to visual stimuli. , 1965, Vision research.

[16]  Radu Grosu,et al.  Neural circuit policies enabling auditable autonomy , 2020, Nature Machine Intelligence.

[17]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[18]  George Azzopardi,et al.  A Push-Pull CORF Model of a Simple Cell with Antiphase Inhibition Improves SNR and Contour Detection , 2014, PloS one.

[19]  Surya Ganguli,et al.  Task-Driven Convolutional Recurrent Models of the Visual System , 2018, NeurIPS.

[20]  Zhaoxiang Zhang,et al.  Scale-Aware Trident Networks for Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[21]  Md Nasir Uddin Laskar,et al.  Correspondence of Deep Neural Networks and the Brain for Visual Textures , 2018, ArXiv.

[22]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[23]  Richard Gordon,et al.  OpenWorm: overview and recent advances in integrative biological simulation of Caenorhabditis elegans , 2018, Philosophical Transactions of the Royal Society B.

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[26]  J. Allman,et al.  Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. , 1985, Annual review of neuroscience.

[27]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[28]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[29]  Jungong Han,et al.  ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[30]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[31]  George Azzopardi,et al.  Robust Inhibition-Augmented Operator for Delineation of Curvilinear Structures , 2019, IEEE Transactions on Image Processing.

[32]  Petros Daras,et al.  Non-linear Convolution Filters for CNN-Based Learning , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Thomas A. Henzinger,et al.  Adversarial Training is Not Ready for Robot Learning , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[35]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  C. Enroth-Cugell,et al.  Properties of the surround response mechanism of cat retinal ganglion cells and centre‐surround interaction , 1972, The Journal of physiology.

[37]  Arnold W. M. Smeulders,et al.  Structured Receptive Fields in CNNs , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Minho Lee,et al.  Convolutional Neural Network with Biologically Inspired Retinal Structure , 2016, BICA.

[39]  Hamid Aghajan,et al.  Surround Modulation: A Bio-inspired Connectivity Structure for Convolutional Neural Networks , 2019, NeurIPS.

[40]  Zeb Kurth-Nelson,et al.  A distributional code for value in dopamine-based reinforcement learning , 2020, Nature.

[41]  M. Aminoff.  Principles of Neural Science. 4th edition , 2001 .

[42]  S. W. Kuffler.  Discharge patterns and functional organization of mammalian retina. , 1953, Journal of neurophysiology.

[43]  D. V. van Essen,et al.  Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. , 1992, Journal of neurophysiology.

[44]  H. K. Hartline,et al.  THE RECEPTIVE FIELDS OF OPTIC NERVE FIBERS , 1940 .

[45]  Chen Wang,et al.  Kervolutional Neural Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Radu Grosu,et al.  c302: a multiscale framework for modelling the nervous system of Caenorhabditis elegans , 2018, Philosophical Transactions of the Royal Society B: Biological Sciences.

[47]  Thomas Serre,et al.  Learning long-range spatial dependencies with horizontal gated-recurrent units , 2018, NeurIPS.

[48]  J. B. Demb,et al.  Different Circuits for ON and OFF Retinal Ganglion Cells Cause Different Contrast Sensitivities , 2003, The Journal of Neuroscience.

[49]  R. Shapley,et al.  Cat and monkey retinal ganglion cells and their visual functional roles , 1986, Trends in Neurosciences.

[50]  Radu Grosu,et al.  Worm-level Control through Search-based Reinforcement Learning , 2017, ArXiv.

[51]  D. Dacey,et al.  Origins of perception : retinal ganglion cell diversity and the creation of parallel visual pathways , 2011 .

[52]  Radu Grosu,et al.  Gershgorin Loss Stabilizes the Recurrent Neural Network Compartment of an End-to-end Robot Learning Scheme , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[53]  Luca Maria Gambardella,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Flexible, High Performance Convolutional Neural Networks for Image Classification , 2022 .