COSFIRE: A Brain-Inspired Approach to Visual Pattern Recognition

The primate visual system has an impressive ability to generalize and to discriminate between numerous objects and it is robust to many geometrical transformations as well as lighting conditions. The study of the visual system has been an active reasearch field in neuropysiology for more than half a century. The construction of computational models of visual neurons can help us gain insight in the processing of information in visual cortex which we can use to provide more robust solutions to computer vision applications. Here, we demonstrate how inspiration from the functions of shape-selective V4 neurons can be used to design trainable filters for visual pattern recognition. We call this approach COSFIRE, which stands for Combination of Shifted Filter Responses. We illustrate how a COSFIRE filter can be configured to be selective for the spatial arrangement of lines and/or edges that form the shape of a given prototype pattern. Finally, we demonstrate the effectiveness of the COSFIRE approach in three applications: the detection of vascular bifurcations in retinal fundus images, the localization and recognition of traffic signs in complex scenes and the recognition of handwritten digits. This work is a further step in understanding how visual information is processed in the brain and how information on pixel intensities is converted into information about objects. We demonstrate how this understanding can be used for the design of effective computer vision algorithms.

[1]  George Azzopardi,et al.  Automatic detection of vascular bifurcations in segmented retinal images using trainable COSFIRE filters , 2013, Pattern Recognit. Lett..

[2]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[3]  John Daugman,et al.  High Confidence Visual Recognition of Persons by a Test of Statistical Independence , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Nicolai Petkov,et al.  Suppression of contour perception by band-limited noise and its relation to nonclassical receptive field inhibition , 2003, Biological cybernetics.

[5]  Nicolai Petkov,et al.  Contour and boundary detection improved by surround suppression of texture edges , 2004, Image Vis. Comput..

[6]  George Azzopardi,et al.  Trainable COSFIRE Filters for Keypoint Detection and Pattern Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  C. W. Tyler.  Selectivity for spatial frequency and bar width in cat visual cortex , 1978, Vision Research.

[8]  Nicolai Petkov,et al.  Comparison of texture features based on Gabor filters , 2002, IEEE Trans. Image Process..

[9]  Alan C. Bovik,et al.  Analysis of multichannel narrow-band filters for image texture segmentation , 1991, IEEE Trans. Signal Process..

[10]  John G. Daugman,et al.  Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression , 1988, IEEE Trans. Acoust. Speech Signal Process..

[11]  Nicolai Petkov,et al.  Contour detection based on nonclassical receptive field inhibition , 2003, IEEE Trans. Image Process..

[12]  Peter Kovesi,et al.  Image Features from Phase Congruency , 1995 .

[13]  D. G. Albrecht,et al.  Cortical cells ; Bar and edge detectors, or spatial frequency filters , 1978 .

[14]  D. Hubel.  Exploration of the primary visual cortex, 1955–78 , 1982, Nature.

[15]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[16]  Nicolai Petkov,et al.  Distance sets for shape filters and shape recognition , 2003, IEEE Trans. Image Process..

[17]  A.D. Hoover,et al.  Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response , 2000, IEEE Transactions on Medical Imaging.

[18]  A. Mizuno,et al.  A change of the leading player in flow Visualization technique , 2006, J. Vis..

[19]  Rüdiger von der Heydt,et al.  Approaches to visual cortical function. , 1987 .

[20]  F. Kingdom,et al.  Multiplication in curvature processing. , 2009, Journal of vision.

[21]  N. Petkov,et al.  Motion detection, noise reduction, texture suppression, and contour enhancement by spatiotemporal Gabor filters with surround inhibition , 2007, Biological Cybernetics.

[22]  D. Pollen,et al.  Relationship between spatial frequency selectivity and receptive field profile of simple cells. , 1979, The Journal of physiology.

[23]  I. D. Macleod,et al.  The visibility of gratings: spatial frequency channels or bar-detecting units? , 1974, Vision research.

[24]  D. G. Albrecht,et al.  Visual cortical neurons: are bars or gratings the optimal stimuli? , 1980, Science.

[25]  D. Hubel,et al.  Sequence regularity and geometry of orientation columns in the monkey striate cortex , 1974, The Journal of comparative neurology.

[26]  C. Connor,et al.  Responses to contour features in macaque area V4. , 1999, Journal of neurophysiology.

[27]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[28]  D. Sagi,et al.  Gabor filters as texture discriminator , 1989, Biological Cybernetics.

[29]  George Azzopardi,et al.  A Shape Descriptor Based on Trainable COSFIRE Filters for the Recognition of Handwritten Digits , 2013, CAIP.

[30]  Rama Chellappa,et al.  A new approach to image feature detection with applications , 1996, Pattern Recognit..

[31]  E. Yund,et al.  Responses of striate cortex cells to grating and checkerboard patterns. , 1979, The Journal of physiology.

[32]  Gabriel Cristóbal,et al.  Space and frequency variant image enhancement based on a Gabor representation , 1994, Pattern Recognit. Lett..

[33]  George Azzopardi,et al.  Contour Detection by CORF Operator , 2012, ICANN.

[34]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[35]  Günther Palm,et al.  Artificial Neural Networks and Machine Learning – ICANN 2012 , 2012, Lecture Notes in Computer Science.

[36]  B. S. Manjunath,et al.  A texture descriptor for browsing and similarity retrieval , 2000, Signal Process. Image Commun..

[37]  Max A. Viergever,et al.  Ridge-based vessel segmentation in color images of the retina , 2004, IEEE Transactions on Medical Imaging.

[38]  Tieniu Tan,et al.  Texture edge detection by modelling visual cortical channels , 1995, Pattern Recognit..

[39]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[40]  Nikolay Petkov,et al.  Biologically motivated computationally intensive approaches to image pattern recognition , 1995, Future Gener. Comput. Syst..

[41]  H. Komatsu,et al.  Influence of the Direction of Elemental Luminance Gradients on the Responses of V4 Cells to Textured Surfaces , 2001, The Journal of Neuroscience.

[42]  Sanja Fidler,et al.  Towards Scalable Representations of Object Categories: Learning a Hierarchy of Parts , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Anil K. Jain,et al.  Unsupervised texture segmentation using Gabor filters , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[44]  Ehsanollah Kabir,et al.  Introducing a very large dataset of handwritten Farsi digits and a study on their varieties , 2007, Pattern Recognit. Lett..

[45]  Leslie G. Ungerleider.  Two cortical visual systems , 1982 .

[46]  L. Spillmann,et al.  Visual Perception: The Neurophysiological Foundations , 1989 .

[47]  S. Zeki,et al.  Colour coding in rhesus monkey prestriate cortex. , 1973, Brain research.

[48]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[49]  S. Grossberg,et al.  15 – COMPUTATIONAL THEORIES OF VISUAL PERCEPTION , 1990 .

[50]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[51]  C. Connor,et al.  Population coding of shape in area V4 , 2002, Nature Neuroscience.

[52]  George Azzopardi,et al.  A CORF computational model of a simple cell that relies on LGN input outperforms the Gabor function model , 2012, Biological Cybernetics.

[53]  M. R. Turner,et al.  Texture discrimination by Gabor functions , 1986, Biological Cybernetics.

[54]  Thomas Serre,et al.  A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex , 2005 .

[55]  J. J. Kulikowski,et al.  Fourier analysis and spatial representation in the visual cortex , 1981, Experientia.

[56]  N. Ranganathan,et al.  Gabor filter-based edge detection , 1992, Pattern Recognit..

[57]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  C. Connor,et al.  Shape representation in area V4: position-specific tuning for boundary conformation. , 2001, Journal of neurophysiology.