Statistics of natural image categories

In this paper we study the statistical properties of natural images belonging to different categories and their relevance for scene and object categorization tasks. We discuss how second-order statistics are correlated with image categories, scene scale and objects. We propose how scene categorization could be computed in a feedforward manner in order to provide top-down and contextual information very early in the visual processing chain. Results show how visual categorization based directly on low-level features, without grouping or segmentation stages, can benefit object localization and identification. We show how simple image statistics can be used to predict the presence and absence of objects in the scene before exploring the image.

[1]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[2]  M. Potter.  Meaning in visual search. , 1975, Science.

[3]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[4]  Jeffrey A. Sloan,et al.  Spatial frequency analysis of the visual environment: Anisotropy and the carpentered environment hypothesis , 1978, Vision Research.

[5]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[6]  B. Tversky,et al.  Categories of environmental scenes , 1983, Cognitive Psychology.

[7]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[8]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[9]  G. J. Burton,et al.  Color and spatial structure in natural scenes. , 1987, Applied optics.

[10]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[11]  Ian Craw,et al.  Parameterising Images for Recognition and Reconstruction , 1991, BMVC.

[12]  Ian Craw,et al.  Parameterising Images for Recognition and Reconstruction , 1991 .

[13]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[14]  Minami Ito,et al.  Columns for visual features of objects in monkey inferotemporal cortex , 1992, Nature.

[15]  D. Tolhurst,et al.  Amplitude spectra of natural images. , 1992, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[16]  Leslie S. Smith,et al.  The principal components of natural images , 1992 .

[17]  Joseph J. Atick,et al.  What Does the Retina Know about Natural Scenes? , 1992, Neural Computation.

[18]  D. Tolhurst,et al.  Amplitude spectra of natural images , 1992 .

[19]  K Tanaka,et al.  Neuronal mechanisms of object recognition. , 1993, Science.

[20]  D. V. van Essen,et al.  Selectivity for polar, hyperbolic, and Cartesian gratings in macaque visual cortex. , 1993, Science.

[21]  H. Shouval,et al.  Localized principal components of natural images-an analytic solution , 1994 .

[22]  D. Ruderman.  The statistics of natural images , 1994 .

[23]  Rosalind W. Picard,et al.  Texture orientation for sorting photos "at a glance" , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[24]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[25]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[26]  R. Baddeley,et al.  Searching for filters with 'interesting' output distributions: an uninteresting direction to explore? , 1996, Network.

[27]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[28]  J. V. van Hateren,et al.  Modelling the power spectra of natural images: statistics and information. , 1996, Vision research.

[29]  W. Richards,et al.  Model structure and reliable inference , 1996 .

[30]  Juyang Weng,et al.  Using Discriminant Eigenfeatures for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[32]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[33]  J. H. van Hateren,et al.  Modelling the Power Spectra of Natural Images: Statistics and Information , 1996, Vision Research.

[34]  A. Oliva,et al.  Coarse Blobs or Fine Edges? Evidence That Information Diagnosticity Changes the Perception of Complex Visual Stimuli , 1997, Cognitive Psychology.

[35]  Roland Baddeley,et al.  The Correlational Structure of Natural Images and the Calibration of Spatial Representations , 1997, Cogn. Sci..

[36]  Daniel L. Ruderman,et al.  Origins of scaling in natural images , 1996, Vision Research.

[37]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[38]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[39]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[40]  Neil Gershenfeld,et al.  The nature of mathematical modeling , 1998 .

[41]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[42]  Anil K. Jain,et al.  On image classification: city images vs. landscapes , 1998, Pattern Recognit..

[43]  Aude Oliva,et al.  Global semantic classification of scenes using power spectrum templates , 1999 .

[44]  David J. Field,et al.  Wavelets, vision and the statistics of natural scenes , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[45]  Anil K. Jain,et al.  Content-based hierarchical classification of vacation images , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[46]  J. Henderson,et al.  The effects of semantic consistency on eye movements during complex scene viewing , 1999 .

[47]  Aude Oliva,et al.  Classification of scene photographs from local orientations features , 2000, Pattern Recognit. Lett..

[48]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[49]  J. Gallant.  The Neural Representation of Shape , 2000 .

[50]  Bruno A. Olshausen,et al.  Learning Sparse Image Codes using a Wavelet Pyramid Architecture , 2000, NIPS.

[51]  A. Oliva,et al.  Diagnostic Colors Mediate Scene Recognition , 2000, Cognitive Psychology.

[52]  David A. Forsyth,et al.  Learning the semantics of words and pictures , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[53]  Rufin van Rullen,et al.  Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex , 2001, Neural Computation.

[54]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[55]  Antonio Torralba,et al.  Contextual Modulation of Target Saliency , 2001, NIPS.

[56]  Antonio Torralba,et al.  Statistical Context Priming for Object Detection , 2001, ICCV.

[57]  C. Connor,et al.  Three-dimensional orientation tuning in macaque area V4 , 2002, Nature Neuroscience.

[58]  Antonio Torralba,et al.  Depth Estimation from Image Structure , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  Michel Vidal-Naquet,et al.  Visual features of intermediate complexity and their use in classification , 2002, Nature Neuroscience.

[60]  Guillaume A. Rousselet,et al.  Parallel processing in high-level categorization of natural images , 2002, Nature Neuroscience.

[61]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[62]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[63]  Bernt Schiele,et al.  Recognition without Correspondence using Multidimensional Receptive Field Histograms , 2004, International Journal of Computer Vision.

[64]  Antonio Torralba,et al.  Contextual Priming for Object Detection , 2003, International Journal of Computer Vision.