Coding Images with Local Features

We develop a qualitative measure for the completeness and complementarity of sets of local features in terms of covering relevant image information. The idea is to interpret feature detection and description as image coding, and relate it to classical coding schemes like JPEG. Given an image, we derive a feature density from a set of local features, and measure its distance to an entropy density computed from the power spectrum of local image patches over scale. Our measure is meant to be complementary to existing ones: After task usefulness of a set of detectors has been determined regarding robustness and sparseness of the features, the scheme can be used for comparing their completeness and assessing effects of combining multiple detectors. The approach has several advantages over a simple comparison of image coverage: It favors response on structured image parts, penalizes features in purely homogeneous areas, and accounts for features appearing at the same location on different scales. Combinations of complementary features tend to converge towards the entropy, while an increased amount of random features does not. We analyse the complementarity of popular feature detectors over different image categories and investigate the completeness of combinations. The derived entropy distribution leads to a new scale and rotation invariant window detector, which uses a fractal image model to take pixel correlations into account. The results of our empirical investigations reflect the theoretical concepts of the detectors.

[1]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[2]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Sanjoy Dasgupta,et al.  Robust Euclidean embedding , 2006, ICML.

[4]  Nassir Navab,et al.  Estimation of Location Uncertainty for Scale Invariant Features Points , 2009, BMVC.

[5]  Tony Lindeberg,et al.  Edge Detection and Ridge Detection with Automatic Scale Selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Azriel Rosenfeld,et al.  Digital Picture Processing , 1976 .

[7]  Aria Nosratinia,et al.  Wavelet-Based Image Coding: An Overview , 1999 .

[8]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[9]  L. Davisson.  Rate-distortion theory and application , 1972 .

[10]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Andrew Zisserman,et al.  An Affine Invariant Salient Region Detector , 2004, ECCV.

[12]  R. Weale.  Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[13]  Bernd Jähne,et al.  Localization accuracy of region detectors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Mads Nielsen,et al.  Feature-Based Image Analysis , 2003, International Journal of Computer Vision.

[15]  Wolfgang Förstner,et al.  Evaluating the Suitability of Feature Detectors for Automatic Image Orientation Systems , 2009, ICVS.

[16]  Terrence J. Sejnowski,et al.  New Directions in Statistical Signal Processing: From Systems to Brains (Neural Information Processing) , 2006 .

[17]  Stefano Soatto,et al.  Edge descriptors for robust wide-baseline correspondence , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[19]  Jean-François Bercher,et al.  Estimating the entropy of a signal with applications , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[20]  Wolfgang Förstner.  Image Preprocessing for Feature Extraction in Digital Intensity, Color and Range Images , 2000 .

[21]  Pietro Perona,et al.  Evaluation of Features Detectors and Descriptors Based on 3D Objects , 2005, ICCV.

[22]  Cordelia Schmid,et al.  Shape recognition with edge-based features , 2003, BMVC.

[23]  J. Bigun.  A structure feature for some image processing applications based on spiral functions , 1990 .

[24]  Peter Green.  The Oxford Handbook of Applied Bayesian Analysis , 2010 .

[25]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[26]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[27]  Richard Szeliski,et al.  Automatic Estimation and Removal of Noise from a Single Image , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  C. McGillem,et al.  Image Registration Error Variance as a Measure of Overlay Quality , 1976, IEEE Transactions on Geoscience Electronics.

[29]  Joachim M. Buhmann,et al.  Histogram clustering for unsupervised image segmentation , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[30]  Wolfgang Förstner,et al.  A Framework for Low Level Feature Extraction , 1994, ECCV.

[31]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Gregory D. Hager,et al.  Image description with features that summarize , 2009, Comput. Vis. Image Underst..

[33]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[34]  Luc Van Gool,et al.  Wide-baseline stereo matching with line segments , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[35]  David Mumford,et al.  Empirical Statistics and Stochastic Models for Visual Signals , 2006 .

[36]  Donald E. McClure.  Image Models in Pattern Theory , 1981 .

[37]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[38]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[39]  Stepán Obdrzálek,et al.  Stable Affine Frames on Isophotes , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[40]  Wolfgang Förstner,et al.  Detecting interpretable and accurate scale-invariant keypoints , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[41]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[42]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[43]  E. B. Andersen,et al.  Information Science and Statistics , 1986 .

[44]  Michael Brady,et al.  Saliency, Scale and Image Description , 2001, International Journal of Computer Vision.

[45]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[46]  Sudeep Sarkar,et al.  Comparison of edge detectors: a methodology and initial study , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.