Learning to detect natural image boundaries using local brightness, color, and texture cues

The goal of this work is to accurately detect and localize boundaries in natural scenes using local image measurements. We formulate features that respond to characteristic changes in brightness, color, and texture associated with natural boundaries. In order to combine the information from these features in an optimal way, we train a classifier using human labeled images as ground truth. The output of this classifier provides the posterior probability of a boundary at each image location and orientation. We present precision-recall curves showing that the resulting detector significantly outperforms existing approaches. Our two main results are 1) that cue combination can be performed adequately with a simple linear model and 2) that a proper, explicit treatment of texture is required to detect boundaries in natural images.

[1]  C. Mallows.  A Note on Asymptotic Joint Normality , 1972 .

[2]  I.E. Abdou,et al.  Quantitative design and evaluation of enhancement/thresholding edge detectors , 1979, Proceedings of the IEEE.

[3]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[5]  Paolo Toth,et al.  Algorithms and codes for the assignment problem , 1988 .

[6]  D. Burr,et al.  Feature detection in human vision: a phase-dependent energy model , 1988, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[7]  P Perona,et al.  Preattentive texture discrimination with early vision mechanisms. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[8]  Jitendra Malik,et al.  Detecting and localizing edges composed of steps, peaks and roofs , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[9]  Mark K. Goldberg,et al.  An efficient parallel algorithm that finds independent sets of guaranteed size , 1993, SODA '90.

[10]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[11]  David Mumford,et al.  Filtering, Segmentation and Depth , 1993, Lecture Notes in Computer Science.

[12]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[13]  Andrew V. Goldberg,et al.  An efficient cost scaling algorithm for the assignment problem , 1995, Math. Program..

[14]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[15]  Andrew V. Goldberg,et al.  On Implementing Push-Relabel Method for the Maximum Flow Problem , 1995, IPCO.

[16]  Josée Rivest,et al.  Localizing contours defined by more than one attribute , 1996, Vision Research.

[17]  C. Tomasi.  Coalescing Texture Descriptors , 1996 .

[18]  Lance R. Williams,et al.  Stochastic Completion Fields: A Neural Model of Illusory Contour Shape and Salience , 1997, Neural Computation.

[19]  Joachim M. Buhmann,et al.  Non-parametric similarity measures for unsupervised texture segmentation and image retrieval , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Andrew V. Goldberg,et al.  On Implementing the Push—Relabel Method for the Maximum Flow Problem , 1997, Algorithmica.

[21]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[22]  Sean Dougherty,et al.  Edge detector evaluation using empirical ROC curves , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[23]  Carlo Tomasi,et al.  Color edge detection with the compass operator , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[24]  Joachim M. Buhmann,et al.  Empirical evaluation of dissimilarity measures for color and texture , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[25]  Alan L. Yuille,et al.  Fundamental bounds on edge detection: an information theoretic evaluation of different edge cues , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[26]  Carlo Tomasi,et al.  Corner detection in textured color images , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[27]  Jianbo Shi,et al.  Learning Segmentation by Random Walks , 2000, NIPS.

[28]  Joachim M. Buhmann,et al.  On learning texture edge detectors , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[29]  Peter J. Bickel,et al.  The Earth Mover's distance is the Mallows distance: some insights from statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[30]  Harry Shum,et al.  Image segmentation by data driven Markov chain Monte Carlo , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[31]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[32]  Joachim M. Buhmann,et al.  Parametric Distributional Clustering for Image Segmentation , 2002, ECCV.

[33]  Jitendra Malik,et al.  A Probabilistic Multi-scale Model for Contour Completion Based on Image Statistics , 2002, ECCV.

[34]  Jitendra Malik,et al.  Learning to Detect Natural Image Boundaries Using Brightness and Texture , 2002, NIPS.

[35]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Jitendra Malik,et al.  Contour and Texture Analysis for Image Segmentation , 2001, International Journal of Computer Vision.

[37]  D. Sagi,et al.  Gabor filters as texture discriminator , 1989, Biological Cybernetics.

[38]  A. Volgenant,et al.  A shortest augmenting path algorithm for dense and sparse linear assignment problems , 1987, Computing.