Scale selection for supervised image segmentation

Finding the right scales for feature extraction is crucial for supervised image segmentation based on pixel classification. There are many scale selection methods in the literature; among them the one proposed by Lindeberg is widely used for image structures such as blobs, edges and ridges. Those schemes are usually unsupervised, as they do not take into account the actual segmentation problem at hand. In this paper, we consider the problem of selecting scales, which aims at an optimal discrimination between user-defined classes in the segmentation. We show the deficiency of the classical unsupervised scale selection paradigms and present a supervised alternative. In particular, the so-called max rule is proposed, which selects a scale for each pixel to have the largest confidence in the classification across the scales. In interpreting the classifier as a complex image filter, we can relate our approach back to Lindeberg's original proposal. In the experiments, the max rule is applied to artificial and real-world image segmentation tasks, which is shown to choose the right scales for different problems and lead to better segmentation results.

[1]  Wiro J. Niessen,et al.  Quantitative comparison of spot detection methods in live-cell fluorescence microscopy imaging , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[2]  Andrew P. Witkin,et al.  Scale-space filtering: A new approach to multi-scale description , 1984, ICASSP.

[3]  J. Koenderink.  The structure of images , 2004, Biological Cybernetics.

[4]  Pietro Perona,et al.  Object class recognition by unsupervised scale-invariant learning , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[6]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Song-Chun Zhu,et al.  Primal sketch: Integrating structure and texture , 2007, Comput. Vis. Image Underst..

[8]  Domènec Puig,et al.  Supervised texture classification by integration of multiple texture methods and evaluation windows , 2007, Image Vis. Comput..

[9]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[11]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[12]  Marco Loog,et al.  Maximum Membership Scale Selection , 2009, MCS.

[13]  J. Koenderink,et al.  Receptive field families , 1990, Biological Cybernetics.

[14]  Jasna Maver,et al.  Self-Similarity and Points of Interest , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Bram van Ginneken,et al.  Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database , 2006, Medical Image Anal..

[16]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[17]  Cordelia Schmid,et al.  A sparse texture representation using local affine regions , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Antonio Criminisi,et al.  TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context , 2007, International Journal of Computer Vision.

[19]  Thomas Brox,et al.  A TV Flow Based Local Scale Measure for Texture Discrimination , 2004, ECCV.

[20]  Nicolas Vandenbroucke,et al.  Color image segmentation by pixel classification in an adapted hybrid color space. Application to soccer image analysis , 2003, Comput. Vis. Image Underst..

[21]  J. Koenderink,et al.  Cartesian differential invariants in scale-space , 1993, Journal of Mathematical Imaging and Vision.

[22]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[23]  Bram van Ginneken,et al.  Comparative study of retinal vessel segmentation methods on a new publicly available database , 2004, SPIE Medical Imaging.

[24]  R. Manmatha,et al.  A scale space approach for automatically segmenting words from historical handwritten documents , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  James L. Crowley,et al.  Local Scale Selection for Gaussian Based Description Techniques , 2000, ECCV.

[26]  Robert T. Collins,et al.  Mean-shift blob tracking through scale space , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[28]  Fredrik Bergholm,et al.  Edge Focusing , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Marco Loog,et al.  Supervised Scale-Invariant Segmentation (and Detection) , 2011, SSVM.

[30]  Wiro J. Niessen,et al.  Quantitative Comparison of Spot Detection Methods in Fluorescence Microscopy , 2010, IEEE Transactions on Medical Imaging.

[31]  Zhiwen Yu,et al.  A modified support vector machine and its application to image segmentation , 2011, Image Vis. Comput..

[32]  J. Koenderink,et al.  Representation of local geometry in the visual system , 1987, Biological Cybernetics.

[33]  J. Weickert,et al.  Information Measures in Scale-Spaces , 1999, IEEE Trans. Inf. Theory.

[34]  Robert P. W. Duin,et al.  Classifier Conditional Posterior Probabilities , 1998, SSPR/SPR.

[35]  Max A. Viergever,et al.  Scale and the differential structure of images , 1992, Image Vis. Comput..

[36]  Martial Hebert,et al.  Discriminative random fields: a discriminative framework for contextual interaction in classification , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[37]  Song-Chun Zhu,et al.  From Information Scaling of Natural Images to Regimes of Statistical Models , 2007 .

[38]  Max A. Viergever,et al.  Ridge-based vessel segmentation in color images of the retina , 2004, IEEE Transactions on Medical Imaging.

[39]  Dorin Comaniciu,et al.  Scale selection for anisotropic scale-space: application to volumetric tumor characterization , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[40]  Donald Geman,et al.  Boundary Detection by Constrained Optimization , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[42]  Andrew P. Witkin,et al.  Scale-Space Filtering , 1983, IJCAI.

[43]  Bram van Ginneken,et al.  Segmentation of the posterior ribs in chest radiographs using iterated contextual pixel classification , 2006, IEEE Transactions on Medical Imaging.

[44]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  Tony Lindeberg,et al.  Edge Detection and Ridge Detection with Automatic Scale Selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[46]  Bram van Ginneken,et al.  Filter learning: Application to suppression of bony structures from chest radiographs , 2006, Medical Image Anal..

[47]  Cordelia Schmid,et al.  Local Grayvalue Invariants for Image Retrieval , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[49]  Michael Brady,et al.  Saliency, Scale and Image Description , 2001, International Journal of Computer Vision.

[50]  Bo Markussen,et al.  Maximum Likely Scale Estimation , 2005, DSSCV.

[51]  Tony F. Chan,et al.  Scale Recognition, Regularization Parameter Selection, and Meyer's G Norm in Total Variation Regularization , 2006, Multiscale Model. Simul..

[52]  Phil Brodatz,et al.  Textures: A Photographic Album for Artists and Designers , 1966 .

[53]  Azriel Rosenfeld,et al.  Image Segmentation by Pixel Classification in (Gray Level, Edge Value) Space , 1978, IEEE Transactions on Computers.

[54]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[55]  Kunio Doi,et al.  Image-processing technique for suppressing ribs in chest radiographs by means of massive training artificial neural network (MTANN) , 2006, IEEE Transactions on Medical Imaging.

[56]  Bin Luo,et al.  Local Scale Measure from the Topographic Map and Application to Remote Sensing Images , 2009, Multiscale Model. Simul..

[57]  Ole Fogh Olsen,et al.  Segmenting Articular Cartilage Automatically Using a Voxel Classification Approach , 2007, IEEE Transactions on Medical Imaging.

[58]  Marco Loog,et al.  Supervised dimensionality reduction and contextual pattern recognition in medical image processing , 2004 .

[59]  Max A. Viergever,et al.  Kohonen networks for multiscale image segmentation , 1994, Image Vis. Comput..

[60]  David G. Stork,et al.  Pattern Classification , 1973 .

[61]  Anil K. Jain,et al.  Texture classification and segmentation using multiresolution simultaneous autoregressive models , 1992, Pattern Recognit..

[62]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  Bart M. ter Haar Romeny,et al.  Front-End Vision and Multi-Scale Image Analysis , 2003, Computational Imaging and Vision.

[64]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[65]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[66]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[67]  Miguel Á. Carreira-Perpiñán,et al.  Multiscale conditional random fields for image labeling , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[68]  W. E. Blanz,et al.  Image segmentation by pixel classification , 1981, Pattern Recognit..

[69]  Frank Nielsen,et al.  Texture Regimes for Entropy-Based Multiscale Image Analysis , 2010, ECCV.

[70]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[71]  Stefano Soatto,et al.  The scale of a texture and its application to segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[72]  Rama Chellappa,et al.  Learning Texture Discrimination Rules in a Multiresolution System , 1994, IEEE Trans. Pattern Anal. Mach. Intell..