An Information-Based Measure for Grouping Quality

We propose a method for measuring the quality of a grouping result, based on the following observation: a better grouping result provides more information about the true, unknown grouping. The amount of information is evaluated using an automatic procedure, relying on the given hypothesized grouping, which generates (homogeneity) queries about the true grouping and answers them using an oracle. The process terminates once the queries suffice to specify the true grouping. The number of queries is a measure of the hypothesis non-informativeness. A relation between the query count and the (probabilistically characterized) uncertainty of the true grouping, is established and experimentally supported. The proposed information-based quality measure is free from arbitrary choices, uniformly treats different types of grouping errors, and does not favor any algorithm. We also found that it approximates human judgment better than other methods and gives better results when used to optimize a segmentation algorithm.

[1]  Peter Meer,et al.  Input Guided Performance Evaluation , 1998, Theoretical Foundations of Computer Vision.

[2]  Horst Bunke,et al.  Classes of cost functions for string edit distance , 2006, Algorithmica.

[3]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[4]  Ilya Blayvas,et al.  Efficient computation of adaptive threshold surfaces for image binarization , 2006, Pattern Recognit..

[5]  Henk L. Muller,et al.  Evaluating Image Segmentation Algorithms Using the Pareto Front , 2002, ECCV.

[6]  Michael Lindenbaum,et al.  On the Performance of Connected Components Grouping , 2004, International Journal of Computer Vision.

[7]  A.W.M. Smeulders,et al.  Requirements for generic grouping in vision and an algorithm , 2001 .

[8]  Robert M. Haralick.  Performance Characterization in Computer Vision , 1992, BMVC.

[9]  Sudeep Sarkar,et al.  A Framework for Performance Characterization of Intermediate-Level Grouping Modules , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Hugues Benoit-Cattin,et al.  Scalable discrepancy measures for segmentation evaluation , 2002, Proceedings. International Conference on Image Processing.

[11]  Martin D. Levine,et al.  Dynamic Measurement of Computer Generated Image Segmentations , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Rakesh Mohan,et al.  Book review: PERCEPTUAL ORGANIZATION AND VISUAL RECOGNITION by David G. Lowe (Kluwer Academic Publishers) , 1987, SGAR.

[13]  Jaakko Hintikka,et al.  On the Logic of Perception , 1969 .

[14]  M. Beauchemin,et al.  On the Hausdorff Distance Used for the Evaluation of Segmentation Results , 1998 .

[15]  David G. Lowe,et al.  Perceptual Organization and Visual Recognition , 2012 .

[16]  A. Witkin,et al.  On the Role of Structure in Vision , 1983 .

[17]  Max A. Viergever,et al.  A methodology for the validation of image segmentation methods , 1992, [1992] Proceedings Fifth Annual IEEE Symposium on Computer-Based Medical Systems.

[18]  Touradj Ebrahimi,et al.  Objective evaluation of segmentation quality using spatio-temporal context , 2002, Proceedings. International Conference on Image Processing.

[19]  W. F. orstner.  Pros and Cons Against Performance Characterization of Vision Algorithms , 1996 .

[20]  Michael Lindenbaum,et al.  A Generic Grouping Algorithm and Its Quantitative Analysis , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  David W. Jacobs,et al.  Robust and Efficient Detection of Salient Convex Groups , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Max A. Viergever,et al.  Model-based Evaluation of Image Segmentation Methods , 1998, Theoretical Foundations of Computer Vision.

[23]  Lance R. Williams,et al.  A Comparison of Measures for Detecting Natural Shapes in Cluttered Backgrounds , 1998, International Journal of Computer Vision.

[24]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[25]  Claude E. Shannon,et al.  Prediction and Entropy of Printed English , 1951 .

[26]  I. Rock.  The Logic of Perception , 1983 .

[27]  Y. J. Zhang,et al.  A survey on evaluation methods for image segmentation , 1996, Pattern Recognit..

[28]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .

[29]  Andrew P. Witkin,et al.  What Is Perceptual Organization For? , 1983, IJCAI.