A comparative evaluation of interactive segmentation algorithms

In this paper we present a comparative evaluation of four popular interactive segmentation algorithms. The evaluation was carried out as a series of user-experiments, in which participants were tasked with extracting 100 objects from a common dataset: 25 with each algorithm, constrained within a time limit of 2min for each object. To facilitate the experiments, a ''scribble-driven'' segmentation tool was developed to enable interactive image segmentation by simply marking areas of foreground and background with the mouse. As the participants refined and improved their respective segmentations, the corresponding updated segmentation mask was stored along with the elapsed time. We then collected and evaluated each recorded mask against a manually segmented ground truth, thus allowing us to gauge segmentation accuracy over time. Two benchmarks were used for the evaluation: the well-known Jaccard index for measuring object accuracy, and a new fuzzy metric, proposed in this paper, designed for measuring boundary accuracy. Analysis of the experimental results demonstrates the effectiveness of the suggested measures and provides valuable insights into the performance and characteristics of the evaluated algorithms.

[1]  Nicholas J. Belkin,et al.  A case for interaction: a study of interactive information retrieval behavior and effectiveness , 1996, CHI.

[2]  J. Cohen,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulas , 1968 .

[3]  Benjamin B. Bederson,et al.  Semi-automatic photo annotation strategies using event based clustering and clothing based person recognition , 2007, Interact. Comput..

[4]  Philippe Salembier,et al.  Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval , 2000, IEEE Trans. Image Process..

[5]  Arnold W. M. Smeulders,et al.  Interaction in the segmentation of medical images: A survey , 2001, Medical Image Anal..

[6]  Raúl Rojas,et al.  SIOX: simple interactive object extraction in still images , 2005, Seventh IEEE International Symposium on Multimedia (ISM'05).

[7]  Song Wang,et al.  New benchmark for image segmentation evaluation , 2007, J. Electronic Imaging.

[8]  Demetri Terzopoulos,et al.  Interactive Medical Image Segmentation with United Snakes , 1999, MICCAI.

[9]  Hui Zhang,et al.  Image segmentation evaluation: A survey of unsupervised methods , 2008, Comput. Vis. Image Underst..

[10]  Tomasz Adamek,et al.  Using contour information and segmentation for object registration, modeling and retrieval , 2006 .

[11]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[12]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[13]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[14]  Jerry L Prince,et al.  Current methods in medical image segmentation. , 2000, Annual review of biomedical engineering.

[15]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[16]  M. Kendall.  A NEW MEASURE OF RANK CORRELATION , 1938 .

[17]  G. Wyszecki,et al.  Color Science Concepts and Methods , 1982 .

[18]  Noel E. O'Connor,et al.  Region-based segmentation of images using syntactic visual features , 2005 .

[19]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[20]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[22]  Jerry L. Prince,et al.  A Survey of Current Methods in Medical Image Segmentation , 1999 .

[23]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[24]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[25]  Noel E. O'Connor,et al.  Image segmentation evaluation using an integrated framework , 2007 .

[26]  Rolf Adams,et al.  Seeded Region Growing , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[28]  Song Wang,et al.  Image-Segmentation Evaluation From the Perspective of Salient Object Extraction , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[29]  Myungcheol Lee,et al.  Graph theory for image analysis: an approach based on the shortest spanning tree , 1986 .

[30]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[31]  Lotfi A. Zadeh,et al.  Fuzzy sets and systems , 1990 .

[32]  Horst Bunke,et al.  Distance Measures for Image Segmentation Evaluation , 2006, EURASIP J. Adv. Signal Process..