Segmentation from a box

Drawing a box around an intended segmentation target has become both a popular user interface and a common output for learning-driven detection algorithms. Despite the ubiquity of using a box to define a segmentation target, it is unclear in the literature whether a box is sufficient to define a unique segmentation or whether segmentation from a box is ill-posed without higher-level (semantic) knowledge of the intended target. We examine this issue by conducting a study of 14 subjects who are asked to segment a boxed target in a set of 50 real images for which they have no semantic attachment. We find that the subjects do indeed perceive and trace almost the same segmentations as each other, despite the inhomogeneity of the image intensities, irregular shapes of the segmentation targets and weakness of the target boundaries. Since the subjects produce the same segmentation, we conclude that the problem is well-posed and then provide a new segmentation algorithm from a box which achieves results close to the perceived target.

[1]  Guillermo Sapiro,et al.  A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[2]  Leo Grady,et al.  Isoperimetric graph partitioning for image segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Guillermo Sapiro,et al.  Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting , 2009, International Journal of Computer Vision.

[4]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Lattre de Tassigny.  Boundary Extraction in Natural Images Using Ultrametric Contour Maps , 2006 .

[6]  Antonio Torralba,et al.  Sharing Visual Features for Multiclass and Multiview Object Detection , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  B. Ginneken,et al.  3D Segmentation in the Clinic: A Grand Challenge , 2007 .

[8]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[9]  Jayaram K. Udupa,et al.  User-Steered Image Segmentation Paradigms: Live Wire and Live Lane , 1998, Graph. Model. Image Process..

[10]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[11]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[12]  C. Mallows,et al.  A Method for Comparing Two Hierarchical Clusterings , 1983 .

[13]  Hugues Talbot,et al.  Globally minimal surfaces by continuous maximal flows , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Yannis Avrithis,et al.  Semantic Image Segmentation and Object Labeling , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[16]  Nam Ik Cho,et al.  Rectification of figures and photos in document images using bounding box interface , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Toby Sharp,et al.  Image segmentation with a bounding box prior , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[18]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods , 1999 .

[19]  Marie-Pierre Jolly,et al.  Automatic femur segmentation and condyle line detection in 3D MR scans for alignment of high resolution MR , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[20]  Horst Bischof,et al.  Interactive Multi-label Segmentation , 2010, ACCV.

[21]  Leo Grady,et al.  Weights and Topology: A Study of the Effects of Graph Construction on 3D Image Segmentation , 2008, MICCAI.

[22]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[23]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[24]  William A. Barrett,et al.  Interactive Segmentation with Intelligent Scissors , 1998, Graph. Model. Image Process..

[25]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[26]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[27]  Daniel Cremers,et al.  TVSeg - Interactive Total Variation Based Image Segmentation , 2008, BMVC.

[28]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Jaime S. Cardoso,et al.  Toward a generic evaluation of image segmentation , 2005, IEEE Transactions on Image Processing.

[30]  M. Stella Atkins,et al.  A Fully Automatic Random Walker Segmentation for Skin Lesions in a Supervised Setting , 2009, MICCAI.

[31]  Leo Grady,et al.  A Seeded Image Segmentation Framework Unifying Graph Cuts And Random Walker Which Yields A New Algorithm , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[32]  Tony F. Chan,et al.  A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model , 2002, International Journal of Computer Vision.

[33]  Leo Grady,et al.  Interactive image segmentation via minimization of quadratic energies on directed graphs , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Martial Hebert,et al.  Toward Objective Evaluation of Image Segmentation Algorithms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[36]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.