Contour Completion Without Region Segmentation

Contour completion plays an important role in visual perception, where the goal is to group fragmented low-level edge elements into perceptually coherent and salient contours. Most existing methods for contour completion have focused on pixelwise detection accuracy. In contrast, fewer methods have addressed the global contour closure effect, despite psychological evidences for its importance. This paper proposes a purely contour-based higher order CRF model to achieve contour closure, through local connectedness approximation. This leads to a simplified problem structure, where our higher order inference problem can be transformed into an integer linear program and be solved efficiently. Compared with the methods based on the same bottom-up edge detector, our method achieves a superior contour grouping ability (measured by Rand index), a comparable precision-recall performance, and more visually pleasing results. Our results suggest that contour closure can be effectively achieved in contour domain, in contrast to a popular view that segmentation is essential for this purpose.

[1]  Berthold K. P. Horn The Curve of Least Energy , 1983, TOMS.

[2]  Zhaoping Li,et al.  A Neural Model of Contour Integration in the Primary Visual Cortex , 1998, Neural Computation.

[3]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[5]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[6]  Ullrich Köthe,et al.  Probabilistic image segmentation with closedness constraints , 2011, 2011 International Conference on Computer Vision.

[7]  Steven W. Zucker,et al.  Trace Inference, Curvature Consistency, and Curve Detection , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Gérard G. Medioni,et al.  Inferring global perceptual contours from local features , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Andrew McCallum,et al.  Piecewise training for structured prediction , 2009, Machine Learning.

[10]  Martin Niss,et al.  History of the Lenz–Ising Model 1950–1965: from irrelevance to relevance , 2009 .

[11]  Rolf Nelson,et al.  When does grouping happen? , 2003, Acta psychologica.

[12]  Anton Osokin,et al.  Fast Approximate Energy Minimization with Label Costs , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Xiaofeng Ren,et al.  Discriminatively Trained Sparse Code Gradients for Contour Detection , 2012, NIPS.

[14]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[15]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  Andrew W. Fitzgibbon,et al.  Global stereo reconstruction under second order smoothness priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Pushmeet Kohli,et al.  Inference Methods for CRFs with Co-occurrence Statistics , 2012, International Journal of Computer Vision.

[20]  Vibhav Vineet,et al.  Filter-Based Mean-Field Inference for Random Fields with Higher-Order Terms and Product Label-Spaces , 2012, International Journal of Computer Vision.

[21]  Stephen Gould,et al.  Multiclass pixel labeling with non-local matching constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Lance R. Williams,et al.  Segmentation of Multiple Salient Closed Contours from Real Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Song-Chun Zhu,et al.  Primal sketch: Integrating structure and texture , 2007, Comput. Vis. Image Underst..

[24]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Daniel Cremers,et al.  Shape statistics in kernel space for variational image segmentation , 2003, Pattern Recognit..

[26]  Lance R. Williams,et al.  A Comparison of Measures for Detecting Natural Shapes in Cluttered Backgrounds , 1998, International Journal of Computer Vision.

[27]  Subhransu Maji,et al.  Semantic contours from inverse detectors , 2011, 2011 International Conference on Computer Vision.

[28]  Hiroshi Ishikawa,et al.  Higher-order clique reduction in binary graph cut , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Peter Kovesi,et al.  Image Features from Phase Congruency , 1995 .

[30]  Donald Geman,et al.  Boundary Detection by Constrained Optimization , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Joseph J. Lim,et al.  Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Jitendra Malik,et al.  Contour Continuity in Region Based Image Segmentation , 1998, ECCV.

[33]  Shimon Ullman,et al.  Class-Specific, Top-Down Segmentation , 2002, ECCV.

[34]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[35]  Jun Wang,et al.  Salient closed boundary extraction with ratio contour , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  G. Medioni,et al.  Grouping . ,-, → ,-, into regions , curves , and junctions , 1999 .

[37]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[38]  Jitendra Malik,et al.  Learning Probabilistic Models for Contour Completion in Natural Images , 2008, International Journal of Computer Vision.

[39]  Anil K. Jain Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[40]  Pierre Hansen,et al.  Roof duality, complementation and persistency in quadratic 0–1 optimization , 1984, Math. Program..

[41]  Jianbo Shi,et al.  Contour cut: Identifying salient contours in images by solving a Hermitian eigenvalue problem , 2011, CVPR 2011.

[42]  Jitendra Malik,et al.  Figure/Ground Assignment in Natural Images , 2006, ECCV.

[43]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Gang Song,et al.  Untangling Cycles for Contour Grouping , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[45]  C. Lawrence Zitnick,et al.  The role of image understanding in contour detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Sven J. Dickinson,et al.  Optimal Contour Closure by Superpixel Grouping , 2010, ECCV.

[47]  Daniel Cremers,et al.  The Elastic Ratio: Introducing Curvature Into Ratio-Based Image Segmentation , 2011, IEEE Transactions on Image Processing.

[48]  I Kovács,et al.  A closed curve is much more than an incomplete one: effect of closure in figure-ground segmentation. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[49]  Shimon Ullman,et al.  Structural Saliency: The Detection Of Globally Salient Structures using A Locally Connected Network , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[50]  Jitendra Malik,et al.  Using contours to detect and localize junctions in natural images , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Ian D. Reid,et al.  Nonlinear shape manifolds as shape priors in level set segmentation and tracking , 2011, CVPR 2011.

[52]  T. Lindeberg Scale-Space Theory : A Basic Tool for Analysing Structures at Different Scales , 1994 .

[53]  Nicolai Petkov,et al.  Edge and line oriented contour detection: State of the art , 2011, Image Vis. Comput..

[54]  Mi-Suen Lee,et al.  Grouping ., -, ->, [formula], into Regions, Curves, and Junctions , 1999, Comput. Vis. Image Underst..

[55]  Xavier Bresson,et al.  A Variational Model for Object Segmentation Using Boundary Information and Shape Prior Driven by the Mumford-Shah Functional , 2006, International Journal of Computer Vision.

[56]  Chi-Keung Tang,et al.  Robust estimation of adaptive tensors of curvature by tensor voting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Ronen Basri,et al.  Completion Energies and Scale , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Andrew C. Gallagher,et al.  Inference for order reduction in Markov random fields , 2011, CVPR 2011.

[59]  Jessika Weiss,et al.  Vision Science Photons To Phenomenology , 2016 .

[60]  Hongdong Li,et al.  Connected contours: A new contour completion model that respects the closure effect , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Steven W. Zucker,et al.  Computing Contour Closure , 1996, ECCV.

[62]  C. Lawrence Zitnick,et al.  Structured Forests for Fast Edge Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[63]  Iasonas Kokkinos,et al.  Highly accurate boundary detection and grouping , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[64]  Xiaofeng Ren,et al.  Multi-scale Improves Boundary Detection in Natural Images , 2008, ECCV.

[65]  Toby Sharp,et al.  Image segmentation with a bounding box prior , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[66]  Edward H. Adelson,et al.  Crisp Boundary Detection Using Pointwise Mutual Information , 2014, ECCV.

[67]  Jitendra Malik,et al.  Recovering human body configurations: combining segmentation and recognition , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[68]  Song-Chun Zhu,et al.  Learning Active Basis Model for Object Detection and Recognition , 2010, International Journal of Computer Vision.

[69]  Vladimir Kolmogorov,et al.  Graph cut based image segmentation with connectivity priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[70]  Zhuowen Tu,et al.  Supervised Learning of Edges and Object Boundaries , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[71]  H. Barlow Vision: A computational investigation into the human representation and processing of visual information: David Marr. San Francisco: W. H. Freeman, 1982. pp. xvi + 397 , 1983 .