Learning Probabilistic Models for Contour Completion in Natural Images

Abstract Using a large set of human segmented natural images, we study the statistics of region boundaries. We observe several power law distributions which likely arise from both multi-scale structure within individual objects and from arbitrary viewing distance. Accordingly, we develop a scale-invariant representation of images from the bottom up, using a piecewise linear approximation of contours and constrained Delaunay triangulation to complete gaps. We model curvilinear grouping on top of this graphical/geometric structure using a conditional random field to capture the statistics of continuity and different junction types. Quantitative evaluations on several large datasets show that our contour grouping algorithm consistently dominates and significantly improves on local edge detection.

[1]  M. Wertheimer.  Laws of organization in perceptual forms. , 1938 .

[2]  F. Attneave.  Some informational aspects of visual perception. , 1954, Psychological review.

[3]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[4]  G. Kanizsa,et al.  Organization in Vision: Essays on Gestalt Perception , 1979 .

[5]  R. von der Heydt,et al.  Illusory contours and cortical neuron responses. , 1984, Science.

[6]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Gunilla Borgefors,et al.  Hierarchical Chamfer Matching: A Parametric Edge Matching Algorithm , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Shimon Ullman,et al.  Structural Saliency: The Detection Of Globally Salient Structures using A Locally Connected Network , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[10]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[11]  Steven W. Zucker,et al.  Trace Inference, Curvature Consistency, and Curve Detection , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  P. Kellman,et al.  A theory of visual interpolation in object perception , 1991, Cognitive Psychology.

[13]  Daniel P. Huttenlocher,et al.  Finding convex edge groupings in an image , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  William Bialek,et al.  Statistics of Natural Images: Scaling in the Woods , 1993, NIPS.

[15]  Rüdiger von der Heydt,et al.  A computational model of neural contour processing: Figure-ground segregation and illusory contours , 1993, 1993 (4th) International Conference on Computer Vision.

[16]  W. Clem Karl,et al.  Multiscale representations of Markov random fields , 1993, IEEE Trans. Signal Process..

[17]  W. Clem Karl,et al.  Multiscale representations of Markov random fields , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[18]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  D. Ruderman.  The statistics of natural images , 1994 .

[20]  C. Bajaj.  Algebraic Geometry and its Applications , 1994 .

[21]  D. Mumford.  Elastica and Computer Vision , 1994 .

[22]  Lance R. Williams,et al.  Stochastic Completion Fields: A Neural Model of Illusory Contour Shape and Salience , 1995, Neural Computation.

[23]  Stan Z. Li,et al.  Markov Random Field Modeling in Computer Vision , 1995, Computer Science Workbench.

[24]  Steven W. Zucker,et al.  Computing Contour Closure , 1996, ECCV.

[25]  Jonathan Richard Shewchuk,et al.  Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator , 1996, WACG.

[26]  Yair Weiss,et al.  Interpreting Images by Propagating Bayesian Beliefs , 1996, NIPS.

[27]  Daniel L. Ruderman,et al.  Origins of scaling in natural images , 1996, Vision Research.

[28]  Song-Chun Zhu,et al.  FRAME: filters, random fields, and minimax entropy towards a unified theory for texture modeling , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Ronen Basri,et al.  Completion energies and scale , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Lance R. Williams,et al.  A Comparison of Measures for Detecting Natural Shapes in Cluttered Backgrounds , 1998, ECCV.

[31]  Song-Chun Zhu,et al.  Embedding Gestalt Laws in Markov Random Fields , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  S. Palmer.  Vision Science : Photons to Phenomenology , 1999 .

[33]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[34]  Yann Gousseau,et al.  Scales in Natural Images and a Consequence on their Bounded Variation Norm , 1999, Scale-Space.

[35]  Alan L. Yuille,et al.  Fundamental bounds on edge detection: an information theoretic evaluation of different edge cues , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[36]  Kaleem Siddiqi,et al.  Contour Fragment Grouping and Shared, Simple Occluders , 1999, Comput. Vis. Image Underst..

[37]  David Mumford,et al.  Statistics of natural images and models , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[38]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[39]  Yair Weiss,et al.  Correctness of Local Probability Propagation in Graphical Models with Loops , 2000, Neural Computation.

[40]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[41]  Pedro F. Felzenszwalb.  Learning models for object recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[42]  Lance R. Williams,et al.  Orientation, Scale, and Discontinuity as Emergent Properties of Illusory Contour Shape , 1998, Neural Computation.

[43]  Jeffrey S. Perry,et al.  Edge co-occurrence in natural images predicts contour grouping performance , 2001, Vision Research.

[44]  Ian H. Jermyn,et al.  Globally Optimal Regions and Boundaries as Minimum Ratio Weight Cycles , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Ann B. Lee.  Occlusion Models for Natural Images : A Statistical Study of a Scale-Invariant Dead Leaves Model , 2001 .

[46]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[47]  Robert Haimes,et al.  Multiscale and Multiresolution Methods , 2002 .

[48]  Xiaoming Huo,et al.  Beamlets and Multiscale Image Analysis , 2002 .

[49]  Shimon Ullman,et al.  Class-Specific, Top-Down Segmentation , 2002, ECCV.

[50]  J. Elder,et al.  Ecological statistics of Gestalt laws for the perceptual organization of contours. , 2002, Journal of vision.

[51]  Jitendra Malik,et al.  A Probabilistic Multi-scale Model for Contour Completion Based on Image Statistics , 2002, ECCV.

[52]  X. Jin.  Factor graphs and the Sum-Product Algorithm , 2002 .

[53]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Tomer Hertz,et al.  Learning and inferring image segmentations using the GBP typical cut algorithm , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[55]  Yizhou Yu,et al.  Two-Level Image Segmentation Based on Region and Edge Integration , 2003, DICTA.

[56]  Zhuowen Tu,et al.  Image Parsing: Unifying Segmentation, Detection, and Recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[57]  R. Zemel,et al.  Multiscale conditional random fields for image labeling , 2004, CVPR 2004.

[58]  Jitendra Malik,et al.  Recovering human body configurations: combining segmentation and recognition , 2004, CVPR 2004.

[59]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Miguel Á. Carreira-Perpiñán,et al.  Multiscale conditional random fields for image labeling , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[61]  Jitendra Malik,et al.  Recovering human body configurations: combining segmentation and recognition , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[62]  Song-Chun Zhu,et al.  Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling , 1998, International Journal of Computer Vision.

[63]  Jitendra Malik,et al.  Recovering human body configurations using pairwise constraints between parts , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[64]  Martial Hebert,et al.  Discriminative Random Fields , 2006, International Journal of Computer Vision.

[65]  Xiaofeng Ren,et al.  Learning and Matching Line Aspects for Articulated Objects , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[66]  Fengxia Li,et al.  Discriminative Random Fields for Behavior Modeling , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.