Graph Cuts and Efficient N-D Image Segmentation

Combinatorial graph cut algorithms have been successfully applied to a wide range of problems in vision and graphics. This paper focusses on possibly the simplest application of graph-cuts: segmentation of objects in image data. Despite its simplicity, this application epitomizes the best features of combinatorial graph cuts methods in vision: global optima, practical efficiency, numerical robustness, ability to fuse a wide range of visual cues and constraints, unrestricted topological properties of segments, and applicability to N-D problems. Graph cuts based approaches to object extraction have also been shown to have interesting connections with earlier segmentation methods such as snakes, geodesic active contours, and level-sets. The segmentation energies optimized by graph cuts combine boundary regularization with region-based properties in the same fashion as Mumford-Shah style functionals. We present motivation and detailed technical description of the basic combinatorial optimization framework for image segmentation via s/t graph cuts. After the general concept of using binary graph cut algorithms for object segmentation was first proposed and tested in Boykov and Jolly (2001), this idea was widely studied in computer vision and graphics communities. We provide links to a large number of known extensions based on iterative parameter re-estimation and learning, multi-scale or hierarchical approaches, narrow bands, and other techniques for demanding photo, video, and medical applications.

[1]  Andrew V. Goldberg,et al.  A new approach to the maximum flow problem , 1986, STOC '86.

[2]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[3]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[4]  Ramesh C. Jain,et al.  Using Dynamic Programming for Solving Variational Problems in Vision , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[6]  Laurent D. Cohen,et al.  On active contour models and balloons , 1991, CVGIP Image Underst..

[7]  Richard M. Leahy,et al.  An Optimal Graph Theoretic Approach to Data Clustering: Theory and Its Application to Image Segmentation , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Colin Studholme,et al.  Hierarchical Segmentation Satisfying Constraints , 1994, BMVC.

[9]  Emanuele Trucco,et al.  Computer and Robot Vision , 1995 .

[10]  Alok Gupta,et al.  Dynamic Programming for Detecting, Tracking, and Matching Deformable Contours , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Ingemar J. Cox,et al.  "Ratio regions": a technique for image segmentation , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[12]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Anthony J. Yezzi,et al.  A geometric snake model for segmentation of medical imagery , 1997, IEEE Transactions on Medical Imaging.

[14]  William J. Cook,et al.  Combinatorial Optimization: Cook/Combinatorial , 1997 .

[15]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  William J. Cook,et al.  Combinatorial optimization , 1997 .

[17]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[18]  Davi Geiger,et al.  Occlusions, Discontinuities, and Epipolar Lines in Stereo , 1998, ECCV.

[19]  William A. Barrett,et al.  Interactive Segmentation with Intelligent Scissors , 1998, Graph. Model. Image Process..

[20]  Dorit S. Hochbaum,et al.  The Pseudoflow Algorithm and the Pseudoflow-Based Simplex for the Maximum Flow Problem , 1998, IPCO.

[21]  Davi Geiger,et al.  Segmentation by grouping junctions , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[22]  Michael Isard,et al.  Active Contours , 2000, Springer London.

[23]  Olga Veksler,et al.  Markov random fields with efficient approximations , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[24]  Jayaram K. Udupa,et al.  User-Steered Image Segmentation Paradigms: Live Wire and Live Lane , 1998, Graph. Model. Image Process..

[25]  Kazuo Murota,et al.  Discrete convex analysis , 1998, Math. Program..

[26]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[27]  Ian H. Jermyn,et al.  Globally optimal regions and boundaries , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[28]  J. A. Sethian,et al.  Fast Marching Methods , 1999, SIAM Rev..

[29]  Richard Szeliski,et al.  An Experimental Comparison of Stereo Algorithms , 1999, Workshop on Vision Algorithms.

[30]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods , 1999 .

[31]  IEEE conference on computer vision and pattern recognition , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[32]  Olga Veksler,et al.  Image segmentation by nested cuts , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[33]  Y.Y. Boykov,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[34]  Kaleem Siddiqi,et al.  Flux Maximizing Geometric Flows , 2001, ICCV.

[35]  G. Sapiro,et al.  Geometric partial differential equations and image analysis [Book Reviews] , 2001, IEEE Transactions on Medical Imaging.

[36]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[37]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[38]  Jon M. Kleinberg,et al.  An Impossibility Theorem for Clustering , 2002, NIPS.

[39]  Vladimir Kolmogorov,et al.  Computing geodesics and minimal surfaces via graph cuts , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[40]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[41]  Ning Xu,et al.  Object segmentation using graph cuts based active contours , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[42]  Hiroshi Ishikawa,et al.  Exact Optimization for Markov Random Fields with Convex Priors , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  S. Osher,et al.  Geometric Level Set Methods in Imaging, Vision, and Graphics , 2011, Springer New York.

[44]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[45]  Alfred M. Bruckstein,et al.  Regularized Laplacian Zero Crossings as Optimal Edge Integrators , 2003, International Journal of Computer Vision.

[46]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  S. Gortler,et al.  A Discrete Global Minimization Algorithm for Continuous Variational Problems , 2004 .

[48]  Harry Shum,et al.  Lazy snapping , 2004, ACM Trans. Graph..

[49]  Laurent D. Cohen,et al.  Global Minimum for Active Contour Models: A Minimal Path Approach , 1997, International Journal of Computer Vision.

[50]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[52]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[53]  Patrick Pérez,et al.  Interactive Image Segmentation Using an Adaptive GMMRF Model , 2004, ECCV.

[54]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[55]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, ACM Trans. Graph..

[56]  Vladimir Kolmogorov,et al.  What metrics can be approximated by geo-cuts, or global optimization of length/area and flux , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[57]  Harry Shum,et al.  Video object cut and paste , 2005, ACM Trans. Graph..

[58]  Andrew Blake,et al.  Digital tapestry [automatic image synthesis] , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[59]  Roberto Cipolla,et al.  Multi-view stereo via volumetric graph-cuts , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[60]  Leo Grady,et al.  A multilevel banded graph cuts method for fast image segmentation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[61]  Andrew Blake,et al.  Bi-layer segmentation of binocular stereo video , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[62]  Andrew Zisserman,et al.  OBJ CUT , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[63]  P. Kohli,et al.  Efficiently solving dynamic Markov random fields using graph cuts , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[64]  Leo Grady,et al.  Multilabel random walker image segmentation using prior models , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[65]  Victor S. Lempitsky,et al.  Oriented Visibility for Multiview Reconstruction , 2006, ECCV.

[66]  Daniel Cremers,et al.  Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation , 2006, International Journal of Computer Vision.

[67]  Daniel Cremers,et al.  An Integral Solution to Surface Evolution PDEs Via Geo-cuts , 2006, ECCV.

[68]  Pushmeet Kohli,et al.  PoseCut: Simultaneous Segmentation and 3D Pose Estimation of Humans Using Dynamic Graph-Cuts , 2006, ECCV.

[69]  Xiaodong Wu,et al.  Optimal Surface Segmentation in Volumetric Images-A Graph-Theoretic Approach , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  Hugues Talbot,et al.  Globally minimal surfaces by continuous maximal flows , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[71]  Olivier Juan,et al.  Active Graph Cuts , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[72]  Olga Veksler,et al.  Graph Cuts in Vision and Graphics: Theories and Applications , 2006, Handbook of Mathematical Models in Computer Vision.

[73]  Pushmeet Kohli,et al.  Measuring Uncertainty in Graph Cut Solutions - Efficiently Computing Min-marginal Energies Using Dynamic Graph Cuts , 2006, ECCV.

[74]  Daniel Cremers,et al.  Dynamical statistical shape priors for level set-based tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[75]  Gareth Funka-Lea,et al.  Automatic heart isolation for CT coronary visualization using graph-cuts , 2006, 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006..