Image/Video Segmentation: Current Status, Trends, and Challenges

Segmentation plays an important role in digital media processing, pattern recognition, and computer vision. The task of image/video segmentation emerges in many application areas, such as image interpretation, video analysis and understanding, video summarization and indexing, and digital entertainment. Over the last two decades, the problem of segmenting image/video data has become a fundamental one and had significant impact on both new pattern recognition algorithms and applications.This chapter has several objectives: (1) to survey the current status of research activities including graph-based, density estimator-based, and temporal-based segmentation algorithms. (2) To discuss recent developments while providing a comprehensive introduction to the fields of image/video segmentation. (3) To identify challenges ahead, and outline perspectives for the years to come.

[1]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Irena Koprinska,et al.  Temporal video segmentation: A survey , 2001, Signal Process. Image Commun..

[3]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[4]  Jitendra Malik,et al.  Recovering human body configurations: combining segmentation and recognition , 2004, CVPR 2004.

[5]  Fei-Fei Li,et al.  Spatially Coherent Latent Topic Model for Concurrent Segmentation and Classification of Objects and Scenes , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[6]  King Ngi Ngan,et al.  Face segmentation using skin-color map in videophone applications , 1999, IEEE Trans. Circuits Syst. Video Technol..

[7]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[8]  Michael G. Strintzis,et al.  Real-time compressed-domain spatiotemporal segmentation and ontologies for video indexing and retrieval , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  David A. Clausi,et al.  IRGS: Image Segmentation Using Edge Penalties and Region Growing , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Wen Gao,et al.  Measuring visual saliency by Site Entropy Rate , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  King Ngi Ngan,et al.  FaceSeg: Automatic Face Segmentation for Real-Time Video , 2009, IEEE Transactions on Multimedia.

[14]  Hyeran Byun,et al.  FRIP: a region-based image retrieval tool using automatic image segmentation and stepwise Boolean AND matching , 2005, IEEE Transactions on Multimedia.

[15]  Rachid Deriche,et al.  A Review of Statistical Approaches to Level Set Segmentation: Integrating Color, Texture, Motion and Shape , 2007, International Journal of Computer Vision.

[16]  John Wright,et al.  Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Carlo Tomasi,et al.  Mean shift is a bound optimization , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, SIGGRAPH 2005.

[20]  Marisa E. Campbell,et al.  SIGGRAPH 2004 , 2004, INTR.

[21]  Andrew Blake,et al.  Cosegmentation of Image Pairs by Histogram Matching - Incorporating a Global Constraint into MRFs , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[22]  Iasonas Kokkinos,et al.  Synergy between Object Recognition and Image Segmentation Using the Expectation-Maximization Algorithm , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Vikas Singh,et al.  An efficient algorithm for Co-segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[26]  Vikas Singh,et al.  Half-integrality based algorithms for cosegmentation of images , 2009, CVPR.

[27]  King Ngi Ngan,et al.  Unsupervised extraction of visual attention objects in color images , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Christof Bornhövd,et al.  A Prototype for Metadata-Based Integration of Internet Sources , 1999, CAiSE.

[29]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[30]  Gareth Funka-Lea,et al.  Multi-label Image Segmentation for Medical Applications Based on Graph-Theoretic Electrical Potentials , 2004, ECCV Workshops CVAMIA and MMBIA.

[31]  Alexei A. Efros,et al.  Discovering object categories in image collections , 2005 .

[32]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[33]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[34]  B. S. Manjunath,et al.  Color image segmentation , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[35]  King Ngi Ngan,et al.  Video segmentation for content-based coding , 1999, IEEE Trans. Circuits Syst. Video Technol..

[36]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[37]  Masatsugu Kidode,et al.  A Random Walk Procedure for Texture Discrimination , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Dorin Comaniciu,et al.  Robust analysis of feature spaces: color image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Camillo Gentile,et al.  Segmentation for robust tracking in the presence of severe occlusion , 2004, IEEE Trans. Image Process..

[40]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[41]  Martial Hebert,et al.  Measures of Similarity , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[42]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[43]  King Ngi Ngan,et al.  Automatic segmentation of moving objects for video object plane generation , 1998, IEEE Trans. Circuits Syst. Video Technol..

[44]  Jian Sun,et al.  Video object cut and paste , 2005, SIGGRAPH 2005.

[45]  Narendra Ahuja,et al.  Unsupervised Category Modeling, Recognition, and Segmentation in Images , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Jitendra Malik,et al.  Learning a classification model for segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[47]  Sankar K. Pal,et al.  A review on image segmentation techniques , 1993, Pattern Recognit..

[48]  J. Alison Noble,et al.  Ultrasound image segmentation: a survey , 2006, IEEE Transactions on Medical Imaging.

[49]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[50]  Vladimir Kolmogorov,et al.  Cosegmentation Revisited: Models and Optimization , 2010, ECCV.

[51]  Yong Jae Lee,et al.  Collect-cut: Segmentation with top-down cues discovered in multi-object images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[52]  Stephen Gould,et al.  Decomposing a scene into geometric and semantically consistent regions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[53]  Lihi Zelnik-Manor,et al.  Context-aware saliency detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[54]  King Ngi Ngan,et al.  Unsupervized Video Segmentation With Low Depth of Field , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[55]  Jean Ponce,et al.  Discriminative clustering for image co-segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[56]  Mark Q. Shaw,et al.  Automatic Image Segmentation by Dynamic Region Growth and Multiresolution Merging , 2009, IEEE Transactions on Image Processing.

[57]  King Ngi Ngan,et al.  Automatic video segmentation and tracking for content-based applications , 2007, IEEE Communications Magazine.

[58]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[59]  Sing Bing Kang,et al.  Stereo for Image-Based Rendering using Image Over-Segmentation , 2007, International Journal of Computer Vision.

[60]  Xiaogang Wang,et al.  Semantic Object Segmentation , 2011 .

[61]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[62]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[63]  Shimon Ullman,et al.  Class-Specific, Top-Down Segmentation , 2002, ECCV.

[64]  King Ngi Ngan,et al.  Saliency model-based face segmentation and tracking in head-and-shoulder video sequences , 2008, J. Vis. Commun. Image Represent..

[65]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[66]  W. Eric L. Grimson,et al.  Spatial Latent Dirichlet Allocation , 2007, NIPS.