Accurate motion layer segmentation and matting

Given a video sequence, obtaining accurate layer segmentation and alpha matting is very important for various applications. However, when a non-textured or smooth area is present in the scene, the segmentation based on only single motion cue usually cannot provide satisfactory results. Conversely, the most matting approaches require a smooth assumption on foreground and background to obtain a good result. In this paper, we combine the merits of motion segmentation and alpha matting technique together to simultaneously achieve high-quality layer segmentation and alpha mattes. First, we explore a general occlusion constraint and design a novel graph cuts framework to solve the layer-based motion segmentation problem for the textured regions using multiple frames. Then, an alpha matting technique is further used to refine the segmentation and resolve the non-textured ambiguities by determining proper alpha values for the foreground and background respectively.

[1]  Mihalis Yannakakis,et al.  The complexity of multiway cuts (extended abstract) , 1992, STOC '92.

[2]  Carlo Tomasi,et al.  Multiway cut for stereo and motion with slanted surfaces , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3]  Mubarak Shah,et al.  Motion layer extraction in the presence of occlusion using graph cuts , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[5]  Qionghai Dai,et al.  An accurate scene-based traffic model for MPEG video stream , 2003, 10th IEEE International Conference on Electronics, Circuits and Systems, 2003. ICECS 2003. Proceedings of the 2003.

[6]  Andrew W. Fitzgibbon,et al.  Bayesian video matting using learnt image priors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[7]  Carlo Tomasi,et al.  Alpha estimation in natural images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[8]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Richard Szeliski,et al.  Boundary Matting for View Synthesis , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[10]  Mubarak Shah,et al.  Two-frame wide baseline matching , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[11]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[12]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[13]  Takeo Kanade,et al.  Robust subspace clustering by combined use of kNND metric and SVD algorithm , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[14]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[15]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[16]  Serge J. Belongie,et al.  What went where , 2003, CVPR 2003.

[17]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.