Illumination Robust Video Foreground Prediction Based on Color Recovering

Video foreground prediction is a technique to estimate the probability of each pixel being foreground in current frame based on a foreground segmentation result of its previous frame. Existing foreground prediction algorithms usually assume that the illumination conditions are constant for consecutive frames. Therefore, they cannot predict foreground accurately when the illumination condition changes sharply between video frames. In this paper, a new robust video foreground prediction algorithm is proposed based on color recovering, which is derived based on an observation that the illumination changes are locally smooth. By integrating color recovering with an optical flow estimation algorithm and an opacity propagation algorithm, the negative impact of the illumination changes could be removed. Experimental results show that the proposed algorithm can get more accurate results for videos with illumination changes compared with the existing foreground prediction algorithms.

[1]  Ying Wu,et al.  Closed-Loop Adaptation for Robust Tracking , 2010, ECCV.

[2]  Li Zhang,et al.  MRF-based adaptive approach for foreground segmentation under sudden illumination change , 2007, 2007 6th International Conference on Information, Communications & Signal Processing.

[3]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Chang-Su Kim,et al.  Real-time segmentation of objects from video sequences with non-stationary backgrounds using spatio-temporal coherence , 2008, 2008 15th IEEE International Conference on Image Processing.

[5]  David Corrigan,et al.  Video Matting Using Motion Extended GrabCut , 2008 .

[6]  Takeshi Naemura,et al.  Real-Time Video Matting Based on Bilayer Segmentation , 2009, ACCV.

[7]  Sang Uk Lee,et al.  Robust Stereo Matching Using Adaptive Normalized Cross-Correlation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Guillermo Sapiro,et al.  Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting , 2009, International Journal of Computer Vision.

[9]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Irfan A. Essa,et al.  Tree-based Classifiers for Bilayer Video Segmentation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Eitan Grinspun,et al.  Sparse matrix solvers on the GPU: conjugate gradients and multigrid , 2003, ACM Trans. Graph..

[12]  J. Li,et al.  Foreground segmentation for dynamic scenes with sudden illumination changes , 2012 .

[13]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[14]  Scott Cohen,et al.  LIVEcut: Learning-based interactive video segmentation by evaluation of multiple propagated cues , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Franklin C. Crow,et al.  Summed-area tables for texture mapping , 1984, SIGGRAPH.

[16]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, ACM Trans. Graph..

[18]  Carsten Rother,et al.  Fast cost-volume filtering for visual correspondence and beyond , 2011, CVPR 2011.

[19]  H. H. Chen,et al.  Video Object Extraction via MRF-Based Contour Tracking , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Jong-Chul Yoon,et al.  Temporally coherent video matting , 2010, SIGGRAPH '10.

[21]  Zhenjiang Miao,et al.  Foreground prediction for bilayer segmentation of videos , 2011, Pattern Recognit. Lett..

[22]  Harry Shum,et al.  An Object-Based Approach to Image/Video-Based Synthesis and Processing for 3-D and Multiview Televisions , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Andrew W. Fitzgibbon,et al.  Bayesian video matting using learnt image priors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[24]  Michael F. Cohen,et al.  Optimized Color Sampling for Robust Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Guillermo Sapiro,et al.  A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[26]  Jin-Jang Leou,et al.  Background initialization and foreground segmentation for bootstrapping video sequences , 2013, EURASIP J. Image Video Process..

[27]  Guillermo Sapiro,et al.  Dynamic Color Flow: A Motion-Adaptive Color Model for Object Segmentation in Video , 2010, ECCV.

[28]  Pieter Peers,et al.  SubEdit: a representation for editing measured heterogeneous subsurface scattering , 2009, SIGGRAPH 2009.

[29]  Qunsheng Peng,et al.  Transductive segmentation of live video with non-stationary background , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Harry Shum,et al.  Video object cut and paste , 2005, ACM Trans. Graph..

[31]  Caifeng Shan,et al.  Background Subtraction under Sudden Illumination Changes , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[32]  Abhinav Gupta,et al.  A novel approach to video matting using automated scribbling by motion analysis , 2008, 2008 IEEE Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems.

[33]  A. Criminisi,et al.  Bilayer Segmentation of Live Video , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[34]  Zhenjiang Miao,et al.  Temporally consistent video matting based on bilayer segmentation , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[35]  Edward J. Delp,et al.  Foreground segmentation with sudden illumination changes using a shading model and a Gaussianity test , 2011, 2011 7th International Symposium on Image and Signal Processing and Analysis (ISPA).

[36]  Guillermo Sapiro,et al.  Video SnapCut: robust video object cutout using localized classifiers , 2009, SIGGRAPH 2009.