Depth-weighted group-wise principal component analysis for video foreground/background separation

We propose a depth-weighted group-wise PCA (DG-PCA) approach to separate moving foreground pixels from the background of a video acquired by a moving camera. Our approach utilizes a corresponding depth signal in addition to the video signal. The problem is formulated as a weighted l2,1-norm PCA problem with depth-based group sparsity being introduced. In particularly, dynamic groups are first generated solely based on depth, and then an iterative solution using depth to define the weights in l2,1-norm is developed. In addition, we propose a depth-enhanced homography model for global motion compensation before the DG-PCA method is executed. We demonstrate through experiments on an RGB-D dataset the superiority of the proposed DG-PCA approach over conventional robust PCA methods.

[1]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[2]  Luis Salgado,et al.  Background foreground segmentation with RGB-D Kinect data: An efficient combination of classifiers , 2014, J. Vis. Commun. Image Represent..

[3]  Gérard G. Medioni,et al.  Detection and tracking of moving objects from a moving platform in presence of strong parallax , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[4]  Eduardo Ros,et al.  Background Subtraction Based on Color and Depth Using Active Sensors , 2013, Sensors.

[5]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Soon Ki Jung,et al.  Depth extended online RPCA with spatiotemporal constraints for robust background subtraction , 2015, 2015 21st Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV).

[7]  Michael Harville,et al.  Foreground segmentation using adaptive mixture models in color and depth , 2001, Proceedings IEEE Workshop on Detection and Recognition of Events in Video.

[8]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Mathematical Programming Computation.

[9]  Trevor Darrell,et al.  Background estimation and removal based on range and color , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[10]  Junzhou Huang,et al.  Learning with dynamic group sparsity , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[11]  Weiqiang Wang,et al.  Foreground Detection Utilizing Structured Sparse Model via l1,2 Mixed Norms , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[12]  Nathan Srebro,et al.  Learning with matrix factorizations , 2004 .

[13]  Brendt Wohlberg,et al.  Incremental Principal Component Pursuit for Video Background Modeling , 2015, Journal of Mathematical Imaging and Vision.

[14]  Ming-Ting Sun,et al.  Global motion estimation from coarsely sampled motion vector field and the applications , 2003, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Ming-Ting Sun,et al.  Global motion estimation from coarsely sampled motion vector field and the applications , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[16]  Hassan Mansour,et al.  Video background subtraction using semi-supervised robust matrix completion , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Y. Zhang,et al.  Augmented Lagrangian alternating direction method for matrix separation based on low-rank factorization , 2014, Optim. Methods Softw..

[18]  Wotao Yin,et al.  Group sparse optimization by alternating direction method , 2013, Optics & Photonics - Optical Engineering + Applications.

[19]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[20]  Ivan V. Bajic,et al.  A Joint Approach to Global Motion Estimation and Motion Segmentation From a Coarsely Sampled Motion Vector Field , 2011, IEEE Transactions on Circuits and Systems for Video Technology.