Region based foreground segmentation combining color and depth sensors via logarithmic opinion pool decision

HighlightsForeground segmentation system combining color camera sensor with depth camera sensor in a Logarithmic Opinion Pool Framework.Combine pixel-wise color and depth models with Region Based Foreground Models.Compute the Hellinger distances in order to detect the degree of similarity among foreground and background classes.Use of each sensor according to the reliability that they present.Enhancement of the foreground segmentation using a trimap borders enhancement. In this paper we present a novel foreground segmentation system that combines color and depth sensors information to perform a more complete Bayesian segmentation between foreground and background classes. The system shows a combination of spatial-color and spatial-depth region-based models for the foreground as well as color and depth pixel-wise models for the background in a Logarithmic Opinion Pool decision framework used to correctly combine the likelihoods of each model. A posterior enhancement step based on a trimap analysis is also proposed in order to correct the precision errors that the depth sensor introduces. The results presented in this paper show that our system is robust in front of color and depth camouflage problems between the foreground object and the background, and also improves the segmentation in the area of the objects' contours by reducing the false positive detections that appear due to the lack of precision of the depth sensors.

[1]  Michael F. Cohen,et al.  Monocular Video Foreground/Background Segmentation by Tracking Spatial-Color Gaussian Mixture Models , 2007, 2007 IEEE Workshop on Motion and Video Computing (WMVC'07).

[2]  R. Beran.  Minimum Hellinger distance estimates for parametric models , 1977 .

[3]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  S Kullback,et al.  LETTER TO THE EDITOR: THE KULLBACK-LEIBLER DISTANCE , 1987 .

[5]  Reinhard Koch,et al.  Time-Consistent Foreground Segmentation of Dynamic Content from Color and Depth Video , 2011, DAGM-Symposium.

[6]  Ruigang Yang,et al.  TofCut: Towards Robust Real-time Foreground Extraction using Time-of-flight Camera , 2016 .

[7]  Reinhard Koch,et al.  Time-of-Flight Sensors in Computer Graphics , 2009, Eurographics.

[8]  P. Lima,et al.  Bayesian Sensor Fusion for Cooperative Object Localization and World Modeling , 2003 .

[9]  Ryan Crabb,et al.  Real-time foreground segmentation via range and color imaging , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[10]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[11]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[12]  J. Aggarwal,et al.  Statistical Decision Integration Using Fisher Criterion , 1999 .

[13]  Michael Werman,et al.  Fusing Time-of-Flight Depth and Color for Real-Time Segmentation and Tracking , 2009, Dyn3D.

[14]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[15]  Rasmus Larsen,et al.  TOF imaging in Smart room environments towards improved people tracking , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[16]  Montse Pardàs,et al.  Bayesian foreground segmentation and tracking using pixel-wise background model and region based foreground model , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[17]  Reinhard Koch,et al.  Improved Video Segmentation by Adaptive Combination of Depth Keying and Mixture-of-Gaussians , 2011, SCIA.

[18]  Marjorie Skubic,et al.  Evaluation of an inexpensive depth camera for passive in-home fall risk assessment , 2011, 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[19]  Yaser Sheikh,et al.  Bayesian modeling of dynamic scenes for object detection , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Jake K. Aggarwal,et al.  Human detection using depth information by Kinect , 2011, CVPR 2011 WORKSHOPS.

[21]  Qiong Wu,et al.  Robust Real-Time Bi-Layer Video Segmentation Using Infrared Video , 2008, 2008 Canadian Conference on Computer and Robot Vision.

[22]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[23]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .