A Framework for High-Level Feedback to Adaptive, Per-Pixel, Mixture-of-Gaussian Background Models

Time-Adaptive, Per-Pixel Mixtures Of Gaussians (TAPPMOGs) have recently become a popular choice for robust modeling and removal of complex and changing backgrounds at the pixel level. However, TAPPMOG-based methods cannot easily be made to model dynamic backgrounds with highly complex appearance, or to adapt promptly to sudden "uninteresting" scene changes such as the repositioning of a static object or the turning on of a light, without further undermining their ability to segment foreground objects, such as people, where they occlude the background for too long. To alleviate tradeoffs such as these, and, more broadly, to allow TAPPMOG segmentation results to be tailored to the specific needs of an application, we introduce a general framework for guiding pixel-level TAPPMOG evolution with feedback from "high-level" modules. Each such module can use pixel-wise maps of positive and negative feedback to attempt to impress upon the TAPPMOG some definition of foreground that is best expressed through "higher-level" primitives such as image region properties or semantics of objects and events. By pooling the foreground error corrections of many high-level modules into a shared, pixel-level TAPPMOG model in this way, we improve the quality of the foreground segmentation and the performance of all modules that make use of it. We show an example of using this framework with a TAPPMOG method and high-level modules that all rely on dense depth data from a stereo camera.

[1]  John Woodfill,et al.  Real-time stereo vision on the PARTS reconfigurable computer , 1997, Proceedings. The 5th Annual IEEE Symposium on Field-Programmable Custom Computing Machines Cat. No.97TB100186).

[2]  Daniel P. Huttenlocher,et al.  Scene modeling for wide area surveillance and image synthesis , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[4]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[5]  David Beymer,et al.  Person counting using stereo , 2000, Proceedings Workshop on Human Motion.

[6]  Tim J. Ellis,et al.  Illumination-Invariant Motion Detection Using Colour Mixture Models , 2001, BMVC.

[7]  Visvanathan Ramesh,et al.  Error analysis of background subtraction , 2000, CVPR 2000.

[8]  Paolo Remagnino,et al.  From connected components to object sequences , 2000 .

[9]  Michael Harville,et al.  Foreground segmentation using adaptive mixture models in color and depth , 2001, Proceedings IEEE Workshop on Detection and Recognition of Events in Video.

[10]  Pradeep K. Khosla,et al.  Motion detection and segmentation using image mosaics , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[11]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Andrew Blake,et al.  Statistical Background Modelling for Tracking with a Virtual Camera , 1995, BMVC.

[13]  Shaogang Gong,et al.  Learning pixel-wise signal energy for understanding semantics , 2003, Image Vis. Comput..

[14]  Xiang Gao,et al.  Error analysis of background adaption , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[16]  Michael Harville,et al.  Stereo Person Tracking with Adaptive Plan-View Statistical Templates , 2002 .

[17]  Stuart J. Russell,et al.  Image Segmentation in Video Sequences: A Probabilistic Approach , 1997, UAI.

[18]  Trevor Darrell,et al.  Plan-view trajectory estimation with dense stereo background models , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[19]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .