Background estimation as a labeling problem

We present a new background estimation algorithm that constructs the background of an image sequence with moving objects by copying areas from input frames. The background estimation problem is formulated as an optimal labeling problem in which the label at an output pixel is the frame number from which to copy the background color. The costs of assigning labels encourage seamless copying from regions that are stationary over a period of time in such a way that implied motion boundaries occur at intensity edges. This is accomplished without explicitly tracking the moving objects or computing optical flow. Experiments demonstrate that our algorithm is effective in difficult areas where the background is visible for only a small fraction of time, and on inputs with both moving objects that are not always in motion and moving objects with textureless areas

[1]  Hamid Aghajan,et al.  Video-based freeway-monitoring system using recursive vehicle tracking , 1995, Electronic Imaging.

[2]  James Davis,et al.  Mosaics of scenes with moving objects , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[3]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[5]  Trevor Darrell,et al.  Background estimation and removal based on range and color , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[6]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Gérard G. Medioni,et al.  A graph-based global registration for 2D mosaics , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[8]  Sergio A. Velastin,et al.  Automatic congestion detection system for underground platforms , 2001, Proceedings of 2001 International Symposium on Intelligent Multimedia, Video and Speech Processing. ISIMP 2001 (IEEE Cat. No.01EX489).

[9]  Anil K. Jain,et al.  A background model initialization algorithm for video surveillance , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[10]  Michael Harville,et al.  Adaptive video background modeling using color and depth , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[11]  Wolfgang Effelsberg,et al.  Robust background estimation for complex video sequences , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[12]  Rita Cucchiara,et al.  Detecting Moving Objects, Ghosts, and Shadows in Video Streams , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Daniel P. Huttenlocher,et al.  Efficient Belief Propagation for Early Vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..