Temporal stabilization of video object segmentation for 3D-TV applications

We present a method for improving the temporal stability of video object segmentation algorithms for 3D-TV applications. First, two quantitative measures to evaluate temporal stability without ground-truth are presented. Then, a pseudo-3D curve evolution method, which spatio-temporally stabilizes the estimated object segments is introduced. Temporal stability is achieved by re-distributing existing object segmentation errors such that they are less disturbing when the scene is rendered and viewed in 3D. Our starting point is the hypothesis that if making segmentation errors are inevitable, they should be made in a temporally consistent way for 3D TV applications. This hypothesis is supported by the experiments, which show that there is significant improvement in segmentation quality both in terms of the objective quantitative measures and in terms of the viewing comfort in subjective perceptual tests. This shows that it is possible to increase the object segmentation quality without increasing the actual segmentation accuracy.

[1]  Paul Smith,et al.  Edge Tracking for Motion Segmentation and Depth Ordering , 1999, BMVC.

[2]  Guojun Lu,et al.  Segmentation of moving objects in image sequence: A review , 2001 .

[3]  A. Murat Tekalp,et al.  Video object tracking with feedback of performance measures , 2003, IEEE Trans. Circuits Syst. Video Technol..

[4]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[5]  André Redert,et al.  Three dimensional video for the home , 2001 .

[6]  Anthony J. Yezzi,et al.  A Fully Global Approach to Image Segmentation via Coupled Curve Evolution Equations , 2002, J. Vis. Commun. Image Represent..

[7]  Peter G. Engeldrum,et al.  Psychometric Scaling: A Toolkit for Imaging Systems Development , 2000 .

[8]  Esther M. Arkin,et al.  An efficiently computable metric for comparing polygonal shapes , 1991, SODA '90.

[9]  Brian Scassellati,et al.  Retrieving images by 2D shape: a comparison of computation methods with human perceptual judgments , 1994, Electronic Imaging.

[10]  Amar Mitiche,et al.  PDE-based region tracking without motion computation by joint space-time segmentation , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[11]  Olivier D. Faugeras,et al.  Image Segmentation Using Active Contours: Calculus of Variations or Shape Gradients? , 2003, SIAM J. Appl. Math..

[12]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[13]  Cornelius W. A. M. van Overveld,et al.  Dense Structure-from-Motion: An Approach Based on Segment Matching , 2002, ECCV.

[14]  A. Murat Tekalp,et al.  Performance measures for video object segmentation and tracking , 2003, IEEE Transactions on Image Processing.

[15]  Amar Mitiche,et al.  Tracking with simultaneous camera motion subtraction by level set spatio-temporal surface evolution , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[16]  Michael J. Black,et al.  Cardboard people: a parameterized model of articulated image motion , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[17]  Marcus A. Magnor,et al.  Joint 3D-reconstruction and background separation in multiple views using graph cuts , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[18]  James M. Rehg,et al.  A multiple hypothesis approach to figure tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[19]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[21]  Hans-Hellmut Nagel,et al.  Tracking Persons in Monocular Image Sequences , 1999, Comput. Vis. Image Underst..

[22]  Fernando Pereira,et al.  Objective evaluation of video segmentation quality , 2003, IEEE Trans. Image Process..

[23]  Anthony J. Yezzi,et al.  A vertex-based representation of objects in an image , 2002, Proceedings. International Conference on Image Processing.

[24]  Adam Baumberg,et al.  Learning deformable models for tracking human motion , 1996 .

[25]  A. Murat Tekalp,et al.  Metrics for performance evaluation of video object segmentation and tracking without ground-truth , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[26]  Michel Barlaud,et al.  DREAM2S: Deformable Regions Driven by an Eulerian Accurate Minimization Method for Image and Video Segmentation , 2002, International Journal of Computer Vision.