Layered Representation of a Video Shot with Mosaicing

Abstract: This paper presents a motion segmentation method useful for representing efficiently a video shot as a static mosaic of the background plus sequences of the objects moving in the foreground. This generates an MPEG-4 compliant, layered representation useful for video coding, editing and indexing. First, a mosaic of the static background is computed by estimating the dominant motion of the scene. This is achieved by tracking features over the video sequence and using a robust technique that discards features attached to the moving objects. The moving objects get removed in the final mosaic by computing the median of the grey levels. Then, segmentation is obtained by taking the pixelwise difference between each frame of the original sequence and the mosaic of the background. To discriminate between the moving object and noise, temporal coherence is exploited by tracking the object in the binarised difference image sequence. The automatic computation of the mosaic and the segmentation procedure are illustrated with real sequences experiments. Examples of coding and content-based manipulation are also shown.

[1]  Graeme A. Jones,et al.  Segmentation of Global Motion using Temporal Probabilistic Classification , 1998, BMVC.

[2]  Emanuele Trucco,et al.  Introductory techniques for 3-D computer vision , 1998 .

[3]  Emanuele Trucco,et al.  Feature Tracking in Video and Sonar Subsea Sequences with Applications , 2000, Comput. Vis. Image Underst..

[4]  Andrew Zisserman,et al.  Metric rectification for perspective images of planes , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[5]  Jean Serra,et al.  Image Analysis and Mathematical Morphology , 1983 .

[6]  Fernando Pereira,et al.  MPEG-4: Context and objectives , 1997, Signal Process. Image Commun..

[7]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[8]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[9]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[10]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[11]  Emanuele Trucco,et al.  Improving Feature Tracking with Robust Statistics , 1999, Pattern Analysis & Applications.

[12]  P. Anandan,et al.  Mosaic based representations of video sequences and their applications , 1995, Proceedings of IEEE International Conference on Computer Vision.

[13]  Shih-Fu Chang,et al.  VideoQ: an automated content based video search system using visual cues , 1997, MULTIMEDIA '97.

[14]  Andrew Zisserman,et al.  Automated mosaicing with super-resolution zoom , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[15]  Harpreet S. Sawhney,et al.  Compact Representations of Videos Through Dominant and Multiple Motion Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  R. Chellappa,et al.  Fast 3D stabilization and mosaic construction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Azriel Rosenfeld,et al.  Robust regression methods for computer vision: A review , 1991, International Journal of Computer Vision.

[18]  Gérard G. Medioni,et al.  Detecting and tracking moving objects for video surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[19]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[20]  Nassir Navab,et al.  Relative Affine Structure: Canonical Model for 3D From 2D Geometry and Applications , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Rachid Deriche,et al.  Using geometric corners to build a 2D mosaic from a set of images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Rachid Deriche,et al.  Recovering and characterizing image features using an efficient model based approach , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Richard Szeliski,et al.  Video mosaics for virtual environments , 1996, IEEE Computer Graphics and Applications.

[24]  Richard I. Hartley,et al.  In defence of the 8-point algorithm , 1995, Proceedings of IEEE International Conference on Computer Vision.

[25]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[26]  Hans-Hellmut Nagel,et al.  New likelihood test methods for change detection in image sequences , 1984, Comput. Vis. Graph. Image Process..

[27]  Howard Wainer,et al.  Robust Regression & Outlier Detection , 1988 .

[28]  Subhasis Chaudhuri,et al.  Automated assembling of images: image montage preparation , 1995, Pattern Recognit..

[29]  R. Brunelli,et al.  A Survey on the Automatic Indexing of Video Data, , 1999, J. Vis. Commun. Image Represent..

[30]  Thomas S. Huang,et al.  Image sequence analysis , 1981 .

[31]  Shmuel Peleg,et al.  Universal mosaicing using pipe projection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[32]  Kristin J. Dana,et al.  Real-time scene stabilization and mosaic construction , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[33]  P. Anandan,et al.  Efficient representations of video sequences and their applications , 1996, Signal Process. Image Commun..