Semiautomatic segmentation and tracking of semantic video objects

This paper introduces a novel semantic video object extraction system using mathematical morphology and a perspective motion model. Inspired by the results from the study of the human visual system, we intend to solve the semantic video object extraction problem in two separate steps: supervised I-frame segmentation, and unsupervised P-frame tracking. First, the precise semantic video object boundary can be found using a combination of human assistance and a morphological segmentation tool. Second, the semantic video objects in the remaining frames are obtained using global perspective motion estimation and compensation of the previous semantic video object plus boundary refinement as used for I frames.

[1]  M. Kunt,et al.  Second-generation image-coding techniques , 1985, Proceedings of the IEEE.

[2]  Josef Bigün,et al.  Segmentation of moving objects by robust motion parameter estimation over multiple frames , 1994, ECCV.

[3]  J.K. Aggarwal,et al.  Correspondence processes in dynamic scene analysis , 1981, Proceedings of the IEEE.

[4]  F. Meyer,et al.  Color image segmentation , 1992 .

[5]  S.-M. Kruse,et al.  Scene segmentation from dense displacement vector fields using randomized Hough transform , 1996, Signal Process. Image Commun..

[6]  Bruce A. Thomas,et al.  Image Models for 2-D Flow Visualization and Compression , 1994, CVGIP Graph. Model. Image Process..

[7]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[8]  C. Gu.  Multivalued morphology and segmentation-based coding , 1996 .

[9]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  P. Bouthemy,et al.  Recovery of moving object masks in an image sequence using local spatiotemporal contextual information , 1993 .

[11]  Michael J. Black.  Combining Intensity and Motion for Incremental Segmentation and Tracking Over Long Image Sequences , 1992, ECCV.

[12]  Gang Xu,et al.  Tracking Human Body Motion Based on a Stick Figure Model , 1994, J. Vis. Commun. Image Represent..

[13]  Shih-Fu Chang,et al.  Digital image/video library and MPEG-7: standardization and research issues , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[14]  Richard Szeliski,et al.  A layered video object coding system using sprite and affine motion model , 1997, IEEE Trans. Circuits Syst. Video Technol..

[15]  Naonori Ueda,et al.  Tracking Moving Contours Using Energy-Minimizing Elastic Contour Models , 1992, ECCV.

[16]  Hans-Hellmut Nagel,et al.  Motion Boundary Detection in Image Sequences by Local Stochastic Tests , 1994, ECCV.

[17]  J. Serra,et al.  An overview of morphological filtering , 1992 .

[18]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Tzay Y. Young,et al.  A Mathematical Model for Computer Image Tracking , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Rachid Deriche,et al.  Tracking line segments , 1990, Image Vis. Comput..

[21]  Michal Irani,et al.  Detecting and Tracking Multiple Moving Objects Using Temporal Integration , 1992, ECCV.

[22]  Azriel Rosenfeld,et al.  Segmentation and Estimation of Image Region Properties through Cooperative Hierarchial Computation , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[23]  Alessandro Verri,et al.  Identifying multiple motions from optical flow , 1992, ECCV.

[24]  Graham R. Martin,et al.  Model-based multiresolution motion estimation in noisy images , 1994 .

[25]  G. Gordon,et al.  On the tracking of featureless objects with occlusion , 1989, [1989] Proceedings. Workshop on Visual Motion.

[26]  Philip H. S. Torr,et al.  Statistical detection of independent movement from a moving camera , 1993, Image Vis. Comput..

[27]  Jörn Ostermann,et al.  Object-oriented analysis-synthesis coding of moving images , 1989, Signal Process. Image Commun..

[28]  Patrick C. Chen,et al.  Image segmentation as an estimation problem , 1979, 1979 18th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[29]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[30]  Jean-Marc Odobez,et al.  Robust Multiresolution Estimation of Parametric Motion Models , 1995, J. Vis. Commun. Image Represent..

[31]  Theodosios Pavlidis,et al.  Picture Segmentation by a Tree Traversal Algorithm , 1976, JACM.

[32]  Rama Chellappa,et al.  Tracking a dynamic set of feature points , 1994, IEEE Trans. Image Process..

[33]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[34]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Josef Kittler,et al.  A Gradient-Based Method for General Motion Estimation and Segmentation , 1993, J. Vis. Commun. Image Represent..

[36]  A. Pentland,et al.  Robust estimation of a multi-layered motion representation , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[37]  Patrick Bouthemy,et al.  Region-Based Tracking Using Affine Motion Models in Long Image Sequences , 1994 .

[38]  Henri Nicolas,et al.  Global motion identification for image sequence analysis and coding , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[39]  Richard Szeliski,et al.  Video mosaics for virtual environments , 1996, IEEE Computer Graphics and Applications.

[40]  Luis Torres,et al.  Region-based video coding using mathematical morphology , 1995 .

[41]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[42]  Steven K. Feiner,et al.  Computer graphics: principles and practice (2nd ed.) , 1990 .

[43]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[44]  Alan L. Yuille,et al.  Deformable templates , 1993 .

[45]  A. Murat Tekalp,et al.  Simultaneous alpha map generation and 2-D mesh tracking for multimedia applications , 1997, Proceedings of International Conference on Image Processing.

[46]  Ishwar K. Sethi,et al.  Finding Trajectories of Feature Points in a Monocular Image Sequence , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[48]  Touradj Ebrahimi,et al.  Morphological moving object segmentation and tracking for content-based video coding , 1995 .

[49]  Robert J. Schalkoff,et al.  A Model and Tracking Algorithm for a Class of Video Targets , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.