An automatic algorithm for semantic object generation and temporal tracking

Automatic semantic video object extraction is an important step for providing content-based video coding, indexing and retrieval. However, it is very difficult to design a generic semantic video object extraction technique, which can provide variant semantic video objects by using the same function. Since the presence and absence of persons in an image sequence provide important clues about video content, automatic face detection and human being generation are very attractive for content-based video database applications. For this reason, we propose a novel face detection and semantic human object generation algorithm. The homogeneous image regions with accurate boundaries are first obtained by integrating the results of color edge detection and region growing procedures. The human faces are detected from these homogeneous image regions by using skin color segmentation and facial filters. These detected faces are then used as object seed for semantic human object generation. The correspondences of the detected faces and semantic human objects along time axis are further exploited by a contour-based temporal tracking procedure.

[1]  Jenny Benois-Pineau,et al.  Joint contour-based and motion-based image sequence segmentation for TV image coding at very low bit rate , 1994, Other Conferences.

[2]  Levent Onural,et al.  Image sequence analysis for emerging interactive multimedia services-the European COST 211 framework , 1998, IEEE Trans. Circuits Syst. Video Technol..

[3]  Jake K. Aggarwal,et al.  The Integration of Image Segmentation Maps using Region and Edge Information , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Fuxi Gan,et al.  Motion estimation based on global and local uncompensability analysis , 1997, IEEE Trans. Image Process..

[5]  Jianping Fan,et al.  Spatiotemporal segmentation for compact video representation , 2001, Signal Process. Image Commun..

[6]  Fuxi Gan,et al.  Adaptive motion-compensated interpolation based on spatiotemporal segmentation , 1998, Signal Process. Image Commun..

[7]  Theodosios Pavlidis,et al.  Integrating Region Growing and Edge Detection , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Rong Wang,et al.  Image sequence segmentation based on 2D temporal entropic thresholding , 1996, Pattern Recognit. Lett..

[9]  Jonathan D. Courtney.  Automatic video indexing via object motion analysis , 1997, Pattern Recognit..

[10]  Ying Xu,et al.  2D image segmentation using minimum spanning trees , 1997, Image Vis. Comput..

[11]  Fernando Pereira.  MPEG-4: Why, what, how and when? , 2000, Signal Process. Image Commun..

[12]  Nariman Farvardin,et al.  A perceptually motivated three-component image model-Part I: description of the model , 1995, IEEE Trans. Image Process..

[13]  M. Hötter,et al.  Image segmentation based on object oriented mapping parameter estimation , 1988 .

[14]  JongWon Kim,et al.  SIVOG: smart interactive video object generation system , 1999, MULTIMEDIA '99.

[15]  John F. Haddon,et al.  Image Segmentation by Unifying Region and Boundary Information , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Thomas S. Huang,et al.  Human face detection in a complex background , 1994, Pattern Recognit..

[17]  Takeo Kanade,et al.  Name-It: Naming and Detecting Faces in Video by the Integration of Image and Natural Language Processing , 1997, IJCAI.

[18]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[19]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Alan L. Yuille,et al.  Feature extraction from faces using deformable templates , 2004, International Journal of Computer Vision.

[21]  Gang Wei,et al.  Face detection for image annotation , 1999, Pattern Recognition Letters.

[22]  Hidenori Itoh,et al.  Image Filtering, Edge Detection, and Edge Tracing Using Fuzzy Reasoning , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Fuxi Gan,et al.  Spatiotemporal segmentation based on two-dimensional spatiotemporal entropic thresholding , 1997 .

[24]  Qian Chen,et al.  Face Detection From Color Images Using a Fuzzy Pattern Matching Method , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Alessandro Neri,et al.  Automatic moving object and background separation , 1998, Signal Process..

[26]  Ming-Chieh Lee,et al.  Semiautomatic segmentation and tracking of semantic video objects , 1998, IEEE Trans. Circuits Syst. Video Technol..

[27]  Konstantinos N. Plataniotis,et al.  Automatic location and tracking of the facial region in color video sequences , 1999, Signal Process. Image Commun..

[28]  Clement T. Yu,et al.  Detecting human faces in color images , 1998, Proceedings International Workshop on Multi-Media Database Management Systems (Cat. No.98TB100249).

[29]  Haibo Li,et al.  Automatic extraction of human facial features , 1996, Signal Process. Image Commun..

[30]  King Ngi Ngan,et al.  Face segmentation using skin-color map in videophone applications , 1999, IEEE Trans. Circuits Syst. Video Technol..

[31]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Eli Saber,et al.  Frontal-view face detection and facial feature extraction using color, shape and symmetry based cost functions , 1998, Pattern Recognit. Lett..

[33]  A. Murat Tekalp,et al.  Temporal video segmentation using unsupervised clustering and semantic object tracking , 1998, J. Electronic Imaging.

[34]  Ioannis Pitas,et al.  A novel method for automatic face segmentation, facial feature extraction and tracking , 1998, Signal Process. Image Commun..

[35]  Norbert Diehl,et al.  Object-oriented motion estimation and segmentation in image sequences , 1991, Signal Process. Image Commun..