A new approach to wire-frame tracking for semantic model-based moving image coding

Automatic wire-frame fitting and automatic wire-frame tracking are the two most important and most difficult issues associated with semantic-based moving image coding. A novel approach to high-speed tracking of important facial features is presented as a part of a complete fitting-tracking system we have developed. The method allows real-time processing of head-and-shoulders sequences using software tools only. The algorithm is based on eigenvalue decomposition of the sub-images extracted from subsequent frames of the video sequence. Since each facial feature (the left eye, the right eye, the nose and the lips) is tracked separately, the algorithm can be easily adapted for a parellel machine. The algorithm was tested on numerous widely used head-and-shoulders video sequences containing speaker's head pan, rotation and zoom with remarkably good results. The experiments we have carried out prove that it is possible to maintain tracking even when the facial features are partially occluded.

[1]  Bülent Sankur,et al.  Facial feature localization and adaptation of a generic face model for model-based coding , 1995, Signal Process. Image Commun..

[2]  Kiyoharu Aizawa,et al.  Model-based analysis synthesis image coding (MBASIC) system for a person's face , 1989, Signal Process. Image Commun..

[3]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[4]  Haibo Li,et al.  Image sequence coding at very low bit rates: a review , 1994, IEEE Trans. Image Process..

[5]  W. J. Welsh.  Model-based coding of videophone images , 1991 .

[6]  Jörn Ostermann,et al.  Object-oriented analysis-synthesis coding of moving images , 1989, Signal Process. Image Commun..

[7]  Haibo Li,et al.  Two-view facial movement estimation , 1994, IEEE Trans. Circuits Syst. Video Technol..

[8]  B. V. K. Vijaya Kumar,et al.  Efficient Calculation of Primary Images from a Set of Images , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Robert Forchheimer,et al.  Image coding-from waveforms in animation , 1989, IEEE Trans. Acoust. Speech Signal Process..

[10]  Hans Georg Musmann.  A layered coding system for very low bit rate video coding , 1995, Signal Process. Image Commun..

[11]  Peter Grant,et al.  Facial feature model fitting in semantic-based scene analysis , 1997 .

[12]  Peter M. Grant,et al.  Facial features motion analysis for wire-frame tracking in model-based moving image coding , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[13]  Arnaud E. Jacquin,et al.  Image coding based on a fractal theory of iterated contractive image transformations , 1992, IEEE Trans. Image Process..

[14]  Peter M. Grant,et al.  Automatic fitting and tracking of facial features in head-and-shoulders sequences , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[15]  Peter M. Grant,et al.  Automatic frame fitting for semantic-based moving image coding using a facial code-book , 1996, 1996 8th European Signal Processing Conference (EUSIPCO 1996).

[16]  Michel Barlaud,et al.  Image coding using wavelet transform , 1992, IEEE Trans. Image Process..

[17]  Adrian F. Clark,et al.  Feature and model tracking for model-based coding , 1992 .

[18]  Allen Gersho,et al.  On the structure of vector quantizers , 1982, IEEE Trans. Inf. Theory.

[19]  Kari Karhunen,et al.  Über lineare Methoden in der Wahrscheinlichkeitsrechnung , 1947 .

[20]  V. Seferidis.  Facial feature estimation for model-based coding , 1991 .

[21]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.