Image-based rendering and tracking of faces

In this paper, we present an image-based method for the tracking and rendering of faces. We use the algorithm in an immersive video conferencing system where multiple participants are placed at a virtual table. This requires viewpoint modification of dynamic objects. Since hair and uncovered areas are difficult to model by pure 3-D geometry-based warping, we add image-based rendering techniques to the system. By interpolating novel views from a 3-D image volume, natural looking results can be achieved. The image-based component is embedded into a geometry-based approach that models temporally changing facial features. Both geometry and image cube information are jointly exploited in facial expression analysis and synthesis.

[1]  Peter Eisert,et al.  Analyzing Facial Expressions for Virtual Conferencing , 1998, IEEE Computer Graphics and Applications.

[2]  Kiyoharu Aizawa,et al.  Model-based image coding , 1994, Other Conferences.

[3]  W. J. Welsh.  Model-based coding of videophone images , 1991 .

[4]  Peter Eisert,et al.  MPEG‐4 facial animation in video analysis and synthesis , 2003, Int. J. Imaging Syst. Technol..

[5]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[6]  Geovanni Martinez,et al.  AUTOMATIC ADAPTATION OF A HUMAN FACE MODEL FOR MODEL-BASED CODING , 2004 .

[7]  Dimitris N. Metaxas,et al.  Deformable model-based shape and motion analysis from images using motion residual error , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[8]  D. E. Pearson,et al.  Developments in model-based video coding , 1995, Proc. IEEE.

[9]  Harry Shum,et al.  Review of image-based rendering techniques , 2000, Visual Communications and Image Processing.

[10]  Jörn Ostermann,et al.  Automatic adaptation of a face model in a layered coder with an object-based analysis-synthesis layer and a knowledge-based layer , 1997, Signal Process. Image Commun..

[11]  Peter Eisert.  Model-Based Camera Calibration Using Analysis by Synthesis Techniques , 2002, VMV.

[12]  J. Ahlberg.  Extraction and Coding of Face Model Parameters , 1999 .

[13]  Ralph Gross,et al.  Constructing and Fitting Active Appearance Models With Occlusion , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.