EXTRACTION OF SEMANTIC 3D MODELS OF HUMAN FACES FROM STEREOSCOPIC IMAGE SEQUENCES

In this paper a system is described for the automatic generation of a specific semantic 3D model of a human face from a recorded stereoscopic image sequence. In the acquisition stage a calibrated stereoscopic camera records the specific face. After rectification of the image sequences a dense disparity field is estimated using a Markov Random Field approach. Next a triangle based 3D-wire frame model of the specific face is reconstructed by triangulation of corresponding left and right pixels. After compensation for rotation and translation, a triangle based 3D-wire frame model of a generic face is elastically matched to the specific face. As a consequence of this approach the resulting 3D model accurately represents the geometry of the subject with the availability of the face semantic information which is required for realistic representation and animation of human faces as well as for accurate facial expressions estimation.

[1]  Emile A. Hendriks,et al.  Synthesis of multi viewpoint images at non-intermediate positions , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[2]  J. Cassell,et al.  Communicative humanoids: a computational model of psychosocial dialogue skills , 1996 .

[3]  Klaus Hopf,et al.  Advanced Videocommunications with Stereoscopy and Individual Perspectives , 1994, IS&N.

[4]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[5]  Keith Waters,et al.  Computer facial animation , 1996 .

[6]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[7]  Demetri Terzopoulos,et al.  Modelling and animating faces using scanned data , 1991, Comput. Animat. Virtual Worlds.

[8]  Larry S. Davis,et al.  Recognizing Human Facial Expressions From Long Image Sequences Using Optical Flow , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Paolo Cignoni,et al.  Multiresolution decimation based on global error , 1996, The Visual Computer.

[10]  H. J. Woltring.  Simultaneous Multiframe Analytical Calibrations (S.M.A.C.) By Recourse To Oblique Observations Of Planar Control Distributions , 1980, Other Conferences.

[11]  Aggelos K. Katsaggelos,et al.  Disparity estimation with modeling of occlusion and object orientation , 1998, Electronic Imaging.

[12]  C. S. Choi,et al.  Human Facial Motion Analysis and Synthesis with Application to Model-Based Coding , 1993 .

[13]  Yoshio Nagashima,et al.  Three-dimensional face model reproduction method using multiview images , 1991, Other Conferences.

[14]  Christoph Stiller,et al.  Object-based estimation of dense motion fields , 1997, IEEE Trans. Image Process..

[15]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Bidyut Baran Chaudhuri,et al.  Elliptic fit of objects in two and three dimensions by moment of inertia optimization , 1991, Pattern Recognit. Lett..

[17]  David J. Burr,et al.  Elastic Matching of Line Drawings , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..