Animated Heads from Ordinary Images: A Least-Squares Approach

We show that we can effectively fit arbitrarily complex animation models to noisy data extracted from ordinary face images. Our approach is based on least-squares adjustment, using of a set of progressively finer control triangulations and takes advantage of three complementary sources of information: stereo data, silhouette edges, and 2D feature points. In this way, complete head models?including ears and hair?can be acquired with a cheap and entirely passive sensor, such as an ordinary video camera. They can then be fed to existing animation software to produce synthetic sequences.

[1]  Pascal Fua,et al.  From synthesis to analysis: fitting human animation models to image data , 1999, 1999 Proceedings Computer Graphics International.

[2]  P. Fua.  From Multiple Stereo Views to Multiple 3-D Surfaces , 2022 .

[3]  Pascal Fua,et al.  Using model-driven bundle-adjustment to model heads from raw video sequences , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Michael Isard,et al.  3D position, attitude and shape input using video tracking of hands and lips , 1994, SIGGRAPH.

[5]  Daniel Thalmann,et al.  Simulation of Facial Muscle Actions Based on Rational Free Form Deformations , 1992, Comput. Graph. Forum.

[6]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[7]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[8]  William A. Barrett,et al.  Intelligent scissors for image composition , 1995, SIGGRAPH.

[9]  Luc Van Gool,et al.  Active acquisition of 3D shape for moving objects , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[10]  Olivier D. Faugeras,et al.  Computing differential properties of 3-D shapes from stereoscopic images without 3-D models , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Aaron F. Bobick,et al.  The direct computation of height from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Demetri Terzopoulos,et al.  Realistic modeling for facial animation , 1995, SIGGRAPH.

[13]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Dimitris N. Metaxas,et al.  Deformable model-based shape and motion analysis from images using motion residual error , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[15]  J. E. Glynn,et al.  Numerical Recipes: The Art of Scientific Computing , 1989 .

[16]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[17]  Thomas S. Huang,et al.  Analysis-based facial expression synthesis , 1994, Proceedings of 1st International Conference on Image Processing.

[18]  Richard Szeliski,et al.  Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares , 1994, J. Vis. Commun. Image Represent..

[19]  Pascal Fua,et al.  Face Models from Uncalibrated Video Sequences , 1998, CAPTECH.

[20]  WON-SOOK LEE,et al.  From Real Faces To Virtual Faces : Problems and Solutions , 1999 .

[21]  Jean-Philippe Tarel,et al.  Camcal v1.0 Manual A Complete Software Solution for Camera Calibration , 1996 .

[22]  Nadia Magnenat-Thalmann,et al.  Modelling and Motion Capture Techniques for Virtual Environments , 1998, Lecture Notes in Computer Science.

[23]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[24]  Pascal Fua,et al.  Imposing Hard Constraints on Deformable Models through Optimization in Orthogonal Subspaces , 1997, Comput. Vis. Image Underst..

[25]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  J. Tukey,et al.  The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data , 1974 .

[27]  S. B. Kang,et al.  Recovering 3 D Shape and Motion from Image Streams using Non-Linear Least Squares , 1993 .

[28]  Philip E. Gill,et al.  Practical optimization , 1981 .

[29]  Timothy F. Cootes,et al.  Locating Objects of Varying Shape Using Statistical Feature Detectors , 1996, ECCV.

[30]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).