Using model-driven bundle-adjustment to model heads from raw video sequences

We show that we can effectively and automatically fit a complex facial animation model to uncalibrated image sequences. Our approach is based on model-driven bundle adjustment followed by least squares fitting. It takes advantage of three complementary sources of information: stereo data, silhouette edges and 2D feature points. In this way, complete head models can be acquired with a cheap and entirely passive sensor such as an ordinary video camera. They can then be fed to existing animation software to produce synthetic sequences.

[1]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Yiannis Aloimonos,et al.  Changes in Surface Convexity and Topology Caused by Distortions of Stereoscopic Visual Space , 1998, ECCV.

[3]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[4]  Thierry Viéville,et al.  Canonical Representations for the Geometries of Multiple Projective Views , 1996, Comput. Vis. Image Underst..

[5]  Andrew Zisserman,et al.  Resolving ambiguities in auto–calibration , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[6]  J. Tukey,et al.  The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data , 1974 .

[7]  Horst A. Beyer,et al.  System Calibration Through Self-Calibration , 2001 .

[8]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[9]  Jean-Philippe Tarel,et al.  Camcal v1.0 Manual A Complete Software Solution for Camera Calibration , 1996 .

[10]  Peter F. Sturm,et al.  Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Pascal Fua,et al.  From Regular Images to Animated Heads: A Least Squares Approach , 1998, ECCV.

[12]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[13]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[14]  Daniel Thalmann,et al.  Simulation of Facial Muscle Actions Based on Rational Free Form Deformations , 1992, Comput. Graph. Forum.