Mixtures of eigenfeatures for real-time structure from texture

We describe a face modeling system which estimates complete facial structure and texture from a real-time video stream. The system begins with a face trading algorithm which detects and stabilizes live facial images into a canonical 3D pose. The resulting canonical texture is then processed by a statistical model to filter imperfections and estimate unknown components such as missing pixels and underlying 3D structure. This statistical model is a soft mixture of eigenfeature selectors which span the 3D deformations and texture changes across a training set of laser scanned faces. An iterative algorithm is introduced for determining the dimensional partitioning of the eigenfeatures to maximize their generalization capability over a cross-validation set of data. The model's abilities to filter and estimate absent facial components are then demonstrated over incomplete 3D data. This ultimately allows the model to span known and regress unknown facial information front stabilized natural video sequences generated by a face tracking algorithm. The resulting continuous and dynamic estimation of the model's parameters over a video sequence generates a compact temporal description of the 3D deformations and texture changes of the face.

[1]  G. Sela,et al.  Fast computation of multiscalar symmetry in foveated images , 1995, Proceedings of Conference on Computer Architectures for Machine Perception.

[2]  P PentlandAlex,et al.  Recursive Estimation of Motion, Structure, and Focal Length , 1995 .

[3]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[4]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Tony Jebara,et al.  3D Pose Estimation and Normalization for Face Recognition , 1995 .

[6]  Alex Pentland,et al.  Recursive Estimation of Motion, Structure, and Focal Length , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Dimitris N. Metaxas,et al.  Deformable model-based face shape and motion estimation , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[8]  Alex Pentland,et al.  Probabilistic visual learning for object detection , 1995, Proceedings of IEEE International Conference on Computer Vision.

[9]  Yehezkel Yeshurun,et al.  Robust detection of facial features by generalized symmetry , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[10]  Gregory D. Hager,et al.  Real-time tracking of image regions with changes in geometry and illumination , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.