Nonrigid Motion Modeling and Analysis in Video Sequences for Realistic Facial Animation

To achieve realistic facial animation, it is important to have an accurate articulation model. Currently, most animation software obtains this model using interactive graphics tools and key frame techniques, which sometimes are tedious and inaccurate. A promising alternative approach is to learn articulation model from real video sequences. A new explanation-based method is introduced in this thesis to handle this problem. Three closely related topics are (a) a new facial articulation model using piecewise Bezier volume deformation, (b) Three-dimensional model-based analysis of face/head motions, and (c) explanation-based learning of facial articulation model. The piecewise Bezier volume deformation model is generated from facial fiducial points and, therefore, is independent of the topology of the actual mesh model. This property is crucial in deforming face models of various structures. Additionally, this deformation model is linear and suitable for analysis. Based on this deformation model, some actuation units are predefined and applied to extract head/face motions under a model-based framework. A least square estimator gives robust tracking results in terms of action units intensities. This approach is sufficient if tracking is the major concern. However, in order to customize the facial motion of each individual, the predefined deformation model (namely, a set of action units) should be modified adaptively. An explanation-based approach uses the results from the model-based method as the initial guess or explanation. At the learning stage, action units are modified and new action units are added if necessary. The tracking results are refined and can be used for animation. Statistical motion analysis and Bezier degree elevation algorithm are the two major tools to accomplish this task. Details of this algorithm will be explained and some face animation results on interactive approach, model-based approach, and explanation-based learning approach will be demonstrated. Topics also covered in this thesis are a two-dimensional modal-analysis-based tracking algorithm called the connected vibrations method, the expression/visual speech level analysis of facial motion, and facial animation standard in MPEG-4.