3-D Motion Estimation in Model-Based Facial Image Coding

An approach to estimating the motion of the head and facial expressions in model-based facial image coding is presented. An affine nonrigid motion model is set up. The specific knowledge about facial shape and facial expression is formulated in this model in the form of parameters. A direct method of estimating the two-view motion parameters that is based on the affine method is discussed. Based on the reasonable assumption that the 3-D motion of the face is almost smooth in the time domain, several approaches to predicting the motion of the next frame are proposed. Using a 3-D model, the approach is characterized by a feedback loop connecting computer vision and computer graphics. Embedding the synthesis techniques into the analysis phase greatly improves the performance of motion estimation. Simulations with long image sequences of real-world scenes indicate that the method not only greatly reduces computational complexity but also substantially improves estimation accuracy. >

[1]  A. Sommerfeld.  Mechanics of Deformable Bodies , 1950 .

[2]  Robert F. Sproull,et al.  Principles in interactive computer graphics , 1973 .

[3]  Thomas S. Huang,et al.  Estimating three-dimensional motion parameters of a rigid planar patch , 1981 .

[4]  Azriel Rosenfeld,et al.  "cvgip" , 1983, Comput. Vis. Graph. Image Process..

[5]  Thomas S. Huang,et al.  Image Sequence Processing and Dynamic Scene Analysis , 1983, NATO ASI Series.

[6]  T. S. Huang,et al.  Image Sequence Processing and Dynamic Analysis , 1983 .

[7]  J. Salz,et al.  Algorithms for estimation of three-dimensional motion , 1985, AT&T Technical Journal.

[8]  Rama Chellappa,et al.  Estimation of Object Motion Parameters from Noisy Images , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Su-Shing Chen,et al.  Shape and motion of nonrigid bodies , 1986, Comput. Vis. Graph. Image Process..

[10]  Ishwar K. Sethi,et al.  Finding Trajectories of Feature Points in a Monocular Image Sequence , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Demetri Terzopoulos,et al.  Constraints on Deformable Models: Recovering 3D Shape and Nonrigid Motion , 1988, Artif. Intell..

[12]  D. Shulman,et al.  (Non-)rigid motion interpretation : a regularized approach , 1988, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[13]  N. Magnenat-Thalmann,et al.  Image Synthesis Theory and Practice , 1988 .

[14]  N. D. Duffy,et al.  A Texture Mapping Approach to 3‐D Facial Image Synthesis , 1988, Comput. Graph. Forum.

[15]  F Kappei,et al.  Modelling Of A Natural 3-D Scene Consisting Of Moving Objects From A Sequence Of Monocular Tv Images , 1988, Other Conferences.

[16]  Jake K. Aggarwal,et al.  On the computation of motion from sequences of images-A review , 1988, Proc. IEEE.

[17]  Robert Forchheimer,et al.  Image coding-from waveforms in animation , 1989, IEEE Trans. Acoust. Speech Signal Process..

[18]  Gilad Adiv,et al.  Inherent Ambiguities in Recovering 3-D Motion and Structure from a Noisy Flow Field , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  H. Busch,et al.  Subdividing non-rigid 3D objects into quasi-rigid parts , 1989 .

[20]  Jörn Ostermann,et al.  Object-oriented analysis-synthesis coding of moving images , 1989, Signal Process. Image Commun..

[21]  Don E. Pearson,et al.  Texture mapping in model-based image coding , 1990, Signal Process. Image Commun..

[22]  Demetri Terzopoulos,et al.  Analysis of facial images using physical and anatomical models , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[23]  Thomas S. Huang,et al.  Modeling, analysis, and visualization of nonrigid object motion , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[24]  Masahide Kaneko,et al.  Coding of facial image sequence based on a 3-D model of the head and motion detection , 1991, J. Vis. Commun. Image Represent..

[25]  Bill Welsh,et al.  Model-based coding of images , 1991 .

[26]  Alex Pentland,et al.  Recovery of Nonrigid Motion and Structure , 1991, IEEE Trans. Pattern Anal. Mach. Intell..