The integration of optical flow and deformable models with applications to human face shape and motion estimation

We present a formal methodology for the integration of optical flow and deformable models. The optical flow constraint equation provides a non-holonomic constraint on the motion of the deformable model. In this augmented system, forces computed from edges and optical flow are used simultaneously. When this dynamic system is solved, a model-based least-squares solution for the optical flow is obtained and improved estimation results are achieved. The use of a 3-D model reduces or eliminates problems associated with optical flow computation. This approach instantiates a general methodology for treating visual cues as constraints on deformable models. We apply this framework to human face shape and motion estimation. Our 3-D deformable face model uses a small number of parameters to describe a rich variety of face shapes and facial expressions. We present experiments in extracting the shape and motion of a face from image sequences.

[1]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[2]  Alex Pentland,et al.  Facial expression recognition using a dynamic model and motion energy , 1995, Proceedings of IEEE International Conference on Computer Vision.

[3]  Dimitris N. Metaxas,et al.  Blended deformable models , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Ahmed A. Shabana,et al.  Dynamics of Multibody Systems , 2020 .

[5]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[6]  Edward H. Adelson,et al.  Probability distributions of optical flow , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Larry S. Davis,et al.  Computing spatio-temporal representations of human faces , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[8]  James S. Duncan,et al.  A model-based integrated approach to track myocardial deformation using displacement and velocity constraints , 1995, Proceedings of IEEE International Conference on Computer Vision.

[9]  Timothy F. Cootes,et al.  A unified approach to coding and interpreting face images , 1995, Proceedings of IEEE International Conference on Computer Vision.

[10]  Dimitris N. Metaxas,et al.  Dynamic deformation of solid primitives with constraints , 1992, SIGGRAPH.

[11]  Demetri Terzopoulos,et al.  Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  L. Farkas.  Anthropometry of the head and face , 1994 .

[13]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[14]  Dimitris N. Metaxas,et al.  Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Pertti Roivainen,et al.  3-D Motion Estimation in Model-Based Facial Image Coding , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Demetri Terzopoulos,et al.  Constraints on Deformable Models: Recovering 3D Shape and Nonrigid Motion , 1988, Artif. Intell..

[17]  Alex Pentland,et al.  Closed-Form Solutions for Physically Based Shape Modeling and Recognition , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Eero P. Simoncelli.  Design of multi-dimensional derivative filters , 1994, Proceedings of 1st International Conference on Image Processing.