Multidimensional morphable models

We describe a flexible model for representing images of objects of a certain class, known a priori, such as faces, and introduce a new algorithm for matching it to a novel image and thereby performing image analysis. We call this model a multidimensional morphable model or just a, morphable model. The morphable model is learned from example images (called prototypes) of objects of a class. In this paper we introduce an effective stochastic gradient descent algorithm that automaticaIly matches a model to a novel image by finding the parameters that minimize the error between the image generated by the model and the novel image. Two examples demonstrate the robustness and the broad range of applicability of the matching algorithm and the underlying morphable model. Our approach can provide novel solutions to several vision tasks, including the computation of image correspondence, object verification, image synthesis and image compression.

[1]  A. Posner.  Learning to see. , 1955, Eye, ear, nose & throat monthly.

[2]  P. J. Burt,et al.  The Pyramid as a Structure for Efficient Computation , 1984 .

[3]  Ramesh C. Jain,et al.  Three-dimensional object recognition , 1985, CSUR.

[4]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  T. Poggio.  A theory of how the brain might work. , 1990, Cold Spring Harbor symposia on quantitative biology.

[6]  Ronen Basri,et al.  Recognition by Linear Combinations of Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Hiroshi Harashima,et al.  A system of analyzing and synthesizing facial images , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[8]  Timothy F. Cootes,et al.  Active Shape Models - 'smart snakes' , 1992, BMVC.

[9]  T. Poggio,et al.  Recognition and Structure from one 2D Model View: Observations on Prototypes, Object Classes and Symmetries , 1992 .

[10]  Tomaso Poggio,et al.  A Novel Approach to Graphics , 1992 .

[11]  Tomaso Poggio,et al.  Example Based Image Analysis and Synthesis , 1993 .

[12]  Timothy F. Cootes,et al.  Using grey-level models to improve active shape model search , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[13]  Amnon Shashua,et al.  Projective Structure from Uncalibrated Images: Structure From Motion and Recognition , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Peter W. Hallinan,et al.  A deformable model for the recognition of human faces under arbitrary illumination , 1995 .

[15]  H H Bülthoff,et al.  How are three-dimensional objects represented in the brain? , 1994, Cerebral cortex.

[16]  David J. Beymer,et al.  Pose-invariant face recognition using real and virtual views , 1996 .

[17]  David Beymer,et al.  Face recognition from one example view , 1995, Proceedings of IEEE International Conference on Computer Vision.

[18]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1995, Proceedings of IEEE International Conference on Computer Vision.

[19]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[20]  Tomaso A. Poggio,et al.  Model-based matching of line drawings by linear combinations of prototypes , 1995, Proceedings of IEEE International Conference on Computer Vision.

[21]  Timothy F. Cootes,et al.  A unified approach to coding and interpreting face images , 1995, Proceedings of IEEE International Conference on Computer Vision.

[22]  Tomaso Poggio,et al.  Image Representations for Visual Learning , 1996, Science.

[23]  Michael J. Jones,et al.  Model-Based Matching by Linear Combinations of Prototypes , 1996 .

[24]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Tomaso A. Poggio,et al.  A bootstrapping algorithm for learning linear models of object classes , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.