Generalization to Novel Views from a Single Face Image

When only a single image of a face is available, can we generate new images of the face across changes in viewpoint or illumination? The approach presented in this paper acquires its knowledge about possible image changes from other faces and transfers this prior knowledge to a novel face image. In previous work we introduced the concept of linear object classes (Vetter and Poggio, 1997; Vetter, 1997): In an image based approach, a flexible image model of faces was used to synthesize new images of a face when only a single 2D image of that face is available.

[1]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[2]  Hiroshi Harashima,et al.  A system of analyzing and synthesizing facial images , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[3]  A. O'Toole,et al.  Three-Dimensional Caricatures of Human Heads: Distinctiveness and the Perception of Facial Age , 1997, Perception.

[4]  Ulf Grenander,et al.  Lectures in pattern theory , 1978 .

[5]  D. Perrett,et al.  Perception and recognition of photographic quality facial caricatures: Implications for the recognition of natural images , 1991 .

[6]  Tomaso Poggio,et al.  Image Representations for Visual Learning , 1996, Science.

[7]  S. Kosslyn.  Image and brain: the resolution of the imagery debate , 1994 .

[8]  D. Mumford.  Pattern theory: a unifying perspective , 1996 .

[9]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[10]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1995, Proceedings of IEEE International Conference on Computer Vision.

[11]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[12]  Tomaso A. Poggio,et al.  A bootstrapping algorithm for learning linear models of object classes , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Timothy F. Cootes,et al.  Automatic interpretation of human faces and hand gestures using flexible models. , 1995 .

[14]  T. Poggio,et al.  Recognition and Structure from one 2D Model View: Observations on Prototypes, Object Classes and Symmetries , 1992 .

[15]  Rainer Herpers,et al.  Edge and keypoint detection in facial regions , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[16]  Tomaso Poggio,et al.  Example Based Image Analysis and Synthesis , 1993 .

[17]  Takaaki Akimoto,et al.  Automatic creation of 3D facial models , 1993, IEEE Computer Graphics and Applications.

[18]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  D. Mumford.  Perception as Bayesian Inference: Pattern theory: A unifying perspective , 1996 .

[20]  A. O'Toole,et al.  Structural aspects of face recognition and the other-race effect , 1994, Memory & cognition.

[21]  S. Ullman.  Aligning pictorial descriptions: An approach to object recognition , 1989, Cognition.