Image-based re-rendering of faces for continuous pose and illumination directions

A scheme for photorealistic re-rendering of faces at arbitrary illumination and pose directions is presented. Our approach relies entirely on image-based rendering techniques. We utilize a scheme for re-illumination of objects based on lineal combination of low-dimensional image representations and incorporate image morphing into the image synthesis process. Important properties of our solution are simplicity and robustness. We provide experimental results to demonstrate the performance of the proposed re-rendering method on synthetic and real-world images.

[1]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[2]  Tomaso Poggio,et al.  Image Representations for Visual Learning , 1996, Science.

[3]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[4]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[5]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1998 .

[6]  Peter W. Hallinan.  A low-dimensional representation of human faces for arbitrary lighting conditions , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Tomaso A. Poggio,et al.  A bootstrapping algorithm for learning linear models of object classes , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[9]  Amnon Shashua,et al.  The quotient image: Class based recognition and synthesis under varying illumination conditions , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[10]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Julie Dorsey,et al.  Effic ient Re-rendering of Naturally Illuminated Environments , 1994 .

[12]  David J. Kriegman,et al.  Illumination cones for recognition under variable lighting: faces , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[13]  Amnon Shashua,et al.  Novel view synthesis in tensor space , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  George Wolberg,et al.  Digital image warping , 1990 .

[15]  Sung Yong Shin,et al.  Image metamorphosis using snakes and free-form deformations , 1995, SIGGRAPH.