Face replacement with large-pose differences

In this paper, we present a novel face replacement system exchanging faces with large-pose differences. Traditional 2D image based face replacement can only replace faces with similar pose and appearance. This significantly limits the application of face replacement. In this paper, we propose to build a 3D head model from a single frontal face photo. The automatically constructed 3D head can be rendered under arbitrary poses and illuminations. This makes it possible to do swapping for faces with large pose variations. In the demo, the user captures a frontal face image using a capture device such as a webcam or a smartphone, and then the algorithm can automatically build the 3D model using feature detection, face alignment and reconstruction. This 3D model is used to swap to any other target face photo the user selects. While our system is automatic, we also provide interactive tools for the user to adjust the feature detection to enhance the results.

[1]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, SIGGRAPH 2008.

[2]  Luc Van Gool,et al.  Unsupervised face alignment by robust nonrigid mapping , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[3]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[4]  Yuan Lin,et al.  Face Swapping under Large Pose Variations: A 3D Model Based Approach , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[5]  Hans-Peter Seidel,et al.  Exchanging Faces in Images , 2004, Comput. Graph. Forum.

[6]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.