Face replacement with large-pose differences

In this paper, we present a novel face replacement system exchanging faces with large-pose differences. Traditional 2D image based face replacement can only replace faces with similar pose and appearance. This significantly limits the application of face replacement. In this paper, we propose to build a 3D head model from a single frontal face photo. The automatically constructed 3D head can be rendered under arbitrary poses and illuminations. This makes it possible to do swapping for faces with large pose variations. In the demo, the user captures a frontal face image using a capture device such as a webcam or a smartphone, and then the algorithm can automatically build the 3D model using feature detection, face alignment and reconstruction. This 3D model is used to swap to any other target face photo the user selects. While our system is automatic, we also provide interactive tools for the user to adjust the feature detection to enhance the results.