Automatic face replacement in photographs based on Active Shape Models

In this paper, we present an automatic face replacement approach in photographs based on Active Shape Models (ASM). Our replacement algorithm has three main modules: face alignment, face morph, and seamless blending. First, given an input and target image, we automatically implement face alignment using ASM. Second, we fit a morphable 2D model to the input face, and warp its shape to match the target face shape. Last, we adjust the lighting and color of the target face image to match those of the input face image, and seamlessly blend the target face image into the input image. Our approach is 2D model-based, fully automatic, and generates natural results. The improved results demonstrate the effectiveness of our approach.

[1]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[2]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[3]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[4]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, SIGGRAPH 2008.

[5]  Shahzad Malik,et al.  Digital Face Replacement in Photographs , 2003 .

[6]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[8]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[9]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[10]  David Salesin,et al.  Interactive digital photomontage , 2004, ACM Trans. Graph..

[11]  Ning Xu,et al.  Videoshop: A new framework for spatio-temporal video editing in gradient domain , 2007, Graph. Model..