Face replacement based on 2D dense mapping

Face replacement, also known as face swapping, is a worth investing problem in privacy protection, intelligent human-computer interaction and video effects. In our paper, a new approach is proposed to transform identities of the input images into a target identity while preserving poses and facial features. It is considered that 2D face replacement can be replaced by the problem of 2D graphics dense mapping using 3D morphable models projection. For mapping the input facial region and the target, our method uses planar parameterization and affine transformation to establish a one-to-one dense mapping between 2D graphics. It can enable to reduce the effect on the rotation of non-frontal faces without image preprocessing. Our method then adjusts positions of eyes to keep gaze direction of the input images, and stitches the transformed image and the target image based on seamless cloning to show the authenticity of the images.

[1]  Andrea Vedaldi,et al.  Texture Networks: Feed-forward Synthesis of Textures and Stylized Images , 2016, ICML.

[2]  Luc Van Gool,et al.  Deep Expectation of Real and Apparent Age from a Single Image Without Facial Landmarks , 2016, International Journal of Computer Vision.

[3]  Hiroshi Ishikawa,et al.  Globally and locally consistent image completion , 2017, ACM Trans. Graph..

[4]  Bernt Schiele,et al.  A Hybrid Model for Identity Obfuscation by Face Replacement , 2018, ECCV.

[5]  Luc Van Gool,et al.  DEX: Deep EXpectation of Apparent Age from a Single Image , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[6]  Ivan Sikiric,et al.  I Know That Person: Generative Full Body and Face De-identification of People in Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Hans-Peter Seidel,et al.  A fast and simple stretch-minimizing mesh parameterization , 2004, Proceedings Shape Modeling Applications, 2004..

[8]  Seong Joon Oh,et al.  Faceless Person Recognition: Privacy Implications in Social Media , 2016, ECCV.

[9]  William A. P. Smith,et al.  Fitting a 3D Morphable Model to Edges: A Comparison Between Hard and Soft Correspondences , 2016, ACCV Workshops.

[10]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[11]  Jong-Il Park,et al.  The image blending method for face swapping , 2014, 2014 4th IEEE International Conference on Network Infrastructure and Digital Content.

[12]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[13]  Sami Romdhani,et al.  A 3D Face Model for Pose and Illumination Invariant Face Recognition , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[14]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, SIGGRAPH 2008.

[15]  Lucas Theis,et al.  Fast Face-Swap Using Convolutional Neural Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Luc Van Gool,et al.  Natural and Effective Obfuscation by Head Inpainting , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Ira Kemelmacher-Shlizerman,et al.  What Makes Tom Hanks Look Like Tom Hanks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Wojciech Matusik,et al.  Video face replacement , 2011, ACM Trans. Graph..

[19]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .