Face-off: automatic alteration of facial features

Pursuing or maintaining beautifulness nowadays has become a trend in modern society, especially among the celebrity community. In some cases, one may choose to adopt drastic procedures to alter his or her facial or body features to achieve the desired beauty, thus the blossom of industry on cosmetic plastic surgeries. In addition, people whose faces got damaged due to accidental burns or wounds may also find these surgeries necessary. However, as performing the related surgeries are still considered intrusive and costly, it is better to “preview” the result before a surgery is actually carried out. As many believe that facial appearance matters most, we have developed a system that allows a user to input a photo and changes the associated individual facial feature in an automatic and user-friendly manner. Overall speaking, our system makes contributions in the following four aspects. First, our system not only offers the previewing functionality, but also allows users to interactively fine-tune the desired results, thus making it a useful companion tool for facial cosmetic surgeries. Second, instead of exchanging the overall look of a face, as being done by some existing approaches, our system offers much finer granularity by allowing each and every facial feature to be changed individually and independently, thus achieving higher face-off flexibility. Third, while existing tools generally entail manual effort to locate or align facial features, our system, through the help of Active Shape Model or ASM for short, characterized by a scheme of automatic feature extraction, eliminates most of the needs of user assistance. Finally, for convenience, we have constructed a database of facial features to facilitate the facial feature alteration process. To justify our claims, we have rendered results and compared them with those from existing approaches to demonstrate the effectiveness of our system. We have also conducted a user study to further confirm the usefulness of such a system.

[1]  Hans-Peter Seidel,et al.  Exchanging Faces in Images , 2004, Comput. Graph. Forum.

[2]  Harry Shum,et al.  To appear in the ACM SIGGRAPH conference proceedings Drag-and-Drop Pasting , 2022 .

[3]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[4]  Manuel Menezes de Oliveira Neto,et al.  Fast Digital Image Inpainting , 2001, VIIP.

[5]  Alexei A. Efros,et al.  Photo clip art , 2007, ACM Trans. Graph..

[6]  Dani Lischinski,et al.  Data-driven enhancement of facial attractiveness , 2008, ACM Trans. Graph..

[7]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[8]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[9]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Chuan-Kai Yang,et al.  An interactive facial expression generation system , 2008, Multimedia Tools and Applications.

[11]  Dani Lischinski,et al.  Digital face beautification , 2006, SIGGRAPH '06.

[12]  Shmuel Peleg,et al.  Seamless Image Stitching in the Gradient Domain , 2004, ECCV.

[13]  Richard Szeliski,et al.  Eliminating ghosting and exposure artifacts in image mosaics , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  Shmuel Peleg,et al.  Elimination of seams from photomosaics , 1981, Computer Graphics and Image Processing.

[15]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, SIGGRAPH 2008.

[16]  William A. Barrett,et al.  Intelligent scissors for image composition , 1995, SIGGRAPH.

[17]  Michael Cohen,et al.  Soft scissors: an interactive tool for realtime high quality matting , 2007, SIGGRAPH 2007.

[18]  H. Gu,et al.  Feature Points Extraction from Faces , 2003 .

[19]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, ACM Trans. Graph..

[20]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[21]  Jian Sun,et al.  Drag-and-drop pasting , 2006, SIGGRAPH 2006.

[22]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[23]  John R. Kender,et al.  Finding skin in color images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[24]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[25]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[26]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[27]  Jiaya Jia,et al.  Image completion with structure propagation , 2005, SIGGRAPH 2005.

[28]  Louis Laborelli,et al.  Missing data correction in still images and image sequences , 2002, MULTIMEDIA '02.

[29]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[30]  Timothy F. Cootes,et al.  Statistical models of appearance for medical image analysis and computer vision , 2001, SPIE Medical Imaging.

[31]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[32]  Alexei A. Efros,et al.  Photo clip art , 2007, SIGGRAPH 2007.