Facial Skin Beautification Using Adaptive Region-Aware Masks

In this paper, we propose a unified facial beautification framework with respect to skin homogeneity, lighting, and color. A novel region-aware mask is constructed for skin manipulation, which can automatically select the edited regions with great precision. Inspired by the state-of-the-art edit propagation techniques, we present an adaptive edge-preserving energy minimization model with a spatially variant parameter and a high-dimensional guided feature space for mask generation. Using region-aware masks, our method facilitates more flexible and accurate facial skin enhancement while the complex manipulations are simplified considerably. In our beautification framework, a portrait is decomposed into smoothness, lighting, and color layers by an edge-preserving operator. Next, facial landmarks and significant features are extracted as input constraints for mask generation. After three region-aware masks have been obtained, a user can perform facial beautification simply by adjusting the skin parameters. Furthermore, the combinations of parameters can be optimized automatically, depending on the data priors and psychological knowledge. We performed both qualitative and quantitative evaluation for our method using faces with different genders, races, ages, poses, and backgrounds from various databases. The experimental results demonstrate that our technique is superior to previous methods and comparable to commercial systems, for example, PicTreat, Portrait+, and Portraiture.

[1]  Lianwen Jin,et al.  Facial Skin Beautification Using Region-Aware Mask , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[2]  R. Szeliski Locally adapted hierarchical basis preconditioning , 2006, SIGGRAPH 2006.

[3]  Fabio Pellacini,et al.  AppProp: all-pairs appearance-space edit propagation , 2008, ACM Trans. Graph..

[4]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[5]  Stella J. Faerber,et al.  African Perceptions of Female Attractiveness , 2012, PloS one.

[6]  Chi-Keung Tang,et al.  Example-Based Cosmetic Transfer , 2007 .

[7]  알렉세이 포소신,et al.  Automatic face and skin beautification using face detection , 2009 .

[8]  Eytan Ruppin,et al.  Facial Attractiveness: Beauty and the Machine , 2006, Neural Computation.

[9]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[10]  Xuelong Li,et al.  Saliency Detection by Multiple-Instance Learning , 2013, IEEE Transactions on Cybernetics.

[11]  Heng Liu,et al.  Portrait beautification: A fast and robust approach , 2007, Image Vis. Comput..

[12]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[13]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[14]  Honglak Lee,et al.  Augmenting CRFs with Boltzmann Machine Shape Priors for Image Labeling , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Dani Lischinski,et al.  Data-driven enhancement of facial attractiveness , 2008, ACM Trans. Graph..

[16]  Jintu Fan,et al.  Prediction of facial attractiveness from facial proportions , 2012, Pattern Recognit..

[17]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[20]  Hans-Peter Seidel,et al.  Computer‐Suggested Facial Makeup , 2011, Comput. Graph. Forum.

[21]  Randy Thornhill,et al.  Facial attractiveness , 1999, Trends in Cognitive Sciences.

[22]  Derek Nowrouzezahrai,et al.  Learning hatching for pen-and-ink illustration of surfaces , 2012, TOGS.

[23]  Jean-Marc Odobez,et al.  Joint Adaptive Colour Modelling and Skin, Hair and Clothes Segmentation using Coherent Probabilistic Index Maps , 2011, BMVC.

[24]  Shree K. Nayar,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Describable Visual Attributes for Face Verification and Image Search , 2022 .

[25]  Sylvain Paris,et al.  Learning photographic global tonal adjustment with a database of input / output image pairs , 2011, CVPR 2011.

[26]  Dani Lischinski,et al.  Digital face beautification , 2006, SIGGRAPH '06.

[27]  Xuelong Li,et al.  L1-Norm-Based 2DPCA , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  D. Perrett,et al.  Cross-cultural effects of color, but not morphological masculinity, on perceived attractiveness of men's faces , 2012 .

[29]  Helmut Pottmann Geometric Computing in Shape Space , 2007 .

[30]  Zeev Farbman,et al.  Interactive local adjustment of tonal values , 2006, ACM Trans. Graph..

[31]  Jan P. Allebach,et al.  An algorithm for automatic skin smoothing in digital portraits , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[32]  Hong Qiao,et al.  An Explicit Nonlinear Mapping for Manifold Learning , 2010, IEEE Transactions on Cybernetics.

[33]  Dani Lischinski,et al.  Personalization of image enhancement , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Chih-Wei Chen,et al.  Automatic Skin Color Beautification , 2009, ArtsIT.

[35]  Jian Sun,et al.  Face Alignment by Explicit Shape Regression , 2012, International Journal of Computer Vision.

[36]  K. Grammer,et al.  Color homogeneity and visual perception of age, health, and attractiveness of female facial skin. , 2007, Journal of the American Academy of Dermatology.

[37]  Richard Russell,et al.  Sex, Beauty, and the Relative Luminance of Facial Features , 2003, Perception.

[38]  Xuelong Li,et al.  Initialization Independent Clustering With Actively Self-Training Method , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[39]  Raanan Fattal,et al.  Diffusion maps for edge-aware image editing , 2010, SIGGRAPH 2010.

[40]  Kun Zhou,et al.  Single-view hair modeling for portrait manipulation , 2012, ACM Trans. Graph..

[41]  Alexei A. Efros,et al.  Image‐based Shaving , 2008, Comput. Graph. Forum.

[42]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[43]  Xiaowu Chen,et al.  Manifold preserving edit propagation , 2012, ACM Trans. Graph..

[44]  Sung Yong Shin,et al.  Scattered Data Interpolation with Multilevel B-Splines , 1997, IEEE Trans. Vis. Comput. Graph..

[45]  Edward H. Adelson,et al.  Personal photo enhancement using example images , 2010, TOGS.

[46]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[47]  D. Perrett,et al.  Symmetry and human facial attractiveness. , 1999 .

[48]  Dong Guo,et al.  Digital face makeup by example , 2009, CVPR.

[49]  Michael Beetz,et al.  Towards Unconstrained Face Recognition Using 3D Face Model , 2011 .

[50]  D. Perrett,et al.  Facial Skin Coloration Affects Perceived Health of Human Faces , 2009, International Journal of Primatology.

[51]  Nan Wang,et al.  What are good parts for hair shape modeling? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Denise C. Park,et al.  A lifespan database of adult facial stimuli , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[53]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[54]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[55]  Zeev Farbman,et al.  Edge-preserving decompositions for multi-scale tone and detail manipulation , 2008, SIGGRAPH 2008.

[56]  Mihai Ciuc,et al.  Facial enhancement and beautification for HD video cameras , 2011, 2011 IEEE International Conference on Consumer Electronics (ICCE).

[57]  David Zhang,et al.  Quantitative analysis of human facial beauty using geometric features , 2011, Pattern Recognit..