Example-driven modeling of portrait bas-relief

Abstract Traditional portrait bas-relief modeling methods are limited by the shape of the input 3D model. In this paper, we introduce a novel approach to generate portrait bas-relief with various poses and expressions. In the alignment stage, we segment the input model into several meaningful regions so that the facial features of each region can be separately aligned. To guaranty the quality of expression simulation, we consider both the landmarks of fitted 3DMM model and input portrait image to drive the bi-Laplacian deformation process. After alignment, we apply normal enhancement in the deformed model, and the bas-relief is achieved by solving a sparse linear system in the normal field. Furthermore, the input model is transformed into a compressed height field to be the constraint in bas-relief modeling. The final bas-relief preserves the subtle details and depth structure. Experimental results demonstrate that our method is effective in generating the attractive appearance of portrait bas-reliefs.

[1]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[2]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Marc Alexa,et al.  As-rigid-as-possible surface modeling , 2007, Symposium on Geometry Processing.

[4]  Stephen M. Pizer,et al.  Structure-sensitive adaptive contrast enhancement methods and their evaluation , 1993, Image Vis. Comput..

[5]  Chi-Wing Fu,et al.  Interactive High-Relief Reconstruction for Organic and Double-Sided Objects from a Photo , 2017, IEEE Transactions on Visualization and Computer Graphics.

[6]  Tal Hassner,et al.  Emotion Recognition in the Wild via Convolutional Neural Networks and Mapped Binary Patterns , 2015, ICMI.

[7]  Hans-Peter Seidel,et al.  Feature sensitive bas relief generation , 2009, 2009 IEEE International Conference on Shape Modeling and Applications.

[8]  Iasonas Kokkinos,et al.  DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Charlie C. L. Wang,et al.  Bas-Relief Modeling from Normal Layers , 2019, IEEE Transactions on Visualization and Computer Graphics.

[10]  Caiming Zhang,et al.  Adaptive Bas‐relief Generation from 3D Object under Illumination , 2016, Comput. Graph. Forum.

[11]  Hans-Peter Seidel,et al.  Computer Assisted Relief Generation—A Survey , 2012, Comput. Graph. Forum.

[12]  Zhigang Deng,et al.  Smooth skinning decomposition with rigid bones , 2012, ACM Trans. Graph..

[13]  Yutaka Ohtake,et al.  A comparison of mesh smoothing methods , 2003 .

[14]  Weiyin Ma,et al.  Bas-Relief Modeling from Normal Images with Intuitive Styles , 2014, IEEE Transactions on Visualization and Computer Graphics.

[15]  Matan Sela,et al.  Learning Detailed Face Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Olga Sorkine-Hornung,et al.  Appearance-mimicking surfaces , 2014, ACM Trans. Graph..

[17]  Maja Pantic,et al.  Optimization Problems for Fast AAM Fitting in-the-Wild , 2013, 2013 IEEE International Conference on Computer Vision.

[18]  Weiyin Ma,et al.  Normal Image Manipulation for Bas-relief Generation with Hybrid Styles , 2018, ArXiv.

[19]  Xi Zhou,et al.  Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network , 2018, ECCV.

[20]  Gang Yu,et al.  Real-time bas-relief generation from a 3D mesh , 2013, Graph. Model..

[21]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[22]  Àgata Lapedriza,et al.  Emotion Recognition in Context , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[24]  Roland Göcke,et al.  A Nonlinear Discriminative Approach to AAM Fitting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[25]  Xiangyu Zhu,et al.  Face Alignment in Full Pose Range: A 3D Total Solution , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Yu-Wei Zhang,et al.  Line-based sunken relief generation from a 3D mesh , 2013, Graph. Model..

[27]  Ramakant Nevatia,et al.  ExpNet: Landmark-Free, Deep, 3D Facial Expressions , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[28]  Yu-Wei Zhang,et al.  Bas-Relief Generation and Shape Editing through Gradient-Based Mesh Deformation , 2015, IEEE Transactions on Visualization and Computer Graphics.

[29]  Sami Romdhani,et al.  A 3D Face Model for Pose and Illumination Invariant Face Recognition , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[30]  Georgios Tzimiropoulos,et al.  How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Hao Li,et al.  Learning Dense Facial Correspondences in Unconstrained Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[32]  Shi-Min Hu,et al.  Preserving detailed features in digital bas-relief making , 2011, Comput. Aided Geom. Des..

[33]  Georgios Tzimiropoulos,et al.  Two-Stage Convolutional Part Heatmap Regression for the 1st 3D Face Alignment in the Wild (3DFAW) Challenge , 2016, ECCV Workshops.

[34]  Jesse Hoey,et al.  From individual to group-level emotion recognition: EmotiW 5.0 , 2017, ICMI.

[35]  Paolo Cignoni,et al.  Computer-Assisted Generation of Bas-and High-Reliefs , 1997, J. Graphics, GPU, & Game Tools.

[36]  Adam Finkelstein,et al.  Digital bas-relief from 3D scenes , 2007, ACM Trans. Graph..

[37]  Ralph R. Martin,et al.  Bas-relief Generation Using Adaptive Histogram Equalisation , 2022 .

[38]  Dani Lischinski,et al.  Gradient Domain High Dynamic Range Compression , 2023 .

[39]  Caiming Zhang,et al.  Portrait relief generation from 3D Object , 2019, Graph. Model..

[40]  Hui Liu,et al.  Portrait Relief Modeling from a Single Image , 2020, IEEE Transactions on Visualization and Computer Graphics.

[41]  Olga Sorkine-Hornung,et al.  On Linear Variational Surface Deformation Methods , 2008, IEEE Transactions on Visualization and Computer Graphics.