Portrait Relief Modeling from a Single Image

We present a novel solution to enable portrait relief modeling from a single image. The main challenges are geometry reconstruction, facial details recovery and depth structure preservation. Previous image-based methods are developed for portrait bas-relief modeling in 2.5D form, but not adequate for 3D-like high relief modeling with undercut features. In this paper, we propose a template-based framework to generate portrait reliefs of various forms. Our method benefits from Shape-from-Shading (SFS). Specifically, we use bi-Laplacian mesh deformation to guide the relief modeling. Given a portrait image, we first use a template face to fit the portrait. We then apply bi-Laplacian mesh deformation to align the facial features. Afterwards, SFS-based reconstruction with a few user interactions is used to optimize the face depth, and create a relief with similar appearance to the input. Both depth structures and geometric details can be well constructed in the final relief. Experiments and comparisons to other methods demonstrate the effectiveness of the proposed method.

[1]  M. Alexa,et al.  Reliefs as images , 2010, ACM Trans. Graph..

[2]  Olga Sorkine-Hornung,et al.  On Linear Variational Surface Deformation Methods , 2008, IEEE Transactions on Visualization and Computer Graphics.

[3]  Adam Finkelstein,et al.  Digital bas-relief from 3D scenes , 2007, ACM Trans. Graph..

[4]  Ralph R. Martin,et al.  Use of non-photorealistic rendering and photometric stereo in making bas-reliefs from photographs , 2014, Graph. Model..

[5]  Weiyin Ma,et al.  Bas-Relief Modeling from Normal Images with Intuitive Styles , 2014, IEEE Transactions on Visualization and Computer Graphics.

[6]  Yiying Tong,et al.  FaceWarehouse: A 3D Facial Expression Database for Visual Computing , 2014, IEEE Transactions on Visualization and Computer Graphics.

[7]  Bailin Deng,et al.  3D Face Reconstruction With Geometry Details From a Single Image , 2017, IEEE Transactions on Image Processing.

[8]  Olga Sorkine-Hornung,et al.  Appearance-mimicking surfaces , 2014, ACM Trans. Graph..

[9]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Yu-Wei Zhang,et al.  Line-based sunken relief generation from a 3D mesh , 2013, Graph. Model..

[11]  Christian Theobalt,et al.  Reconstructing detailed dynamic face geometry from monocular video , 2013, ACM Trans. Graph..

[12]  Ira Kemelmacher-Shlizerman,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 3d Face Reconstruction from a Single Image Using a Single Reference Face Shape , 2022 .

[13]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[14]  Gang Yu,et al.  Real-time bas-relief generation from a 3D mesh , 2013, Graph. Model..

[15]  Xiangyu Zhu,et al.  High-fidelity Pose and Expression Normalization for face recognition in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Hans-Peter Seidel,et al.  Feature sensitive bas relief generation , 2009, 2009 IEEE International Conference on Shape Modeling and Applications.

[17]  Sabine Süsstrunk,et al.  High Reliefs from 3D Scenes , 2015, Comput. Graph. Forum.

[18]  Ralph R. Martin,et al.  Making bas-reliefs from photographs of human faces , 2013, Comput. Aided Des..

[19]  Shi-Min Hu,et al.  Preserving detailed features in digital bas-relief making , 2011, Comput. Aided Geom. Des..

[20]  Charlie C. L. Wang,et al.  Bas-Relief Modeling from Normal Layers , 2019, IEEE Transactions on Visualization and Computer Graphics.

[21]  Caiming Zhang,et al.  Adaptive Bas‐relief Generation from 3D Object under Illumination , 2016, Comput. Graph. Forum.

[22]  Yu-Wei Zhang,et al.  Bas-Relief Generation and Shape Editing through Gradient-Based Mesh Deformation , 2015, IEEE Transactions on Visualization and Computer Graphics.

[23]  Sami Romdhani,et al.  A 3D Face Model for Pose and Illumination Invariant Face Recognition , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[24]  Ralph R. Martin,et al.  Bas-relief Generation Using Adaptive Histogram Equalisation , 2022 .

[25]  Ralph R. Martin,et al.  Region-based bas-relief generation from a single image , 2014, Graph. Model..

[26]  Jianfei Cai,et al.  CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face Images , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Weiyin Ma,et al.  Normal Image Manipulation for Bas-relief Generation with Hybrid Styles , 2018, ArXiv.

[28]  William A. P. Smith,et al.  A Linear Approach to Face Shape and Texture Recovery using a 3D Morphable Model , 2010, BMVC.

[29]  Christian Theobalt,et al.  Reconstruction of Personalized 3D Face Rigs from Monocular Video , 2016, ACM Trans. Graph..

[30]  Song Wang,et al.  Restoration of Brick and Stone Relief from Single Rubbing Images , 2012, IEEE Transactions on Visualization and Computer Graphics.

[31]  Matan Sela,et al.  3D Face Reconstruction by Learning from Synthetic Data , 2016, 2016 Fourth International Conference on 3D Vision (3DV).