High Relief from Brush Painting

Relief is an art form part way between 3D sculpture and 2D painting. We present a novel approach for generating a texture-mapped high-relief model from a single brush painting. Our aim is to extract the brushstrokes from a painting and generate the individual corresponding relief proxies rather than recovering the exact depth map from the painting, which is a tricky computer vision problem, requiring assumptions that are rarely satisfied. The relief proxies of brushstrokes are then combined together to form a 2.5D high-relief model. To extract brushstrokes from 2D paintings, we apply layer decomposition and stroke segmentation by imposing boundary constraints. The segmented brushstrokes preserve the style of the input painting. By inflation and a displacement map of each brushstroke, the features of brushstrokes are preserved by the resultant high-relief model of the painting. We demonstrate that our approach is able to produce convincing high-reliefs from a variety of paintings(with humans, animals, flowers, etc.). As a secondary application, we show how our brushstroke extraction algorithm could be used for image editing. As a result, our brushstroke extraction algorithm is specifically geared towards paintings with each brushstroke drawn very purposefully, such as Chinese paintings, Rosemailing paintings, etc.

[1]  Nancy S. Pollard,et al.  Soft Stacking , 2012, Comput. Graph. Forum.

[2]  Adrien Bousseau,et al.  Vectorising Bitmaps into Semi‐Transparent Gradient Layers , 2014, Comput. Graph. Forum.

[3]  Yotam I. Gingold,et al.  Decomposing images into layers via RGB-space geometry , 2017, TOGS.

[4]  A. Loon.  Color changes and chemical reactivity in seventeenth-century oil paintings , 2008 .

[5]  Ralph R. Martin,et al.  Region-based bas-relief generation from a single image , 2014, Graph. Model..

[6]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[7]  Alex M. Andrew,et al.  Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science (2nd edition) , 2000 .

[8]  Frédo Durand,et al.  A gentle introduction to bilateral filtering and its applications , 2007, SIGGRAPH Courses.

[9]  Vincent van Gogh,et al.  The Works of Vincent van Gogh: His Paintings and Drawings , 1970 .

[10]  Thomas F. Stahovich,et al.  A machine learning approach to automatic stroke segmentation , 2014, Comput. Graph..

[11]  Zongben Xu,et al.  Image Inpainting by Patch Propagation Using Patch Sparsity , 2010, IEEE Transactions on Image Processing.

[12]  Alexei A. Efros,et al.  Recovering Surface Layout from an Image , 2007, International Journal of Computer Vision.

[13]  Digital bas-relief from 3D scenes , 2007, SIGGRAPH 2007.

[14]  Ilan Shimshoni,et al.  Reconstruction of relief objects from line drawings , 2011, CVPR 2011.

[15]  Tom Duff,et al.  Compositing digital images , 1984, SIGGRAPH.

[16]  P. Lions,et al.  Shape-from-shading, viscosity solutions and edges , 1993 .

[17]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Ralph R. Martin,et al.  Estimating depth from line drawing , 2002, SMA '02.

[19]  Olga Sorkine-Hornung,et al.  Ink-and-ray: Bas-relief meshes for adding global illumination effects to hand-drawn characters , 2014, TOGS.

[20]  Chi-Wing Fu,et al.  Interactive High-Relief Reconstruction for Organic and Double-Sided Objects from a Photo , 2017, IEEE Transactions on Visualization and Computer Graphics.

[21]  Stephen Gould,et al.  Single image depth estimation from predicted semantic labels , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Zachary Salim Benzaid.  Analysis of Bas-Relief Generation Techniques , 2017 .

[23]  Seungyong Lee,et al.  Coherent line drawing , 2007, NPAR '07.

[24]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[25]  Jitendra Malik,et al.  Interpreting line drawings of curved objects , 1986, International Journal of Computer Vision.

[26]  Ross B. Girshick.  Simulating Chinese brush painting: the parametric hairy brush , 2004, SIGGRAPH '04.

[27]  M. Alexa,et al.  Reliefs as images , 2010, ACM Trans. Graph..

[28]  Alexei A. Efros,et al.  Geometric context from a single image , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[29]  Dimosthenis Karatzas,et al.  A fast hierarchical method for multi-script and arbitrary oriented scene text extraction , 2014, International Journal on Document Analysis and Recognition (IJDAR).

[30]  David Salesin,et al.  Animating Chinese paintings through stroke-based decomposition , 2006, TOGS.

[31]  Marko Subasic,et al.  Level Set Methods and Fast Marching Methods , 2003 .

[32]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[33]  David Nistér,et al.  Linear Time Maximally Stable Extremal Regions , 2008, ECCV.

[34]  Steven M. Seitz,et al.  Single-view modelling of free-form scenes , 2002, Comput. Animat. Virtual Worlds.

[35]  Hans-Peter Seidel,et al.  Computer Assisted Relief Generation—A Survey , 2012, Comput. Graph. Forum.

[36]  Olivier D. Faugeras,et al.  Unifying Approaches and Removing Unrealistic Assumptions in Shape from Shading: Mathematics Can Help , 2004, ECCV.

[37]  Lei Yao,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Rhythmic Brushstrokes Distinguish Van Gogh from His Contemporaries: Findings via Automated Brushstroke Extraction , 2022 .

[38]  Horst Bischof,et al.  Efficient Maximally Stable Extremal Region (MSER) Tracking , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[39]  Hans-Peter Seidel,et al.  Feature sensitive bas relief generation , 2009, 2009 IEEE International Conference on Shape Modeling and Applications.

[40]  M. Newall.  The Art School , 2018, A Philosophy of the Art School.

[41]  Olivier D. Faugeras,et al.  Shape From Shading , 2006, Handbook of Mathematical Models in Computer Vision.

[42]  Ralph R. Martin,et al.  Making bas-reliefs from photographs of human faces , 2013, Comput. Aided Des..

[43]  Nancy S. Pollard,et al.  Local layering , 2009, SIGGRAPH 2009.

[44]  Olivier D. Faugeras,et al.  "Perspective shape from shading" and viscosity solutions , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.