Surface flows for image-based shading design

We present a novel method for producing convincing pictures of shaded objects based entirely on 2D image operations. This approach, which we call image-based shading design, offers direct artistic control in the picture plane by deforming image primitives so that they appear to conform to specific 3D shapes. Using a differential analysis of reflected radiance, we identify the two types of surface flows involved in the depiction of shaded objects, which are consistent with recent perceptual studies. We then introduce two novel deformation operators that closely mimic surface flows while providing direct artistic controls in real-time.

[1]  R. Hetherington.  The Perception of the Visual World , 1952 .

[2]  Phillip J. Marlow,et al.  The perception of gloss depends on highlight congruence with surface shading. , 2011, Journal of vision.

[3]  Scott F. Johnston.  Lumo: illumination for cel animation , 2002, NPAR '02.

[4]  H. Bülthoff,et al.  Estimation of 3D shape from image orientations , 2011, Proceedings of the National Academy of Sciences.

[5]  Pierre Poulin,et al.  Sketch and Paint-based Interface for Highlight Modeling , 2008, SBIM.

[6]  Adrien Bousseau,et al.  CrossShade: shading concept sketches using cross-section curves , 2012, ACM Trans. Graph..

[7]  William V. Baxter,et al.  Tweakable light and shade for cartoon animation , 2006, NPAR.

[8]  Harry Shum,et al.  ShapePalettes: interactive normal transfer via sketching , 2007, ACM Trans. Graph..

[9]  Andrea Li,et al.  Perception of three-dimensional shape from texture is based on patterns of oriented energy , 2000, Vision Research.

[10]  Steven W. Zucker,et al.  Shadows and shading flow fields , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  John Hart,et al.  Textureshop: texture synthesis as a photograph editing tool , 2004, SIGGRAPH 2004.

[12]  Szymon Rusinkiewicz,et al.  Illustration of complex real-world objects using images with normals , 2007, NPAR '07.

[13]  Pascal Barla,et al.  Improving Shape Depiction under Arbitrary Rendering , 2011, IEEE Transactions on Visualization and Computer Graphics.

[14]  A. Torralba,et al.  Specular reflections and the perception of shape. , 2004, Journal of vision.

[15]  Bruce Walter,et al.  Visual equivalence: towards a new standard for image fidelity , 2007, SIGGRAPH 2007.

[16]  Adam Finkelstein,et al.  Lighting with paint , 2007, TOGS.

[17]  F. E. Nicodemus,et al.  Geometrical considerations and nomenclature for reflectance , 1977 .

[18]  Stefan Jeschke,et al.  Estimating Color and Texture Parameters for Vector Graphics , 2011, Comput. Graph. Forum.

[19]  L. Maloney,et al.  Visual Perception of Thick Transparent Materials , 2011, Psychological science.

[20]  Ravi Ramamoorthi,et al.  A first-order analysis of lighting, shading, and shadows , 2007, TOGS.

[21]  J. Koenderink,et al.  Photometric Invariants Related to Solid Shape , 1980 .

[22]  Silvio Savarese,et al.  Detecting Specular Surfaces on Natural Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  J. Gibson.  The perception of visual surfaces. , 1950, The American journal of psychology.

[24]  Erik Reinhard,et al.  Image-based material editing , 2005, SIGGRAPH '05.

[25]  J. Todd,et al.  Perception of three-dimensional form from patterns of optical texture. , 1987, Journal of experimental psychology. Human perception and performance.

[26]  Fabio Pellacini.  envyLight : an interface for editing natural illumination , 2010, SIGGRAPH 2010.

[27]  Michael S. Brown,et al.  Matting and compositing of transparent and refractive objects , 2011, TOGS.

[28]  D. Shepard.  A two-dimensional interpolation function for irregularly-spaced data , 1968, ACM National Conference.

[29]  Alexandrina Orzan,et al.  Texture Design and Draping in 2D Images , 2016 .

[30]  Ohad Ben-Shahar,et al.  Shape from specular flow: Is one flow enough? , 2011, CVPR 2011.

[31]  Nancy S. Pollard,et al.  Real-time gradient-domain painting , 2008, SIGGRAPH 2008.

[32]  Marshall F. Tappen.  Recovering shape from a single image of a mirrored surface from curvature constraints , 2011, CVPR 2011.

[33]  Yasuyuki Matsushita,et al.  Illumination Brush: Interactive Design of All-Frequency Lighting , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[34]  Hans-Peter Seidel,et al.  Interactive reflection editing , 2009, ACM Trans. Graph..

[35]  Sumanta N. Pattanaik,et al.  BRDF-Shop: creating physically correct bidirectional reflectance distribution functions , 2006, IEEE Computer Graphics and Applications.

[36]  Edward H. Adelson,et al.  Luminance re-mapping for the control of apparent material , 2005, APGV '05.

[37]  BarlaPascal,et al.  Surface flows for image-based shading design , 2012 .

[38]  Barton L Anderson,et al.  Image statistics do not explain the perception of gloss and lightness. , 2009, Journal of vision.

[39]  Greg Humphreys,et al.  Physically Based Rendering, Second Edition: From Theory To Implementation , 2010 .

[40]  Frédo Durand,et al.  A gentle introduction to bilateral filtering and its applications , 2007, SIGGRAPH Courses.

[41]  Pascal Barla,et al.  Diffusion curves: a vector representation for smooth-shaded images , 2008, ACM Trans. Graph..

[42]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[43]  J. Bigun,et al.  Optimal Orientation Detection of Linear Symmetry , 1987, ICCV 1987.

[44]  Hans-Peter Seidel,et al.  Interactive on-surface signal deformation , 2010, SIGGRAPH 2010.

[45]  Steven W. Zucker,et al.  Folds and cuts: how shading flows into edges , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[46]  Harry Shum,et al.  Image vectorization using optimized gradient meshes , 2007, ACM Trans. Graph..

[47]  Hans-Peter Seidel,et al.  Interactive on-surface signal deformation , 2010 .

[48]  Christophe Schlick,et al.  A Fast Alternative to Phong's Specular Model , 1994, Graphics Gems.

[49]  I. Motoyoshi.  Highlight-shading relationship as a cue for the perception of translucent and transparent materials. , 2010, Journal of vision.

[50]  Bruce Walter,et al.  Visual equivalence: towards a new standard for image fidelity , 2007, ACM Trans. Graph..

[51]  E. Mingolla,et al.  Perception of surface curvature and direction of illumination from patterns of shading. , 1983, Journal of experimental psychology. Human perception and performance.

[52]  Fabio Pellacini,et al.  envyLight: an interface for editing natural illumination , 2010, ACM Trans. Graph..

[53]  Barton L Anderson,et al.  Image statistics and the perception of surface gloss and lightness. , 2010, Journal of vision.

[54]  Hans-Peter Seidel,et al.  Interactive reflection editing , 2009, SIGGRAPH 2009.

[55]  Peter-Pike J. Sloan,et al.  The Lit Sphere: A Model for Capturing NPR Shading from Art , 2001, Graphics Interface.

[56]  Edward H. Adelson,et al.  Shape from Sheen , 2009 .