Stylized depiction of images based on depth perception

Recent works in image editing are opening up new possibilities to manipulate and enhance input images. Within this context, we leverage well-known characteristics of human perception along with a simple depth approximation algorithm to creatively relight images for the purpose of generating non-photorealistic renditions that would be difficult to achieve with existing methods. Our realtime implementation on graphics hardware allows the user to efficiently explore artistic possibilities for each image. We show results produced with four different styles proving the versatility of our approach, and validate our assumptions and simplifications by means of a user study.

[1]  Kevin G. Suffern,et al.  Painting with light , 2002, SIGGRAPH '02.

[2]  Ariel Shamir,et al.  Relief analysis and extraction , 2009, ACM Trans. Graph..

[3]  Szymon Rusinkiewicz,et al.  Exaggerated shading for depicting shape and detail , 2006, SIGGRAPH 2006.

[4]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[5]  Steven W. Zucker,et al.  Casting Light on Illumination: A Computational Model and Dimensional Analysis of Sources , 1997, Comput. Vis. Image Underst..

[6]  J. Alton.  Painting with Light , 1949 .

[7]  Ariel Shamir,et al.  Relief analysis and extraction , 2009, SIGGRAPH 2009.

[8]  Kevin Grant,et al.  Stylized black and white images from photographs , 2008, NPAR.

[9]  Stefan Schlechtweg,et al.  Non-photorealistic computer graphics: modeling, rendering, and animation , 2002 .

[10]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[11]  Szymon Rusinkiewicz,et al.  Exaggerated shading for depicting shape and detail , 2006, ACM Trans. Graph..

[12]  Ramesh Raskar,et al.  Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging , 2004, ACM Trans. Graph..

[13]  Yasuyuki Matsushita,et al.  Single-View Relighting with Normal Map Painting , 2006 .

[14]  Maneesh Agrawala,et al.  The assumed light direction for perceiving shape from shading , 2008, APGV '08.

[15]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[16]  Nancy S. Pollard,et al.  Real-time gradient-domain painting , 2008, ACM Trans. Graph..

[17]  Erik Reinhard,et al.  Human facial illustrations: Creation and psychophysical evaluation , 2004, TOGS.

[18]  Scott F. Johnston.  Lumo: illumination for cel animation , 2002, NPAR '02.

[19]  Bruce Gooch,et al.  Non-photorealistic rendering , 2001 .

[20]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[21]  H. Bülthoff,et al.  Depth Discrimination from Shading under Diffuse Lighting , 2000, Perception.

[22]  Victor Ostromoukhov.  Digital facial engraving , 1999, SIGGRAPH '99.

[23]  Patrick Cavanagh,et al.  Perceiving Illumination Inconsistencies in Scenes , 2005, Perception.

[24]  Erik Reinhard,et al.  Depicting procedural caustics in single images , 2008, SIGGRAPH Asia '08.

[25]  Chung-Ren Yan,et al.  Stylized Rendering Using Samples of a Painted Image , 2008, IEEE Transactions on Visualization and Computer Graphics.

[26]  Frédo Durand,et al.  Two-scale tone management for photographic look , 2006, ACM Trans. Graph..

[27]  Steven W. Zucker,et al.  Computing Contour Closure , 1996, ECCV.

[28]  William V. Baxter,et al.  Locally controllable stylized shading , 2007, ACM Trans. Graph..

[29]  Refractor.  Vision , 2000, The Lancet.

[30]  Gunter Loffler,et al.  Perception of contours and shapes: Low and intermediate stage mechanisms , 2008, Vision Research.

[31]  J T Todd,et al.  Ambiguity and the ‘Mental Eye’ in Pictorial Relief , 2001, Perception.

[32]  Pat Hanrahan,et al.  Conveying shape and features with image-based relighting , 2003, IEEE Visualization, 2003. VIS 2003..

[33]  Michael F. Cohen,et al.  Stylizing 2.5-D video , 2005, SIGGRAPH '05.

[34]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[35]  Frédo Durand,et al.  An invitation to discuss computer depiction , 2002, NPAR '02.

[36]  Frédo Durand,et al.  A gentle introduction to bilateral filtering and its applications , 2007, SIGGRAPH Courses.

[37]  Sing Bing Kang,et al.  Depth Painting for Image-based Rendering Applications , 1999 .

[38]  Harry Shum,et al.  Lazy snapping , 2004, ACM Trans. Graph..

[39]  Erik Reinhard,et al.  Image-based material editing , 2005, SIGGRAPH '05.

[40]  Ramesh Raskar,et al.  Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging , 2004, SIGGRAPH 2004.

[41]  Jean-Denis Durou,et al.  Numerical methods for shape-from-shading: A new survey with benchmarks , 2008, Comput. Vis. Image Underst..

[42]  Giovanni Civardi.  Drawing light & shade : understanding chiaroscuro , 2006 .