Stylized Depiction of Images with Normals

This paper investigates the creation of non-photorealistic illustrations from a type of data lying between simple 2D images and full 3D models: images with both a color (albedo) and a surface normal stored at each pixel. Images with normals combine an acquisition process only mildly more complex than that for digital photographs (and significantly easier than 3D scanning) with the power and flexibility of tools similar to those originally developed for full 3D models. We investigate methods for signal processing on images with normals, developing algorithms for scale-space analysis, derivative (i.e., curvature) estimation, and segmentation. These are used to implement analogues of stylized rendering techniques such as toon shading, line drawing, curvature shading, and exaggerated shading. We show that our rendering pipeline can produce detailed yet understandable illustrations in medical, technical, and archaeological domains.

[1]  Thomas Malzbender,et al.  New Reflection Transformation Imaging Methods for Rock Art and Multiple-Viewpoint Display , 2006, VAST.

[2]  S. Rusinkiewicz,et al.  Exaggerated shading for depicting shape and detail , 2006, ACM Trans. Graph..

[3]  Oliver Deussen,et al.  Image enhancement by unsharp masking the depth buffer , 2006, ACM Trans. Graph..

[4]  Thomas Malzbender,et al.  Surface enhancement using real-time photometric stereo and reflectance transformation , 2006, EGSR '06.

[5]  Amitabh Varshney,et al.  Geometry-dependent lighting , 2006, IEEE Transactions on Visualization and Computer Graphics.

[6]  Mark Mudge,et al.  Reflection Transformation Imaging and Virtual Representations of Coins from the Hospice of the Grand St. Bernard , 2005, VAST.

[7]  Seungyong Lee,et al.  Detail control in line drawings of 3D meshes , 2005, The Visual Computer.

[8]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, ACM Trans. Graph..

[9]  Guillermo Sapiro,et al.  Three-dimensional shape rendering from multiple images , 2005, Graph. Model..

[10]  Budirijanto Purnomo,et al.  iClay: Digitizing Cuneiform , 2004, VAST.

[11]  Ramesh Raskar,et al.  Shape-Enhanced Surgical Visualizations and Medical Illustrations with Multi-flash Imaging , 2004, MICCAI.

[12]  Harry Shum,et al.  Video tooning , 2004, ACM Trans. Graph..

[13]  Hans-Peter Seidel,et al.  Ridge-valley lines on meshes via implicit surface fitting , 2004, ACM Trans. Graph..

[14]  Ramesh Raskar,et al.  Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging , 2004, ACM Trans. Graph..

[15]  Adam Finkelstein,et al.  Interactive rendering of suggestive contours with temporal coherence , 2004, NPAR '04.

[16]  Ross T. Whitaker,et al.  Curvature-based transfer functions for direct volume rendering: methods and applications , 2003, IEEE Visualization, 2003. VIS 2003..

[17]  Athinodoros S. Georghiades,et al.  Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[18]  Adam Finkelstein,et al.  Suggestive contours for conveying shape , 2003, ACM Trans. Graph..

[19]  Philip Dutré,et al.  The free-form light stage , 2002, SIGGRAPH '02.

[20]  R. Basri,et al.  International Journal of Computer Vision c ○ 2006 Springer Science + Business Media, LLC. Manufactured in The Netherlands. DOI: 10.1007/s11263-006-8815-7 Photometric Stereo with General, Unknown Lighting , 2005 .

[21]  Thomas Malzbender,et al.  Polynomial texture maps , 2001, SIGGRAPH.

[22]  Peter-Pike J. Sloan,et al.  The Lit Sphere: A Model for Capturing NPR Shading from Art , 2001, Graphics Interface.

[23]  P. Rheingans,et al.  Volume illustration: non-photorealistic rendering of volume models , 2000, Proceedings Visualization 2000. VIS 2000 (Cat. No.00CH37145).

[24]  Aaron Hertzmann,et al.  Illustrating smooth surfaces , 2000, SIGGRAPH.

[25]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[26]  Holly E. Rushmeier,et al.  Computing consistent normals and colors from photometric data , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[27]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[28]  Sergey Zhukov,et al.  An Ambient Light Illumination Model , 1998, Rendering Techniques.

[29]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[30]  Lee Markosian,et al.  Real-time nonphotorealistic rendering , 1997, SIGGRAPH.

[31]  Alexis Gourdon,et al.  The 3D Marching Lines Algorithm , 1996, CVGIP Graph. Model. Image Process..

[32]  Victoria Interrante,et al.  Enhancing transparent skin surfaces with ridge and valley lines , 1995, Proceedings Visualization '95.

[33]  Gavin S. P. Miller,et al.  Efficient algorithms for local and global accessibility shading , 1994, SIGGRAPH.

[34]  Rui J. P. de Figueiredo,et al.  A Theory of Photometric Stereo for a Class of Diffuse Non-Lambertian Surfaces , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Gershon Elber,et al.  Hidden curve removal for free form surfaces , 1990, SIGGRAPH.

[36]  Takafumi Saito,et al.  Comprehensible rendering of 3-D shapes , 1990, SIGGRAPH.

[37]  Michael F. Cohen,et al.  Automatic illustration of 3D geometric models: lines , 1990, I3D '90.

[38]  Neil Hunt,et al.  The triangle processor and normal vector shader: a VLSI system for high performance graphics , 1988, SIGGRAPH.

[39]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[40]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[41]  M. Gross,et al.  Analysis of human faces using a measurement-based skin reflectance model , 2006, ACM Trans. Graph..

[42]  Gabriel Taubin,et al.  Building a Digital Model of Michelangelo's Florentine Pietà , 2002, IEEE Computer Graphics and Applications.

[43]  Berthold K. P. Horn,et al.  Hill shading and the reflectance map , 1981, Proceedings of the IEEE.