Illustration of complex real-world objects using images with normals

This paper investigates the creation of non-photorealistic illustrations from a type of data lying between simple 2D images and full 3D models: images with both a color (albedo) and a surface normal stored at each pixel. Images with normals combine an acquisition process only mildly more complex than that for digital photographs (and significantly easier than 3D scanning) with the power and flexibility of tools similar to those originally developed for full 3D models. We investigate methods for signal processing on images with normals, developing algorithms for scale-space analysis, derivative (i.e., curvature) estimation, and segmentation. These are used to implement analogues of stylized rendering techniques such as toon shading, line drawing, curvature shading, and exaggerated shading. We also introduce new stylization effects based on multiscale mean curvature shading, as well as fast discontinuity shadows. We show that our rendering pipeline can produce detailed yet understandable illustrations in medical, technical, and archaeological domains.

[1]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[2]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[3]  Berthold K. P. Horn,et al.  Hill shading and the reflectance map , 1981, Proceedings of the IEEE.

[4]  Takafumi Saito,et al.  Comprehensible rendering of 3-D shapes , 1990, SIGGRAPH.

[5]  Gershon Elber,et al.  Hidden curve removal for free form surfaces , 1990, SIGGRAPH.

[6]  Michael F. Cohen,et al.  Automatic illustration of 3D geometric models: lines , 1990, I3D '90.

[7]  H. Reeves,et al.  The Guild handbook of scientific illustration , 1991 .

[8]  David Salesin,et al.  Computer-generated pen-and-ink illustration , 1994, SIGGRAPH.

[9]  Gavin S. P. Miller,et al.  Efficient algorithms for local and global accessibility shading , 1994, SIGGRAPH.

[10]  Victoria Interrante,et al.  Enhancing transparent skin surfaces with ridge and valley lines , 1995, Proceedings Visualization '95.

[11]  Alexis Gourdon,et al.  The 3D Marching Lines Algorithm , 1996, CVGIP Graph. Model. Image Process..

[12]  Lee Markosian,et al.  Real-time nonphotorealistic rendering , 1997, SIGGRAPH.

[13]  Sergey Zhukov,et al.  An Ambient Light Illumination Model , 1998, Rendering Techniques.

[14]  Elaine Cohen,et al.  A non-photorealistic lighting model for automatic technical illustration , 1998, SIGGRAPH.

[15]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[16]  Holly E. Rushmeier,et al.  Computing consistent normals and colors from photometric data , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[17]  David S. Ebert,et al.  Volume illustration: non-photorealistic rendering of volume models , 2000, Proceedings Visualization 2000. VIS 2000 (Cat. No.00CH37145).

[18]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[19]  Aaron Hertzmann,et al.  Illustrating smooth surfaces , 2000, SIGGRAPH.

[20]  Thomas Malzbender,et al.  Polynomial texture maps , 2001, SIGGRAPH.

[21]  Peter-Pike J. Sloan,et al.  The Lit Sphere: A Model for Capturing NPR Shading from Art , 2001, Graphics Interface.

[22]  David S. Ebert,et al.  Volume Illustration: Nonphotorealistic Rendering of Volume Models , 2001, IEEE Trans. Vis. Comput. Graph..

[23]  Philip Dutré,et al.  The free-form light stage , 2002, SIGGRAPH '02.

[24]  Gabriel Taubin,et al.  Building a Digital Model of Michelangelo's Florentine Pietà , 2002, IEEE Computer Graphics and Applications.

[25]  Adam Finkelstein,et al.  Suggestive contours for conveying shape , 2003, ACM Trans. Graph..

[26]  Ross T. Whitaker,et al.  Curvature-based transfer functions for direct volume rendering: methods and applications , 2003, IEEE Visualization, 2003. VIS 2003..

[27]  Michael Cohen,et al.  Video tooning , 2004, SIGGRAPH 2004.

[28]  H. Seidel,et al.  Ridge-valley lines on meshes via implicit surface fitting , 2004, SIGGRAPH 2004.

[29]  Hans-Peter Seidel,et al.  Ridge-valley lines on meshes via implicit surface fitting , 2004, ACM Trans. Graph..

[30]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[31]  Ramesh Raskar,et al.  Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging , 2004, SIGGRAPH 2004.

[32]  Budirijanto Purnomo,et al.  iClay: Digitizing Cuneiform , 2004, VAST.

[33]  Ramesh Raskar,et al.  Shape-Enhanced Surgical Visualizations and Medical Illustrations with Multi-flash Imaging , 2004, MICCAI.

[34]  Adam Finkelstein,et al.  Interactive rendering of suggestive contours with temporal coherence , 2004, NPAR '04.

[35]  Ramesh Raskar,et al.  Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging , 2004, ACM Trans. Graph..

[36]  Szymon Rusinkiewicz,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, ACM Trans. Graph..

[37]  Guillermo Sapiro,et al.  Three-dimensional shape rendering from multiple images , 2005, Graph. Model..

[38]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, SIGGRAPH 2005.

[39]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, ACM Trans. Graph..

[40]  Mark Mudge,et al.  Reflection Transformation Imaging and Virtual Representations of Coins from the Hospice of the Grand St. Bernard , 2005, VAST.

[41]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[42]  Seungyong Lee,et al.  Detail control in line drawings of 3D meshes , 2005, The Visual Computer.

[43]  Oliver Deussen,et al.  Image enhancement by unsharp masking the depth buffer , 2006, ACM Trans. Graph..

[44]  Szymon Rusinkiewicz,et al.  Exaggerated shading for depicting shape and detail , 2006, ACM Trans. Graph..

[45]  Amitabh Varshney,et al.  Geometry-dependent lighting , 2006, IEEE Transactions on Visualization and Computer Graphics.

[46]  Szymon Rusinkiewicz,et al.  Exaggerated shading for depicting shape and detail , 2006, SIGGRAPH 2006.

[47]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[48]  Thomas Malzbender,et al.  Surface enhancement using real-time photometric stereo and reflectance transformation , 2006, EGSR '06.

[49]  Oliver Deussen,et al.  Image enhancement by unsharp masking the depth buffer , 2006, SIGGRAPH 2006.

[50]  Thomas Malzbender,et al.  New Reflection Transformation Imaging Methods for Rock Art and Multiple-Viewpoint Display , 2006, VAST.