Surface enhancement using real-time photometric stereo and reflectance transformation

Photometric stereo recovers per-pixel estimates of surface orientation from images of a surface under varying lighting conditions. Transforming reflectance based on recovered normal directions is useful for enhancing the appearance of subtle surface detail. We present the first system that achieves real-time photometric stereo and reflectance transformation. A high-speed video camera, computer controlled light sources and fast GPU implementations of the algorithms enable both methods. We also present novel GPU-accelerated normal transformations before relighting that "amplify" shape detail. By applying standard image processing methods to our computed normal image, we can selectively enhance surface detail at different frequencies. Our system allows users in fields such as forensics, archeology and dermatology to investigate objects and surfaces by simply holding them in front of the camera. Real-time analysis of surface roughness for metrology can also be performed from the extracted normal field.

[1]  Luc Van Gool,et al.  Easy and cost-effective cuneiform digitizing , 2005 .

[2]  David J. Kriegman,et al.  Beyond Lambert: reconstructing specular surfaces using color , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[4]  Mike J. Chantler,et al.  Recovery of Fingerprints using Photometric Stereo , 2001 .

[5]  Thomas Malzbender,et al.  Polynomial texture maps , 2001, SIGGRAPH.

[6]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, SIGGRAPH 2005.

[7]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[8]  P. Belhumeur,et al.  Learning Object Representations from LightingVariationsR , 1996 .

[9]  Henry Gouraud,et al.  Computer Display of Curved Surfaces , 1971, Outstanding Dissertations in the Computer Sciences.

[10]  E. North Coleman,et al.  Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry , 1982, Comput. Graph. Image Process..

[11]  R. Woodham.  Gradient and Curvature from Photometric Stereo Including Local Condence Estimation , 1994 .

[12]  T. Malzbender,et al.  IMAGING FOSSILS USING REFLECTANCE TRANSFORMATION AND INTERACTIVE MANIPULATION OF VIRTUAL LIGHT SOURCES , 2002 .

[13]  Edwin E. Catmull,et al.  Computer display of curved surfaces , 1998 .

[14]  Mike J. Chantler,et al.  Resolving handwriting from background printing using photometric stereo , 2003, Pattern Recognit..

[15]  Paolo Cignoni,et al.  A simple normal enhancement technique for interactive non-photorealistic renderings , 2005, Comput. Graph..

[16]  Gabriel Taubin,et al.  Appying Shape from Lighting Variation to Bump Map Capture , 1997, Rendering Techniques.

[17]  John C. Russ,et al.  Image Processing Handbook, Fourth Edition , 2002 .

[18]  J. Revelli,et al.  Book Rvw, The Image Processing Handbook, 4th Edition , by John Russ , 2003 .

[19]  Alan L. Yuille,et al.  Learning Object Representation form Lighting Variations , 1996, Object Representation in Computer Vision.

[20]  Joseph Revelli,et al.  The Image Processing Handbook, 4th Edition , 2003, J. Electronic Imaging.

[21]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[22]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, ACM Trans. Graph..

[23]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[24]  Gabriel Taubin,et al.  Building a Digital Model of Michelangelo's Florentine Pietà , 2002, IEEE Computer Graphics and Applications.

[25]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[26]  Bui Tuong Phong.  Illumination for computer generated pictures , 1975, Commun. ACM.

[27]  Guillermo Sapiro,et al.  Three-dimensional shape rendering from multiple images , 2005, Graph. Model..

[28]  George Loizou,et al.  Computer vision and pattern recognition , 2007, Int. J. Comput. Math..

[29]  James F. Blinn,et al.  Computer display of curved surfaces. , 1978 .

[30]  Athinodoros S. Georghiades,et al.  Recovering 3-D Shape and Reflectance From a Small Number of Photographs , 2003, Rendering Techniques.

[31]  Szymon Rusinkiewicz,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, ACM Trans. Graph..

[32]  Bui Tuong Phong.  Illuminat~on for computer generated images , 1973 .

[33]  Rui J. P. de Figueiredo,et al.  A Theory of Photometric Stereo for a Class of Diffuse Non-Lambertian Surfaces , 1991, IEEE Trans. Pattern Anal. Mach. Intell..