Photometric Stereo with General, Unknown Lighting

Work on photometric stereo has shown how to recover the shape and reflectance properties of an object using multiple images taken with a fixed viewpoint and variable lighting conditions. This work has primarily relied on known lighting conditions or the presence of a single point source of light in each image. In this paper we show how to perform photometric stereo assuming that all lights in a scene are distant from the object but otherwise unconstrained. Lighting in each image may be an unknown and may include arbitrary combination of diffuse, point and extended sources. Our work is based on recent results showing that for Lambertian objects, general lighting conditions can be represented using low order spherical harmonics. Using this representation we can recover shape by performing a simple optimization in a low-dimensional space. We also analyze the shape ambiguities that arise in such a representation. We demonstrate our method by reconstructing the shape of objects from images obtained under a variety of lightings. We further compare the reconstructed shapes against shapes obtained with a laser scanner.

[1]  Rama Chellappa,et al.  Characterization of Human Faces under Illumination Variations Using Rank, Integrability, and Symmetry Constraints , 2004, ECCV.

[2]  Amnon Shashua,et al.  On Photometric Issues in 3D Visual Recognition from a Single 2D Image , 2004, International Journal of Computer Vision.

[3]  E. North Coleman,et al.  Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry , 1982, Comput. Graph. Image Process..

[4]  Nahum Kiryati,et al.  Depth from gradient fields and control points: bias correction in photometric stereo , 2004, Image Vis. Comput..

[5]  Kenichi Kanatani,et al.  Geometric computation for machine vision , 1993 .

[6]  R. Woodham,et al.  Photometric Stereo: Lambertian Reflectance and Light Sources with Unknown Direction and Strength , 1991 .

[7]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[8]  James J. Clark.  Active photometric stereo , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Athinodoros S. Georghiades,et al.  Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  Ismet Erkmen,et al.  An improved method of photometric stereo using local shape from shading , 2003, Image Vis. Comput..

[11]  Ping-Sing Tsai,et al.  Photomotion , 1996, Comput. Vis. Image Underst..

[12]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[14]  Pascal Fua,et al.  The Radiometry of Multiple Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Takayuki Okatani,et al.  On Uniqueness of Solutions of the Three-Light-Source Photometric Stereo: Conditions on Illumination Configuration and Surface Reflectance , 2001, Comput. Vis. Image Underst..

[16]  Maria Petrou,et al.  The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Ronen Basri,et al.  Photometric stereo with general, unknown lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[19]  David W. Jacobs,et al.  Linear Fitting with Missing Data for Structure-from-Motion , 2001, Comput. Vis. Image Underst..

[20]  Daniel Snow,et al.  Determining Generative Models of Objects Under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability , 1999, International Journal of Computer Vision.

[21]  P. Hanrahan,et al.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[22]  Takeo Kanade,et al.  Determining shape and reflectance of hybrid surfaces by photometric sampling , 1989, IEEE Trans. Robotics Autom..

[23]  Alfred M. Bruckstein,et al.  Integrability disambiguates surface recovery in two-image photometric stereo , 1992, International Journal of Computer Vision.

[24]  Rui J. P. de Figueiredo,et al.  A Theory of Photometric Stereo for a Class of Diffuse Non-Lambertian Surfaces , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Katsushi Ikeuchi,et al.  Determining Surface Orientations of Specular Surfaces by Using the Photometric Stereo Method , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Ravi Ramamoorthi ravir.  Analytic PCA construction for theoretical analysis of lighting variability , including attached shadows , in a single image of a convex Lambertian object , 2001 .

[27]  Li Zhang,et al.  Shape and motion under varying illumination: unifying structure from motion, photometric stereo, and multiview stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[28]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[29]  Ravi Ramamoorthi,et al.  Analytic PCA Construction for Theoretical Analysis of Lighting Variability in Images of a Lambertian Object , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Athinodoros S. Georghiades,et al.  Recovering 3-D Shape and Reflectance From a Small Number of Photographs , 2003, Rendering Techniques.

[31]  Lawrence B. Wolff,et al.  Surface Curvature and Shape Reconstruction from Unknown Multiple Illumination and Integrability , 1997, Comput. Vis. Image Underst..

[32]  Melvyn L. Smith,et al.  The analysis of surface texture using photometric stereo acquisition and gradient space domain mapping , 1999, Image Vis. Comput..

[33]  Steven M. Seitz,et al.  Example-based photometric stereo: shape reconstruction with general, varying BRDFs , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[35]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.

[36]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[38]  Ronen Basri,et al.  Dense shape reconstruction of a moving object under arbitrary, unknown lighting , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[39]  Ronen Basri,et al.  Accuracy of Spherical Harmonic Approximations for Images of Lambertian Objects under Far and Near Lighting , 2004, ECCV.

[40]  Ronen Basri,et al.  Comparing images under variable illumination , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[41]  Harry Shum,et al.  Principal Component Analysis with Missing Data and Its Application to Polyhedral Object Modeling , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Hideki Hayakawa.  Photometric stereo under a light source with arbitrary motion , 1994 .

[43]  Byungil Kim,et al.  Depth and shape from shading using the photometric stereo method , 1991, CVGIP Image Underst..

[44]  Andrea J. van Doorn,et al.  The Generic Bilinear Calibration-Estimation Problem , 2004, International Journal of Computer Vision.