Median Photometric Stereo as Applied to the Segonko Tumulus and Museum Objects

One of the necessary techniques for constructing a virtual museum is to estimate the surface normal and the albedo of the artwork which has high specularity. In this paper, we propose a novel photometric stereo method which is robust to the specular reflection of the object surface. Our method can also digitize the artwork arranged inside a glass or acrylic display case without bringing the artwork out of the display case. Our method treats the specular reflection at the object surface or at the display case as an outlier, and finds a good surface normal evading the influence of the outliers. We judiciously design the cost function so that the outlier will be automatically removed under the assumption that the object’s shape and color are smooth. At the end of this paper, we also show some archived 3D data of Segonko Tumulus and objects in the University Museum at The University of Tokyo that were generated by applying the proposed method.

[1]  Jun Yang,et al.  Determining a polyhedral shape using interreflections , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Tai-Pang Wu,et al.  Dense Photometric Stereo by Expectation Maximization , 2006, ECCV.

[3]  Takeo Kanade,et al.  Determining shape and reflectance of hybrid surfaces by photometric sampling , 1989, IEEE Trans. Robotics Autom..

[4]  Kiriakos N. Kutulakos,et al.  A theory of inverse light transport , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[5]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[6]  Shoji Tominaga,et al.  Image Rendering of Art Paintings -Total Archives Considering Surface Properties and Chromatic Adaptation , 2004, Color Imaging Conference.

[7]  Frédo Durand,et al.  Flash photography enhancement via intrinsic relighting , 2004, SIGGRAPH 2004.

[8]  Harry Shum,et al.  Flash matting , 2006, ACM Trans. Graph..

[9]  Tien-Tsin Wong,et al.  Dense photometric stereo using tensorial belief propagation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Michal Irani,et al.  Separating Transparent Layers through Layer Information Exchange , 2004, ECCV.

[11]  K. Ikeuchi,et al.  Reflectance analysis under solar illumination , 1994, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[12]  Y. Weiss,et al.  Separating reflections from a single image using local features , 2004, CVPR 2004.

[13]  Edwin R. Hancock,et al.  Surface normals and height from non-Lambertian image data , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[14]  David J. Kriegman,et al.  Photometric stereo with non-parametric and spatially-varying reflectance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[16]  R. Woodham,et al.  Photometric Stereo: Lambertian Reflectance and Light Sources with Unknown Direction and Strength , 1991 .

[17]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[18]  Cheng Lu,et al.  Practical Scene Illuminant Estimation via Flash/No-Flash Pairs , 2006, Color Imaging Conference.

[19]  R. Y. Tsai,et al.  An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision , 1986, CVPR 1986.

[20]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[21]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[22]  Ramesh Raskar,et al.  Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging , 2004, SIGGRAPH 2004.

[23]  David J. Kriegman,et al.  ShadowCuts: Photometric Stereo with Shadows , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Katsushi Ikeuchi,et al.  The separation of reflected and transparent layers from real-world image sequence , 2007, Machine Vision and Applications.

[25]  Michal Irani,et al.  Separating transparent layers of repetitive dynamic behaviors , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[26]  Peter H. Tu,et al.  Surface reconstruction via Helmholtz reciprocity with a single image pair , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  E. North Coleman,et al.  Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry , 1982, Comput. Graph. Image Process..

[28]  Rama Chellappa,et al.  What Is the Range of Surface Reconstructions from a Gradient Field? , 2006, ECCV.

[29]  Athinodoros S. Georghiades,et al.  Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[30]  Chu-Song Chen,et al.  The 4-Source Photometric Stereo Under General Unknown Lighting , 2006, ECCV.

[31]  Ronen Basri,et al.  Photometric stereo with general, unknown lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[32]  Takeo Kanade,et al.  The measurement of highlights in color images , 1988, International Journal of Computer Vision.

[33]  Daniel Snow,et al.  Determining Generative Models of Objects Under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability , 1999, International Journal of Computer Vision.

[34]  Steven M. Seitz,et al.  Shape and Spatially-Varying BRDFs from Photometric Stereo , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[36]  Anat Levin,et al.  User Assisted Separation of Reflections from a Single Image Using a Sparsity Prior , 2004, ECCV.

[37]  David J. Kriegman,et al.  Passive photometric stereo from motion , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[38]  Takeshi Shakunaga,et al.  Analysis of photometric factors based on photometric linearization. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[39]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[40]  Katsushi Ikeuchi,et al.  Extracting the Shape and Roughness of Specular Lobe Objects Using Four Light Photometric Stereo , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Daniel Rodríguez,et al.  Adobe Photoshop 7 , 2002 .

[42]  Ronen Basri,et al.  Dense shape reconstruction of a moving object under arbitrary, unknown lighting , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[43]  Edwin R. Hancock,et al.  Surface normals and height from non-Lambertian image data , 2004 .

[44]  Hideki Hayakawa.  Photometric stereo under a light source with arbitrary motion , 1994 .

[45]  Takashi Machida,et al.  Efficient photometric stereo technique for three-dimensional surfaces with unknown BRDF , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[46]  Tai-Pang Wu,et al.  Dense photometric stereo using a mirror sphere and graph cut , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[47]  Budirijanto Purnomo,et al.  Digital Hammurabi: design and development of a 3D scanner for cuneiform tablets , 2006, Electronic Imaging.

[48]  J Shamir,et al.  Polarization and statistical analysis of scenes containing a semireflector. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[49]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[50]  David J. Kriegman,et al.  Toward a stratification of Helmholtz stereopsis , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[51]  Byungil Kim,et al.  Depth and shape from shading using the photometric stereo method , 1991, CVGIP Image Underst..

[52]  Harry Shum,et al.  Lazy snapping , 2004, ACM Trans. Graph..

[53]  Atsushi Nakazawa,et al.  Fast simultaneous alignment of multiple range images using index images , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[54]  Takahiro Okabe,et al.  Shape Reconstruction Based on Similarity in Radiance Changes under Varying Illumination , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[55]  Maria Petrou,et al.  The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Ronen Basri,et al.  Separation of Transparent Layers using Focus , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[57]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, ACM Trans. Graph..

[58]  H Farid,et al.  Separating reflections from images by use of independent component analysis. , 1999, Journal of the Optical Society of America. A, Optics, image science, and vision.

[59]  Stephen Lin,et al.  Resolution-Enhanced Photometric Stereo , 2006, ECCV.

[60]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[61]  Martin Jägersand,et al.  Variational Shape and Reflectance Estimation Under Changing Light and Viewpoints , 2006, ECCV.

[62]  David J. Kriegman,et al.  Beyond Lambert: reconstructing surfaces with arbitrary BRDFs , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[63]  Takashi Matsuyama,et al.  Shape from Shading with Interreflections Under a Proximal Light Source: Distortion-Free Copying of an Unfolded Book , 1997, International Journal of Computer Vision.

[64]  Richard Szeliski,et al.  Layer extraction from multiple images containing reflections and transparency , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[65]  Steven M. Seitz,et al.  Example-based photometric stereo: shape reconstruction with general, varying BRDFs , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Tien-Tsin Wong,et al.  Dense Photometric Stereo: A Markov Random Field Approach , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Jiuai Sun,et al.  Object surface recovery using a multi-light photometric stereo technique for non-Lambertian surfaces subject to shadows and specularities , 2007, Image Vis. Comput..

[68]  Li Zhang,et al.  Shape and motion under varying illumination , 2003, IEEE International Conference on Computer Vision.

[69]  Pascal Fua,et al.  Object-centered surface reconstruction: Combining multi-image stereo and shading , 1995, International Journal of Computer Vision.

[70]  David J. Kriegman,et al.  Isotropy, Reciprocity and the Generalized Bas-Relief Ambiguity , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[71]  R. Courant,et al.  Methods of Mathematical Physics , 1962 .

[72]  Mark S. Drew.  Reduction of rank-reduced orientation-from-color problem with many unknown lights to two-image known-illuminant photometric stereo , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[73]  Shree K. Nayar,et al.  Structured light in scattering media , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[74]  James J. Little,et al.  Reflectance function estimation and shape recovery from image sequence of a rotating object , 1995, Proceedings of IEEE International Conference on Computer Vision.

[75]  Takeo Kanade,et al.  Shape from interreflections , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[76]  Takeo Kanade,et al.  Surface Reflection: Physical and Geometrical Perspectives , 1989, IEEE Trans. Pattern Anal. Mach. Intell..