Shape Estimation of Transparent Objects by Using Polarization Analyses

†† Recently, techniques developed in the field of computer graphics and virtual reality have been applied to many environments, with the result that measuring the 3D shapes of real objects has become increasingly important. However, few methods have been proposed to measure the 3D shape of transparent objects such as glass and acrylics. In this paper, we introduce three methods that estimate the surface shape of transparent objects by using polarization analysis. The first method determines the surface shape of a transparent object by using knowledge established in the research field of thermodynamics. The second method determines the surface shape of a transparent object by using knowledge established in the research field of differential geometry. The third method gives an initial value of the surface shape and then determines the true surface shape of a transparent object by iterative computation. At the end of the paper, we discuss the advantages and the disadvantages of these three methods.

[1]  Katsushi Ikeuchi,et al.  Polarization-based shape estimation of transparent objects by using raytracing and PLZT camera , 2005, SPIE Optics + Photonics.

[2]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[3]  Takashi Matsuyama,et al.  Shape from Shading with Interreflections Under a Proximal Light Source: Distortion-Free Copying of an Unfolded Book , 1997, International Journal of Computer Vision.

[4]  Stefan Rahmann.  Reconstruction of Quadrics from Two Polarization Views , 2003, IbPRIA.

[5]  Takeo Kanade,et al.  Computational model of image formation process in DIC microscopy , 1998, Photonics West - Biomedical Optics.

[6]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[7]  Katsushi Ikeuchi,et al.  Appearance sampling for obtaining a set of basis images for variable illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[8]  Lawrence B. Wolff,et al.  Ray tracing with polarization parameters , 1990, IEEE Computer Graphics and Applications.

[9]  Cyril Soler,et al.  Graphics gems revisited: fast and physically-based rendering of gemstones , 2004, ACM Trans. Graph..

[10]  D. Jordan,et al.  Measurements of the effect of surface roughness on the polarization state of thermally emitted radiation. , 1994, Optics letters.

[11]  John M. Snyder,et al.  Realistic Reflections and Refractions on Graphics Hardware with Hybrid Rendering and Layered Environment Maps , 2001, Rendering Techniques.

[12]  Kiriakos N. Kutulakos,et al.  A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation , 2005, ICCV.

[13]  Stefan Rahmann,et al.  Inferring 3D scene structure from a single polarization image , 1999, Industrial Lasers and Inspection.

[14]  F. E. Nicodemus.  Reflectance nomenclature and directional reflectance and emissivity. , 1970, Applied optics.

[15]  Lee E. Weiss,et al.  Specular surface inspection using structured highlight and Gaussian images , 1990, IEEE Trans. Robotics Autom..

[16]  Yan-Bin Jia,et al.  Tactile recognition of algebraic shapes using differential invariants , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[17]  Berthold K. P. Horn.  Height and gradient from shading , 1989, International Journal of Computer Vision.

[18]  F. E. Nicodemus.  Directional Reflectance and Emissivity of an Opaque Surface , 1965 .

[19]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[20]  David Salesin,et al.  Environment matting extensions: towards higher accuracy and real-time capture , 2000, SIGGRAPH.

[21]  O. Sandus.  A Review of Emission Polarization , 1965 .

[22]  Andreas G. Andreou,et al.  Liquid crystal polarization camera , 1997, IEEE Trans. Robotics Autom..

[23]  Seiji Hata,et al.  Shape extraction of transparent object using genetic algorithm , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[24]  Lee E. Weiss,et al.  Structured Highlight Inspection of Specular Surfaces , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[26]  Paul E. Debevec,et al.  A photometric approach to digitizing cultural artifacts , 2001, VAST '01.

[27]  Shree K. Nayar,et al.  A Theory of Specular Surface Geometry , 2004, International Journal of Computer Vision.

[28]  Pieter Peers,et al.  Wavelet Environment matting , 2003, Rendering Techniques.

[29]  Shree K. Nayar,et al.  Transparent Grippers: Seeing While Grasping , 1996 .

[30]  Takeo Kanade,et al.  Shape from interreflections , 2004, International Journal of Computer Vision.

[31]  Emanuele Trucco,et al.  Using light polarization in laser scanning , 1997, Image Vis. Comput..

[32]  Katsushi Ikeuchi,et al.  Appearance Based Object Modeling using Texture Database: Acquisition Compression and Rendering , 2002, Rendering Techniques.

[33]  John Park.  MULTI-PEAK RANGE IMAGING FOR ACCURATE 3 D RECONSTRUCTION OF SPECULAR OBJECTS , 2003 .

[34]  Andrew M. Wallace,et al.  Improving Depth Image Acquisition Using Polarized Light , 1999, International Journal of Computer Vision.

[35]  Andrew Gardner,et al.  A lighting reproduction approach to live-action compositing , 2002, SIGGRAPH.

[36]  Sang Wook Lee,et al.  Detection of Specularity Using Stereo in Color and Polarization Space , 1997, Comput. Vis. Image Underst..

[37]  Yutaka Takase,et al.  The Great Buddha Project: Modelling Cultural Heritage through Observation , 2001 .

[38]  H. Craighead,et al.  Liquid-crystal micropolarizer array for polarization-difference imaging. , 2002, Applied optics.

[39]  Yoav Y Schechner,et al.  Polarization-based vision through haze. , 2008, Applied optics.

[40]  W. Shurcliff.  Polarized light; production and use , 1962 .

[41]  H Farid,et al.  Separating reflections from images by use of independent component analysis. , 1999, Journal of the Optical Society of America. A, Optics, image science, and vision.

[42]  Hiroshi Murase,et al.  Surface Shape Reconstruction of a Nonrigid Transport Object Using Refraction and Motion , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Assaf Zomet,et al.  Separating reflections from a single image using local features , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[44]  Lawrence B. Wolff,et al.  Polarization-Based Material Classification from Specular Reflection , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Yoshiaki Shirai,et al.  A model-based recognition of glossy objects using their polarimetrical properties , 1987, Adv. Robotics.

[46]  Katsushi Ikeuchi,et al.  Roughness and shape of specular lobe surfaces using photometric sampling method , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[47]  J Shamir,et al.  Polarization and statistical analysis of scenes containing a semireflector. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[48]  Katsushi Ikeuchi,et al.  Polarization-based inverse rendering from a single view , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[49]  Wojciech Matusik,et al.  Acquisition and Rendering of Transparent and Refractive Objects , 2002, Rendering Techniques.

[50]  David W. Jacobs,et al.  Using specularities for recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[51]  R. Courant,et al.  Methods of Mathematical Physics , 1962 .

[52]  T. Aida,et al.  Electrically-Controllable Liquid Crystal Polarizing Filter for Eliminating Reflected Light , 1998 .

[53]  Katsushi Ikeuchi,et al.  Implementation and Evaluation of a Three-Dimensional Photometric Sampler , 1992 .

[54]  Norihiro Abe,et al.  3D Surface Estimation and Model Construction From Specular Motion in Image Sequences , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  E. Jakeman,et al.  Emission polarization of roughened glass and aluminum surfaces. , 1996, Applied optics.

[56]  Stefan Rahmann,et al.  Polarization images: a geometric interpretation for shape analysis , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[57]  Peter K. Allen,et al.  Acquisition and interpretation of 3-D sensor data from touch , 1990, IEEE Trans. Robotics Autom..

[58]  Manfredo P. do Carmo,et al.  Differential geometry of curves and surfaces , 1976 .

[59]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[60]  Hua Chen,et al.  Polarization Phase-Based Method For Material Classification In Computer Vision , 1998, International Journal of Computer Vision.

[61]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[62]  山崎 俊太郎.  Photo-realistic rendering of real-world objects based on insufficient measurement , 2004 .

[63]  Guy Godin,et al.  Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  K. Ikeuchi,et al.  Determining surface orientations of transparent objects based on polarization degrees in visible and infrared wavelengths. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[65]  Katsushi Ikeuchi,et al.  Transparent surface modeling from a pair of polarization images , 2004 .

[66]  Katsushi Ikeuchi,et al.  Determining Surface Orientations of Specular Surfaces by Using the Photometric Stereo Method , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Katsushi Ikeuchi,et al.  Inverse polarization raytracing: estimating surface shapes of transparent objects , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[68]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[69]  M. Docarmo.  Differential geometry of curves and surfaces , 1976 .

[70]  Shree K. Nayar,et al.  What does motion reveal about transparency? , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[71]  Richard Szeliski,et al.  Layer extraction from multiple images containing reflections and transparency , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[72]  Terrance E. Boult,et al.  Constraining Object Features Using a Polarization Reflectance Model , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[73]  Katsushi Ikeuchi,et al.  Temporal-color space analysis of reflection , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[74]  Ronen Basri,et al.  Separation of Transparent Layers using Focus , 2004, International Journal of Computer Vision.

[75]  Shree K. Nayar,et al.  Stereo and Specular Reflection , 1998, International Journal of Computer Vision.

[76]  Emil Wolf,et al.  Principles of Optics: Contents , 1999 .

[77]  Shree K. Nayar,et al.  Separation of Reflection Components Using Color and Polarization , 1997, International Journal of Computer Vision.

[78]  Norimichi Tsumura,et al.  Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin , 2003, ACM Trans. Graph..

[79]  Shree K. Nayar,et al.  Shape Recovery using Physical Models of Reflection and Interreflection , 1991 .

[80]  H. Piaggio.  Differential Geometry of Curves and Surfaces , 1952, Nature.

[81]  Takeo Kanade,et al.  Surface Reflection: Physical and Geometrical Perspectives , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[82]  Richard Szeliski,et al.  Stereo Matching with Transparency and Matting , 1999, International Journal of Computer Vision.

[83]  Makoto Mizukawa,et al.  3D modeling of micro transparent object with integrated vision , 2003, Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI2003..

[84]  Katsushi Ikeuchi,et al.  Creating Photorealistic Virtual Model with Polarization based Vision System , 2005 .

[85]  Stefan Rahmann,et al.  Reconstruction of specular surfaces using polarization imaging , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[86]  Andrew W. Fitzgibbon,et al.  Image-based environment matting , 2002, SIGGRAPH '02.

[87]  Pochi Yeh,et al.  Extended Jones matrix method. II , 1993 .

[88]  James F. Shackelford,et al.  The CRC Materials Science And Engineering Handbook , 1991 .

[89]  Kiriakos N. Kutulakos,et al.  Dynamic Refraction Stereo , 2005, ICCV.

[90]  Atsushi Nakazawa,et al.  Estimating Light Position and Surface Reflectance from Specular Reflection under Perspective Projection , 2002, MVA.

[91]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[92]  O. Drbohlav,et al.  Unambiguous determination of shape from photometric stereo with unknown light sources , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[93]  Tatsuto Takeuchi,et al.  Representation of Pseudo Inter‐reflection and Transparency by Considering Characteristics of Human Vision , 2002, Comput. Graph. Forum.

[94]  S. Hata,et al.  Shape detection of small specular surface using color stripe lighting , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[95]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[96]  Sukeyasu Kanno,et al.  Extraction and recognition of 3-dimensional information by projecting a pair of slit-ray beams , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[97]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[98]  K. Ikeuchi,et al.  Measurement of surface orientations of transparent objects by use of polarization in highlight , 1999 .

[99]  Jun Yang,et al.  Determining a polyhedral shape using interreflections , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[100]  Takeo Kanade,et al.  Determining shape and reflectance of hybrid surfaces by photometric sampling , 1989, IEEE Trans. Robotics Autom..

[101]  Andrew J. Lundberg,et al.  Image understanding from thermal emission polarization , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).