A neural network scheme for transparent surface modelling

This paper presents a new neural network (NN) scheme for recovering three dimensional (3D) transparent surface. We view the transparent surface modeling, not as a separate problem, but as an extension of opaque surface modeling. The main insight of this work is we simulate transparency not only for generating visually realistic images, but for recovering the object shape. We construct a formulation of transparent surface modeling using ray tracing framework into our NN. We compared this ray tracing method, with a texture mapping method that simultaneously map the silhouette images and smooth shaded images (obtained form our NN), and textured images (obtained from the teacher image) to an initial 3D model. By minimizing the images error between the output images of our NN and the teacher images, observed in multiple views, we refine vertices position of the initial 3D model. We show that our NN can refine the initial 3D model obtained by polarization images and converge into more accurate surface.

[1]  Victoria Interrante,et al.  Conveying the 3D Shape of Smoothly Curving Transparent Surfaces via Texture , 1997, IEEE Trans. Vis. Comput. Graph..

[2]  Stefan Rahmann,et al.  Reconstruction of specular surfaces using polarization imaging , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Itsuo Kumazawa,et al.  SA-Optimized Multiple View Smooth Polyhedron Representation NN , 2003, Discovery Science.

[4]  K. Ikeuchi,et al.  Determining surface orientations of transparent objects based on polarization degrees in visible and infrared wavelengths. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[5]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[6]  Lawrence B. Wolff.  Shape from polarization images , 1992 .

[7]  Lawrence B. Wolff,et al.  Ray tracing with polarization parameters , 1990, IEEE Computer Graphics and Applications.

[8]  Itsuo Kumazawa,et al.  A Combinatorial Transparent Surface Modeling from Polarization Images , 2004, IWCIA.

[9]  Katsushi Ikeuchi,et al.  Polarization-based transparent surface modeling from two views , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[11]  Richard Szeliski,et al.  Layer extraction from multiple images containing reflections and transparency , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[12]  Itsuo Kumazawa,et al.  A scheme for reconstructing face from shading using smooth projected polygon representation NN , 2002, Proceedings. International Conference on Image Processing.

[13]  V. Ramachandran,et al.  Transparency: Relation to Depth, Subjective Contours, Luminance, and Neon Color Spreading , 1990, Perception.

[14]  Robert L. Cook,et al.  Distributed ray tracing , 1998 .

[15]  Hiroshi Murase,et al.  Surface Shape Reconstruction of a Nonrigid Transport Object Using Refraction and Motion , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[17]  M. Pauline Baker,et al.  Computer Graphics, C Version , 1996 .

[18]  Andrew W. Fitzgibbon,et al.  Image-based environment matting , 2002, SIGGRAPH '02.

[19]  Seiji Hata,et al.  Shape extraction of transparent object using genetic algorithm , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[20]  Itsuo Kumazawa,et al.  A neural network for recovering 3D shape from erroneous and few depth maps of shaded images , 2004, Pattern Recognit. Lett..

[21]  S. Pizer,et al.  Conveying the 3D Shape of Transparent Surfaces via Texture , 1997 .

[22]  David Salesin,et al.  Environment matting extensions: towards higher accuracy and real-time capture , 2000, SIGGRAPH.

[23]  Katsushi Ikeuchi,et al.  Measurement of surface orientations of transparent objects using polarization in highlight , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[24]  Itsuo Kumazawa,et al.  Multiple-view shape extraction from shading as local regression by analytic NN scheme , 2004, Math. Comput. Model..

[25]  Wojciech Matusik,et al.  Acquisition and Rendering of Transparent and Refractive Objects , 2002, Rendering Techniques.