Simultaneous Registration of 2D Images onto 3D Models for Texture Mapping

We present a new registration method that automatically and simultaneously aligns multiple 2D images onto a 3D model. Correspondences between 2D edge pixels and 3D edge points are automatically searched and updated. Besides these 2D-3D edge correspondences, 2D-2D edge correspondences on 3D surface model are also considered simultaneously for global optimization among all the images. Errors are minimized by utilizing conjugate gradient search, which is processed using M-estimator for robustness. The effectiveness of the proposed method on real input images is shown. Moreover, we also demonstrate that the method is useful for creating digital cultural assets.

[1]  Ioannis Stamos,et al.  3D modeling of historic sites using range and image data , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[2]  Wei Zhang,et al.  Multi-Scale Blur Estimation and Edge Type Classification for Scene Analysis , 1997, International Journal of Computer Vision.

[3]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Paul Debevec.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 2008, SIGGRAPH Classes.

[6]  David S. Ebert,et al.  Visualization and computer graphics , 2007 .

[7]  Katsushi Ikeuchi,et al.  Acquiring a Radiance Distribution to Superimpose Virtual Objects onto Real Scene , 2001, MVA.

[8]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[9]  Katsushi Ikeuchi,et al.  Simultaneous 2D images and 3D geometric model registration for texture mapping utilizing reflectance attribute , 2002 .

[10]  Mark D. Wheeler,et al.  Automatic Modeling and Localization for Object Recognition , 1996 .

[11]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[12]  Yutaka Takase,et al.  The Great Buddha Project: Modelling Cultural Heritage through Observation , 2001 .

[13]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[14]  Ioannis Stamos,et al.  3-D model construction using range and image data , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  Katsushi Ikeuchi,et al.  Eigen-Texture Method: Appearance Compression and Synthesis Based on a 3D Model , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Shree K. Nayar,et al.  High dynamic range imaging: spatially varying pixel exposures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[17]  Hans-Peter Seidel,et al.  Automated texture registration and stitching for real world models , 2000, Proceedings the Eighth Pacific Conference on Computer Graphics and Applications.

[18]  Katsushi Ikeuchi,et al.  Illumination Distribution from Brightness in Shadows: Adaptive Estimation of Illumination Distribution with Unknown Reflectance Properties in Shadow Regions , 1999, ICCV.

[19]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[20]  Richard Szeliski,et al.  Recovering the Position and Orientation of Free-Form Objects from Image Contours Using 3D Distance Maps , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Zhengyou Zhang,et al.  Determining the Epipolar Geometry and its Uncertainty: A Review , 1998, International Journal of Computer Vision.

[23]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Atsushi Nakazawa,et al.  Iterative refinement of range images with anisotropic error distribution , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Konrad Klein,et al.  Texturing 3D Models of Real World Objects from Multiple Unregistered Photographic Views , 1999, Comput. Graph. Forum.

[26]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[27]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[28]  Paolo Cignoni,et al.  Multiple Texture Stitching and Blending on 3D Objects , 1999, Rendering Techniques.

[29]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH.

[30]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[31]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[32]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[33]  K. Ikeuchi,et al.  Robust Simultaneous Registration of Multiple Range Images , 2008 .

[34]  Paul E. Debevec.  Image-Based Lighting , 2002, IEEE Computer Graphics and Applications.

[35]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[36]  Katsushi Ikeuchi,et al.  Robust and adaptive integration of multiple range images with photometric attributes , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[37]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[38]  Ryo Kurazume,et al.  Mapping textures on 3D geometric model using reflectance image , 2005 .