Feature-based texture mapping from video sequence

This paper introduces a system with a processing pipeline for mapping video images onto an arbitrary 3D model. Given a set of correspondences between 3D model and the video images, the system maps the video images onto the 3D model. Instead of finding the 2D to 3D correspondences between the image and the model, we develop a different formulation that only requires 2D-to-2D matching–a much well studied problem, without the need for camera pose tracking. Starting from a registered image, new images are incrementally registered in 2D to create a mosaic texture map. The 3D model will then be parameterized over the mosaic image through the minimization of the geometric distortion metric using an optimization-based method. Our method does not require calibrated cameras or tracking information, and can also deal with nonrigid deformation to some extent.

[1]  Andrei Khodakovsky,et al.  Globally smooth parameterizations with low distortion , 2003, ACM Trans. Graph..

[2]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[3]  Matthew A. Brown,et al.  Recognising panoramas , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Alla Sheffer,et al.  Cross-parameterization and compatible remeshing of 3D models , 2004, ACM Trans. Graph..

[5]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[6]  Richard Szeliski,et al.  Recovering the Position and Orientation of Free-Form Objects from Image Contours Using 3D Distance Maps , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  David A. Rottenberg,et al.  Cortical surface flattening using least square conformal mapping with minimal metric distortion , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[8]  Toyohisa Kaneko,et al.  Efficient and Handy Texture Mapping on 3D Surfaces , 1999, Comput. Graph. Forum.

[9]  Terry M. Peters,et al.  Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: creating stereoscopic panoramas , 2002, IEEE Transactions on Medical Imaging.

[10]  Katsushi Ikeuchi,et al.  Simultaneous 2D images and 3D geometric model registration for texture mapping utilizing reflectance attribute , 2002 .

[11]  Henrique S. Malvar,et al.  Making Faces , 2019, Topoi.

[12]  Bruno Lévy,et al.  Constrained texture mapping for polygonal meshes , 2001, SIGGRAPH.

[13]  Bruno Lévy,et al.  Least squares conformal maps for automatic texture atlas generation , 2002, ACM Trans. Graph..

[14]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[15]  Craig Gotsman,et al.  Texture Mapping with Hard Constraints , 2001, Comput. Graph. Forum.

[16]  Philip W. Smith,et al.  Stereo-based registration of multi-sensor imagery for enhanced visualization of remote environments , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[17]  Vladislav Kraevoy,et al.  Cross-parameterization and compatible remeshing of 3D models , 2004, SIGGRAPH 2004.

[18]  Konrad Klein,et al.  Texturing 3D Models of Real World Objects from Multiple Unregistered Photographic Views , 1999, Comput. Graph. Forum.

[19]  Marc Rioux,et al.  Registration of range and color images using gradient constraints and range intensity images , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[20]  Richard Szeliski,et al.  Video mosaics for virtual environments , 1996, IEEE Computer Graphics and Applications.

[21]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[22]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Richard Szeliski,et al.  Efficiently registering video into panoramic mosaics , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[24]  Hugues Hoppe,et al.  Inter-surface mapping , 2004, ACM Trans. Graph..

[25]  M. Rioux,et al.  Registration of range and color images using gradient constraints and range intensity images , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[26]  Linda G. Shapiro,et al.  View-base Rendering: Visualizing Real Objects from Scanned Range and Color Data , 1997, Rendering Techniques.

[27]  Ronald Fedkiw,et al.  Visual simulation of smoke , 2001, SIGGRAPH.

[28]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Richard Szeliski,et al.  Using Force Fields Derived from 3D Distance Maps for Inferring the Attitude of a 3D Rigid Object , 1992, ECCV.

[30]  Alla Sheffer,et al.  Matchmaker: constructing constrained texture maps , 2003, ACM Trans. Graph..

[31]  Mark Meyer,et al.  Intrinsic Parameterizations of Surface Meshes , 2002, Comput. Graph. Forum.

[32]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.