A systematic approach for 2D-image to 3D-range registration in urban environments

The photorealistic modeling of large-scale objects, such as urban scenes, requires the combination of range sensing technology and digital photography. In this paper, we attack the key problem of camera pose estimation, in an automatic and efficient way. First, the camera orientation is recovered by matching vanishing points (extracted from 2D images) with 3D directions (derived from a 3D range model). Then, a hypothesis-and-test algorithm computes the camera positions with respect to the 3D range model by matching corresponding 2D and 3D linear features. The camera positions are further optimized by minimizing a line-to-line distance. The advantage of our method over earlier work has to do with the fact that we do not need to rely on extracted planar facades, or other higher-order features; we are utilizing low-level linear features. That makes this method more general, robust, and efficient. We have also developed a user-interface for allowing users to accurately texture-map 2D images onto 3D range models at interactive rates. We have tested our system in a large variety of urban scenes.

[1]  Ioannis Stamos,et al.  A systematic approach for 2D-image to 3D-range registration in urban environments , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[2]  Yanxi Liu,et al.  Detecting and matching repeated patterns for automatic geo-tagging in urban environments , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Larry S. Davis,et al.  Iterative Pose Estimation Using Coplanar Feature Points , 1996, Comput. Vis. Image Underst..

[4]  Gerd Häusler,et al.  Feature-Based Object Recognition and Localization in 3D-Space, Using a Single Video Image , 1999, Comput. Vis. Image Underst..

[5]  Wenyi Zhao,et al.  Alignment of Continuous Video onto 3D Point Clouds , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[7]  V. Michael Bove,et al.  Vision-assisted modeling for model-based video representations , 1997 .

[8]  Emanuele Trucco,et al.  Introductory techniques for 3-D computer vision , 1998 .

[9]  William M. Wells,et al.  Statistical Approaches to Feature-Based Object Recognition , 2004, International Journal of Computer Vision.

[10]  Ioannis Stamos,et al.  Geometry and Texture Recovery of Scenes of Large Scale , 2002, Comput. Vis. Image Underst..

[11]  Zuzana Kukelova,et al.  A general solution to the P4P problem for camera with unknown focal length , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Charles V. Stewart,et al.  Estimating the Location of a Camera with Respect to a 3D Model , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[13]  Seth J. Teller,et al.  Scalable Extrinsic Calibration of Omni-Directional Image Networks , 2002, International Journal of Computer Vision.

[14]  Linda G. Shapiro,et al.  Acquisition and visualization of colored 3D objects , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[15]  Ryosuke Shibasaki,et al.  Reconstructing a textured CAD model of an urban environment using vehicle-borne laser range scanners and line cameras , 2003, Machine Vision and Applications.

[16]  Ioannis Stamos,et al.  Automatic registration of 2-D with 3-D imagery in urban environments , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[17]  Frédéric Jurie,et al.  Solution of the Simultaneous Pose and Correspondence Problem Using Gaussian Error Model , 1999, Comput. Vis. Image Underst..

[18]  Ioannis Stamos,et al.  Automatic 3D to 2D registration for the photorealistic rendering of urban scenes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Olivier Faugeras,et al.  Three-Dimensional Computer Vision , 1993 .

[20]  David W. Jacobs.  Matching 3-D Models to 2-D Images , 2004, International Journal of Computer Vision.

[21]  Christian Früh,et al.  Constructing 3D City Models by Merging Aerial and Ground Views , 2003, IEEE Computer Graphics and Applications.

[22]  B. Caprile,et al.  Using vanishing points for camera calibration , 1990, International Journal of Computer Vision.

[23]  Stéphane Christy,et al.  Iterative Pose Computation from Line Correspondences , 1999, Comput. Vis. Image Underst..

[24]  George Wolberg,et al.  Multiview Geometry for Texture Mapping 2D Images Onto 3D Range Data , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[25]  Shimon Ullman,et al.  Recognizing solid objects by alignment with an image , 1990, International Journal of Computer Vision.

[26]  Peter K. Allen,et al.  A Shadow Based Method for Image to Model Registration , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[27]  Fadi Dornaika,et al.  Object Pose: The Link between Weak Perspective, Paraperspective, and Full Perspective , 1997, International Journal of Computer Vision.

[28]  Ioannis Stamos,et al.  Integrating Automated Range Registration with Multiview Geometry for the Photorealistic Modeling of Large-Scale Scenes , 2008, International Journal of Computer Vision.

[29]  Long Quan,et al.  Linear N-Point Camera Pose Determination , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[31]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[33]  Allen R. Hanson,et al.  Robust methods for estimating pose and a sensitivity analysis , 1994 .

[34]  Todd A. Cass.  Polynomial-Time Geometric Matching for Object Recognition , 2004, International Journal of Computer Vision.

[35]  Chia-Ling Tsai,et al.  The dual-bootstrap iterative closest point algorithm with application to retinal image registration , 2003, IEEE Transactions on Medical Imaging.