An automatic mosaicking method for building facade texture mapping using a monocular close-range image sequence

Abstract This paper presents an automatic mosaicking method for generating building facade textures from a monocular close-range digital image sequence. The process begins with the computation of the camera parameters (except the coordinates of the projective center), which are determined by combining vanishing point geometry with constraints of a straight line bundle as well as prior information of parallel lines in object space. The raw images are later rectified for the purpose of eliminating their salient geometric distortion. Next, automatic retrieval of the relevant image segment is implemented using the detecting range variance by means of the histogram of projective differences between the corresponding points for each of the facades from the raw image sequence. A strip model of the least-squares adjustment, which is similar to the strip block adjustment in aerial triangulation, is employed to determine the spatial alignment of each of the image segments in order to generate the facade textures from the relevant image segments. Afterwards, the entire building facade texture is mosaicked by ortho-image generation. Two refining strategies are proposed to optimize the mosaic result. One is refining the mosaic region where corresponding points are difficult to match but plenty of horizontal lines are available, and the constraint of corresponding horizontal lines is introduced to implement this process. The other strategy is to refine the unsatisfactory mosaic region by densifying the corresponding points by means of the spatial alignment of the relevant image segment computed by the strip method. The experimental results indicate that this method is widely applicable and compares well with other reported approaches with regard to automation level and applicability, for uncalibrated images as well as images with large geometric distortions.

[1]  C. Brenner,et al.  AUTOMATIC MATCHING OF TERRESTRIAL SCAN DATA AS A BASIS FOR THE GENERATION OF DETAILED 3D CITY MODELS , 2004 .

[2]  Ramakant Nevatia,et al.  Integrating ground and aerial views for urban site modeling , 2002, Object recognition supported by user interaction for service robots.

[3]  Seth J. Teller,et al.  Automatic recovery of relative camera rotations for urban scenes , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Tan Yumin,et al.  Lens Distortion Calibration Based on Curvature Variance , 2006 .

[5]  Yong-In Yoon,et al.  An efficient method to build panoramic image mosaics , 2003, Pattern Recognit. Lett..

[6]  Tee-Ann Teo,et al.  Integration of GPS, GIS and Photogrammetry for Texture Mapping in Photo-Realistic City Modeling , 2006, PSIVT.

[7]  G. Gartner,et al.  Lecture Notes in Geoinformation and Cartography , 2006 .

[8]  Norbert Haala,et al.  Panoramic Scenes for Texture Mapping of 3 D City Models , 2005 .

[9]  C. Brenner.  FAST PRODUCTION OF VIRTUAL REALITY CITY MODELS , 2003 .

[10]  George Loizou,et al.  Computer vision and pattern recognition , 2007, Int. J. Comput. Math..

[11]  George Vosselman,et al.  Reconstruction of 3D building models from aerial images and maps , 2003 .

[12]  Christian Früh,et al.  Automated texture mapping of 3D city models with oblique aerial imagery , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[13]  Ryosuke Shibasaki,et al.  Reconstructing a textured CAD model of an urban environment using vehicle-borne laser range scanners and line cameras , 2003, Machine Vision and Applications.

[14]  Christian Früh,et al.  Data Processing Algorithms for Generating Textured 3D Building Facade Meshes from Laser Scans and Camera Images , 2005, International Journal of Computer Vision.

[15]  Yongjun Zhang,et al.  3D Building Modelling with Digital Map, Lidar Data and Video Image Sequences , 2005 .

[16]  Supun Samarasekera,et al.  3D manipulation of motion imagery , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[17]  Eric Maisel,et al.  Using vanishing points for camera calibration and coarse 3D reconstruction from a single image , 2000, The Visual Computer.

[18]  Ramakant Nevatia,et al.  Extraction and integration of window in a 3D building model from ground view images , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[19]  Fuan Tsai,et al.  Semiautomatic texture generation and transformation for cyber city building models , 2005, Proceedings. 2005 IEEE International Geoscience and Remote Sensing Symposium, 2005. IGARSS '05..

[20]  Ioannis Stamos,et al.  3-D model construction using range and image data , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[21]  Jan Boehm.  Multi-image fusion for occlusion-free faÇade texturing , 2004 .

[22]  H. Mayer,et al.  Building facade interprétation from uncalibrated wide-baseline image sequences , 2007 .

[23]  M. Varshosaz.  OCCLUSION-FREE 3 D REALISTIC MODELLING OF BUILDINGS IN URBAN AREAS , 2000 .

[24]  R. Reulke,et al.  Remote Sensing and Spatial Information Sciences , 2005 .

[25]  A. Grün,et al.  Semi-automated approaches to site recording and modeling , 2000 .

[26]  Joseph R. Spann,et al.  Photogrammetry using 3D graphics and projective textures , 2000 .

[27]  Seth J. Teller,et al.  Spherical Mosaics with Quaternions and Dense Correlation , 2000, International Journal of Computer Vision.

[28]  Sisi Zlatanova,et al.  Rapidly Realizing 3D Visualisation for Urban Street Based on Multi-Source Data Integration , 2007 .

[29]  A. G. Fabbri,et al.  Geo-information for disaster management; large scale 3D needed by urban areas , 2005 .

[30]  B. Caprile,et al.  Using vanishing points for camera calibration , 1990, International Journal of Computer Vision.

[31]  A. Zakhor,et al.  Fast 3D model generation in urban environments , 2001, Conference Documentation International Conference on Multisensor Fusion and Integration for Intelligent Systems. MFI 2001 (Cat. No.01TH8590).

[32]  Wolfram Burgard,et al.  A real-time algorithm for mobile robot mapping with applications to multi-robot and 3D mapping , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[33]  Sukhan Lee,et al.  Multisensor Fusion and Integration for Intelligent Systems , 2009 .

[34]  Fabio Remondino,et al.  Image‐based 3D Modelling: A Review , 2006 .

[35]  Fuan Tsai,et al.  Texture Generation and Mapping Using Video Sequences for 3D Building Models , 2006, 3D-GIS.

[36]  Zuxun Zhang,et al.  THE RENDERING OF BUILDING TEXTURE FROM LAND-BASED VIDEO , 2004 .

[37]  Roberto Cipolla,et al.  Camera Calibration from Vanishing Points in Image of Architectural Scenes , 1999, BMVC.

[38]  Ryosuke Shibasaki,et al.  Reconstructing Textured CAD Model of Urban Environment Using Vehicle-Borne Laser Range Scanners and Line Cameras , 2001, ICVS.

[39]  Seth J. Teller,et al.  Extracting textured vertical facades from controlled close-range imagery , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[40]  Sisi Zlatanova,et al.  GEO-INFORMATION SUPPORT IN MANAGEMENT OF URBAN DISASTERS , 2006 .