Mobile thermal mapping for matching of infrared images with 3D building models and 3D point clouds

Abstract Two workflows for mobile thermal mapping including geometric camera calibration, automatic image orientation, 3D reconstruction and extraction of textures of building façades from thermal IR image sequences are introduced. In the first method, a coregistration of a terrestrial image sequence is done with a given 3D building model with the 3D building model being included in the image sequence orientation directly. Homologue image points are tracked through the sequence and checked whether they are on the 3D model façade. This is done to remove outliers and to refine the image sequence orientation and 3D reconstruction process as additional constraint in the bundle adjustment. The second method is used to match a high resoluted RGB-image based 3D point cloud and a TIR-image based 3D point cloud to assign thermal intensities to the dense RGB point cloud. The matching is done based on given pre-orientation of the two-point clouds performing Iterative Closest Point to minimise the distance of the two-point clouds. The corrected orientation is transferred to the orientation parameters of the TIR-images. The final image orientations are used to extract thermal textures for the 3D building model. Both methods are evaluated on their geometric accuracy using experimental image sequences.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Uwe Stilla,et al.  Automatic generation of façade textures from terrestrial thermal infrared image sequences. , 2014 .

[3]  Pedro Arias,et al.  Calibration and verification of thermographic cameras for geometric measurements , 2011 .

[4]  T. Luhmann,et al.  GEOMETRIC CALIBRATION OF THERMOGRAPHIC CAMERAS , 2013 .

[5]  Helmut Mayer.  3D Reconstruction and Visualization of Urban Scenes from Uncalibrated Wide-Baseline Image Sequences , 2008 .

[6]  Donald Prevost,et al.  Passive calibration board for alignment of VIS-NIR, SWIR and LWIR images , 2017 .

[7]  Alexander Hornberg.  Handbook of Machine Vision , 2006 .

[8]  Xinhua Zhuang,et al.  Pose estimation from corresponding point data , 1989, IEEE Trans. Syst. Man Cybern..

[9]  Uwe Stilla,et al.  Texture mapping of 3D building models with oblique direct geo-referenced airborne IRimage sequences , 2009 .

[10]  T. Läbe.  AUTOMATIC RELATIVE ORIENTATION OF IMAGES , 2006 .

[11]  L. Zhi,et al.  A Complete Linear 4-Point Algorithm for Camera Pose Determination 1 ) , 2003 .

[12]  Steve Goodhew,et al.  Building defect detection: External versus internal thermography , 2016 .

[13]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[14]  Bill Triggs,et al.  Camera pose and calibration from 4 or 5 known 3D points , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[15]  Radu Horaud,et al.  An analytic solution for the perspective 4-point problem , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  M HaralickRobert,et al.  Review and analysis of solutions of the three point perspective pose estimation problem , 1994 .

[17]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Soteris A. Kalogirou,et al.  Infrared thermography (IRT) applications for building diagnostics: A review , 2014 .

[19]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Uwe Stilla,et al.  TOWARDS PEOPLE DETECTION FROM FUSED TIME-OF-FLIGHT AND THERMAL INFRARED IMAGES , 2014 .

[21]  Long Quan,et al.  Linear N-Point Camera Pose Determination , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[23]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[24]  Heiko Hirschmüller,et al.  Dense 3D Reconstruction from Wide Baseline Image Sets , 2011, Theoretical Foundations of Computer Vision.

[25]  U. Stilla,et al.  TEXTURE MAPPING OF 3 D BUILDING MODELS WITH OBLIQUE DIRECT GEO-REFERENCED AIRBORNE IR IMAGE SEQUENCES , 2009 .

[26]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[27]  Richard I. Hartley,et al.  Lines and Points in Three Views and the Trifocal Tensor , 1997, International Journal of Computer Vision.

[28]  Uwe Stilla,et al.  Reconstruction of façades in point clouds from multi aspect oblique ALS , 2013, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[29]  Vincent Lepetit,et al.  Point matching as a classification problem for fast and robust object pose estimation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[30]  V. Rodehorst,et al.  ROBUST SPATIO-TEMPORAL FEATURE TRACKING , 2008 .

[31]  Uwe Stilla,et al.  Gestalt Grouping on Façade Textures from IR Image Sequences: Comparing Different Production Systems , 2012 .

[32]  Uwe Stilla,et al.  Evaluation of Methods for Coregistration and Fusion of Rpas-Based 3d Point Clouds and Thermal Infrared Images , 2016 .

[33]  Uwe Stilla,et al.  Automatic extraction and database integration of textures of 3D building models acquired from infrared image sequences , 2007 .

[34]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[35]  Bodo Rosenhahn,et al.  Outdoor and Large-Scale Real-World Scene Analysis , 2012, Lecture Notes in Computer Science.

[36]  Uwe Stilla,et al.  BUILDING FACADE OBJECT DETECTION FROM TERRESTRIAL THERMAL INFRARED IMAGE SEQUENCES COMBINING DIFFERENT VIEWS , 2015 .

[37]  Uwe Stilla,et al.  Line based Matching of Uncertain 3D Building Models with IR Image Sequences for Precise Texture Extraction , 2012 .