Fusion of thermal imagery with point clouds for building façade thermal attribute mapping

Abstract Thermal image data are widely used to assess the insulation quality of buildings and to detect thermal leakages. In our approach, we merge terrestrial thermal image data and 3D point clouds to perform thermal texture mapping for building facades. Since geo-referencing data of a hand-held thermal camera is usually not available in such applications, registration between thermal images and a 3D point cloud (for instance generated from RGB image data by structure-from-motion techniques) is essential. In our approach, thermal image data registration is conducted in four steps: First, another point cloud is generated from the thermal image data. Next, a coarse registration between thermal point cloud and RGB point cloud is performed using the fast global registration (FGR) algorithm. The best corresponding thermal-RGB image pairs are acquired by picking up the lowest Euclidean distance between the exterior orientation parameters of thermal images and transformed exterior orientation parameters of RGB images. Subsequently, radiation-invariant feature transform (RIFT), normalized barycentric coordinate system (NBCS) and random sample consensus (RANSAC) are employed to extract reliable matching features on thermal-RGB image pairs. Afterwards, a fine registration is performed by mono-plotting of the RGB image, followed by image resection of the thermal image. Finally, in terms of texture mapping algorithms, in order to remove the blur effects caused by small misalignments for different candidate images, a global image pose refinement approach, which aims to minimize the temperature disagreements provided by different images for the same object points, is proposed. In addition, in order to ensure high geometric and radiant accuracy, camera calibrations are performed. Experiments showed that the proposed method could not only achieve high geometric registration accuracy, but also provide a good radiometric accuracy with RMSE lower than 1.5 °C.

[1]  Uwe Stilla,et al.  Mobile thermal mapping for matching of infrared images with 3D building models and 3D point clouds , 2018 .

[2]  Vladlen Koltun,et al.  Color map optimization for 3D reconstruction with consumer depth cameras , 2014, ACM Trans. Graph..

[3]  Hideo Saito,et al.  Registration of RGB and Thermal Point Clouds Generated by Structure From Motion , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[4]  Youngjib Ham,et al.  An automated vision-based method for rapid 3D energy performance modeling of existing buildings using thermal and digital imagery , 2013, Adv. Eng. Informatics.

[5]  Ravi Ramamoorthi,et al.  Patch-based optimization for image-based texture mapping , 2017, ACM Trans. Graph..

[6]  Roland Siegwart,et al.  Comparing ICP variants on real-world data sets , 2013, Auton. Robots.

[7]  Vladlen Koltun,et al.  Robust reconstruction of indoor scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Emanuele Mandanici,et al.  Structure from Motion for aerial thermal imagery at city scale: Pre-processing, camera calibration, accuracy assessment , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[9]  Uwe Stilla,et al.  Evaluation of Methods for Coregistration and Fusion of Rpas-Based 3d Point Clouds and Thermal Infrared Images , 2016 .

[10]  H. Maas,et al.  AUTOMATIC SPATIO-TEMPORAL FLOW VELOCITY MEASUREMENT IN SMALL RIVERS USING THERMAL IMAGE SEQUENCES , 2018, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[11]  Martin Drauschke,et al.  CHALLENGES IN FUSION OF HETEROGENEOUS POINT CLOUDS , 2018 .

[12]  H. Maas,et al.  An advanced radiometric calibration approach for uncooled thermal cameras , 2018 .

[13]  David Hernández-López,et al.  Uncooled Thermal Camera Calibration and Optimization of the Photogrammetry Process for UAV Applications in Agriculture , 2017, Sensors.

[14]  Uwe Stilla,et al.  THERMAL 3D MAPPING FOR OBJECT DETECTION IN DYNAMIC SCENES , 2014 .

[15]  Daniel Cohen-Or,et al.  Seamless Montage for Texturing Models , 2010, Comput. Graph. Forum.

[16]  Qingwu Hu,et al.  Robust feature matching for geospatial images via an affine‐invariant coordinate system , 2017 .

[17]  Uwe Stilla,et al.  Camera pose refinement by matching uncertain 3D building models with thermal infrared image sequences for high quality texture extraction , 2017 .

[18]  Hideo Saito,et al.  Superimposing Thermal-Infrared Data on 3D Structure Reconstructed by RGB Visual Odometry , 2018, IEICE Trans. Inf. Syst..

[19]  Uwe Stilla,et al.  QUALITY ASSESSMENT OF BUILDING TEXTURES EXTRACTED FROM OBLIQUE AIRBORNE THERMAL IMAGERY , 2016 .

[20]  Danilo Schneider,et al.  THERMAL TEXTURE SELECTION AND CORRECTION FOR BUILDING FACADE INSPECTION BASED ON THERMAL RADIANT CHARACTERISTICS , 2018, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.