FUSION OF 3D POINT CLOUDS WITH TIR IMAGES FOR INDOOR SCENE RECONSTRUCTION

Abstract. Obtaining accurate 3D descriptions in the thermal infrared (TIR) is a quite challenging task due to the low geometric resolutions of TIR cameras and the low number of strong features in TIR images. Combining the radiometric information of the thermal infrared with 3D data from another sensor is able to overcome most of the limitations in the 3D geometric accuracy. In case of dynamic scenes with moving objects or a moving sensor system, a combination with RGB cameras and profile laserscanners is suitable. As a laserscanner is an active sensor in the visible red or near infrared (NIR) and the thermal infrared camera captures the radiation emitted by the objects in the observed scene, the combination of these two sensors for close range applications are independent from external illumination or textures in the scene. This contribution focusses on the fusion of point clouds from terrestrial laserscanners and RGB cameras with images from thermal infrared mounted together on a robot for indoor 3D reconstruction. The system is geometrical calibrated including the lever arm between the different sensors. As the field of view is different for the sensors, the different sensors record the same scene points not exactly at the same time. Thus, the 3D scene points of the laserscanner and the photogrammetric point cloud from the RGB camera have to be synchronized before point cloud fusion and adding the thermal channel to the 3D points.

[1]  Boris Jutzi,et al.  A STEP TOWARDS DYNAMIC SCENE ANALYSIS WITH ACTIVE MULTI-VIEW RANGE IMAGING SYSTEMS , 2012 .

[2]  E. Coiras,et al.  Segment-based registration technique for visual-infrared images , 2000 .

[3]  Heiko Hirschmüller,et al.  Dense 3D Reconstruction from Wide Baseline Image Sets , 2011, Theoretical Foundations of Computer Vision.

[4]  Uwe Stilla,et al.  Evaluation of Methods for Coregistration and Fusion of Rpas-Based 3d Point Clouds and Thermal Infrared Images , 2016 .

[5]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[6]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[7]  Uwe Stilla,et al.  TOWARDS PEOPLE DETECTION FROM FUSED TIME-OF-FLIGHT AND THERMAL INFRARED IMAGES , 2014 .

[8]  Uwe Stilla,et al.  BUILDING FACADE OBJECT DETECTION FROM TERRESTRIAL THERMAL INFRARED IMAGE SEQUENCES COMBINING DIFFERENT VIEWS , 2015 .

[9]  Uwe Stilla,et al.  THERMAL 3D MAPPING FOR OBJECT DETECTION IN DYNAMIC SCENES , 2014 .

[10]  Ruigang Yang,et al.  Automatic Real-Time Video Matting Using Time-of-Flight Camera and Multichannel Poisson Equations , 2012, International Journal of Computer Vision.

[11]  Kyung-Hoon Bae,et al.  Image fusion in infrared image and visual image using normalized mutual information , 2008, SPIE Defense + Commercial Sensing.

[12]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[13]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[14]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[15]  Uwe Stilla,et al.  Mobile thermal mapping for matching of infrared images with 3D building models and 3D point clouds , 2018 .

[16]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[17]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  T. Luhmann,et al.  INVESTIGATIONS ON A COMBINED RGB / TIME-OF-FLIGHT APPROACH FOR CLOSE RANGE APPLICATIONS , 2012 .

[19]  Thomas Luhmann,et al.  Nahbereichsphotogrammetrie : Grundlagen, Methoden und Anwendungen , 2000 .

[20]  T. Luhmann,et al.  GEOMETRIC CALIBRATION OF THERMOGRAPHIC CAMERAS , 2013 .

[21]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .