On-the-fly extrinsic calibration of multimodal sensing system for fast 3D thermographic scanning.

The fusion of three-dimensional (3D) geometrical and two-dimensional (2D) thermal information provides a promising method for characterizing temperature distribution of 3D objects, extending infrared imaging from 2D to 3D to support various thermal inspection applications. In this paper, we present an effective on-the-fly calibration approach for accurate alignment of depth and thermal data to facilitate dynamic and fast-speed 3D thermal scanning tasks. For each pair of depth and thermal frames, we estimate their relative pose by minimizing the objective function that measures the temperature consistency between a 2D infrared image and the reference 3D thermographic model. Our proposed frame-to-model mapping scheme can be seamlessly integrated into a generic 3D thermographic reconstruction framework. Through graphics-processing-unit-based acceleration, our method requires less than 10 ms to generate a pair of well-aligned depth and thermal images without hardware synchronization and improves the robustness of the system against significant camera motion.

[1]  Song Zhang,et al.  High-resolution, real-time simultaneous 3D surface geometry and temperature measurement. , 2016, Optics express.

[2]  Girum G. Demisse,et al.  Interpreting Thermal 3D Models of Indoor Environments for Energy Efficiency , 2013, 2013 16th International Conference on Advanced Robotics (ICAR).

[3]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[4]  H. Macher,et al.  FIRST EXPERIENCES WITH KINECT V2 SENSOR FOR CLOSE RANGE 3D MODELLING , 2015 .

[5]  Andreas Nüchter,et al.  Thermal 3D Mapping of Building Façades , 2012, IAS.

[6]  John J. Leonard,et al.  Real-time large-scale dense RGB-D SLAM with volumetric fusion , 2014, Int. J. Robotics Res..

[7]  A. Ianiro,et al.  Temperature maps measurements on 3D surfaces with infrared thermography , 2012 .

[8]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Peyman Moghadam,et al.  HeatWave : a handheld 3D thermography system for energy auditing , 2013 .

[10]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Andreas Kroll,et al.  Generating High Fidelity 3-D Thermograms With a Handheld Real-Time Thermal Imaging System , 2017, IEEE Sensors Journal.

[12]  Bin Yang,et al.  Accurate calibration of a multi-camera system based on flat refractive geometry. , 2017, Applied optics.

[13]  Jae-Hean Kim,et al.  Convenient calibration method for unsynchronized camera networks using an inaccurate small reference object. , 2012, Optics express.

[14]  Xin Li,et al.  Depth and thermal sensor fusion to enhance 3D thermographic reconstruction. , 2018, Optics express.

[15]  J. Armesto,et al.  Energy efficiency studies through 3D laser scanning and thermographic technologies , 2011 .

[16]  Xavier Maldague,et al.  Carbon fiber composites inspection and defect characterization using active infrared thermography: numerical simulations and experimental results. , 2016, Applied optics.

[17]  Sridha Sridharan,et al.  Real-Time Mobile 3D Temperature Mapping , 2015, IEEE Sensors Journal.

[18]  Konrad Polthier,et al.  Anisotropic smoothing of point sets, , 2005, Comput. Aided Geom. Des..

[19]  Christoph van Treeck,et al.  Real-time human skin temperature analysis using thermal image recognition for thermal comfort assessment , 2018 .

[20]  Gennaro Cardone,et al.  Measurement of surface temperature and emissivity with stereo dual-wavelength IR thermography , 2010 .

[21]  Majeed M Hayat,et al.  Detection theory for accurate and non-invasive skin cancer diagnosis using dynamic thermal imaging. , 2017, Biomedical optics express.

[22]  Bisheng Yang,et al.  Automatic registration of panoramic image sequence and mobile laser scanning data using semantic features , 2018 .