High-quality as-is 3D thermal modeling in MEP systems using a deep convolutional network

Abstract With the growing need for automated condition monitoring and analysis in existing buildings, significant effort has been spent on the development of three-dimensional (3D) thermal models. However, little attention has been paid to ensuring the quality of these 3D thermal models, which can directly impact the accuracy of condition monitoring and analysis results. This study aims to propose a method to generate a high-quality 3D thermal model for mechanical, electrical, and plumbing (MEP) systems by bridging the quality discrepancy between high-resolution laser scan data and low-resolution thermal images using a deep convolutional neural network. The proposed method consists of two main parts: (1) improving the resolution of thermal images based on a deep convolutional network and (2) generating a high-quality 3D thermal model by mapping improved thermal images. The performance of the thermal image resolution improvement was validated using a dataset consisting of 312 thermal images. The results demonstrated that the quality of the improved thermal images based on a deep convolutional network was higher than conventional bicubic interpolation in terms of root mean square error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM). Qualitative analysis of a 3D thermal model utilizing the resolution-improved thermal images was also conducted. This was further qualitatively analyzed to have resulted in improved overall quality of the 3D thermal model. The ability to generate a high-quality 3D thermal model can help auditors to perform automated condition monitoring and analysis in buildings based on objective and accurate data.

[1]  Youngjib Ham,et al.  3D as-is building energy modeling and diagnostics: A review of the state-of-the-art , 2015, Adv. Eng. Informatics.

[2]  Kin-Man Lam,et al.  Example-based image super-resolution with class-specific predictors , 2009, J. Vis. Commun. Image Represent..

[3]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[4]  Mariano Alcañiz Raya,et al.  Design and validation of an augmented book for spatial abilities development in engineering students , 2010, Comput. Graph..

[5]  Florian Bugarin,et al.  Automated thermal 3D reconstruction based on a robot equipped with uncalibrated infrared stereovision cameras , 2018, Adv. Eng. Informatics.

[6]  Mani Golparvar-Fard,et al.  Automated Diagnostics and Visualization of Potential Energy Performance Problems in Existing Buildings Using Energy Performance Augmented Reality Models , 2014, J. Comput. Civ. Eng..

[7]  Pileun Kim,et al.  Robotic sensing and object recognition from thermal-mapped point clouds , 2017, International Journal of Intelligent Robotics and Applications.

[8]  Jeff Wood,et al.  Super‐resolution musculoskeletal MRI using deep learning , 2018, Magnetic resonance in medicine.

[9]  Ingrid Bouwer Utne,et al.  A structured approach to improved condition monitoring , 2012 .

[10]  Paolo Pittiglio,et al.  The management of mechanical integrity inspections at small-sized "Seveso" facilities , 2009, Reliab. Eng. Syst. Saf..

[11]  Frédéric Bosché,et al.  As-built data acquisition and its use for production monitoring and automated layout in civil infrastructure: a survey , 2015 .

[12]  Youngjib Ham,et al.  An automated vision-based method for rapid 3D energy performance modeling of existing buildings using thermal and digital imagery , 2013, Adv. Eng. Informatics.

[13]  Bernard Victorri,et al.  Transformation invariance in pattern recognition: Tangent distance and propagation , 2000 .

[14]  Diego González-Aguilera,et al.  Novel approach to 3D thermography and energy efficiency evaluation , 2012 .

[15]  Junjun Jiang,et al.  Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means , 2017, IEEE Transactions on Multimedia.

[16]  T. Jayakumar,et al.  Infrared thermography for condition monitoring – A review , 2013 .

[17]  Changmin Kim,et al.  3D reconstruction of as-built industrial instrumentation models from laser-scan data and a 3D CAD database based on prior knowledge , 2015 .

[18]  Namil Kim,et al.  Thermal Image Enhancement using Convolutional Neural Network , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Lingfeng Wang,et al.  Ensemble based deep networks for image super-resolution , 2017, Pattern Recognit..

[20]  Rogério Schmidt Feris,et al.  Edge-Guided Single Depth Image Super Resolution , 2016, IEEE Transactions on Image Processing.

[21]  Yunsong Li,et al.  Hyperspectral image reconstruction by deep convolutional neural network for classification , 2017, Pattern Recognit..

[22]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Lucía Díaz-Vilariño,et al.  Automatic thermographic and RGB texture of as-built BIM for energy rehabilitation purposes , 2013 .

[24]  Peyman Moghadam,et al.  HeatWave : a handheld 3D thermography system for energy auditing , 2013 .

[25]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[26]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[27]  Hu Chen,et al.  Low-dose CT via convolutional neural network. , 2017, Biomedical optics express.

[28]  Yong K. Cho,et al.  As-Is 3D Thermal Modeling for Existing Building Envelopes Using a Hybrid LIDAR System , 2013 .

[29]  S. D. Probert,et al.  Reducing the cost of preventive maintenance (PM) through adopting a proactive reliability-focused culture , 2006 .

[30]  Peyman Milanfar,et al.  RAISR: Rapid and Accurate Image Super Resolution , 2016, IEEE Transactions on Computational Imaging.

[31]  Pedro Arias,et al.  Automation of thermographic 3D modelling through image fusion and image matching techniques , 2011 .

[32]  Jun Guo,et al.  An empirical convolutional neural network approach for semantic relation classification , 2016, Neurocomputing.

[33]  Xiangjun Zhang,et al.  Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation , 2008, IEEE Transactions on Image Processing.

[34]  Karen O. Egiazarian,et al.  Single Image Super-Resolution Based on Wiener Filter in Similarity Domain , 2017, IEEE Transactions on Image Processing.

[35]  Mani Golparvar-Fard,et al.  EPAR: Energy Performance Augmented Reality models for identification of building energy performance deviations between actual measurements and simulation results , 2013 .

[36]  Mani Golparvar-Fard,et al.  Three-Dimensional Thermography-Based Method for Cost-Benefit Analysis of Energy Efficiency Building Envelope Retrofits , 2015, J. Comput. Civ. Eng..