Image-based construction of building energy models using computer vision

Abstract Improving existing buildings' energy performance requires energy models that accurately represent the building. Computer vision methods, particularly image-based 3D reconstruction, can effectively support the creation of 3D building models. In this paper, we present an image-based 3D reconstruction pipeline that supports the semi-automated modeling of existing buildings. We developed two methods for the robust estimation of the building planes from a 3D point cloud that (i) independently estimate each plane and (ii) impose a perpendicularity constraint to plane estimation. We also estimate external walls' thermal transmittance values using an infrared thermography-based method, with the surface temperatures measured by a thermal camera. We validate our approach (i) by testing the pipeline's ability in constructing accurate surface models subject to different image sets with varying sizes and levels of image quality, and (ii) through a comparative analysis between the calculated energy performance metrics of a ground truth and calculated energy simulation model.

[1]  Fu Xiao,et al.  Quantitative energy performance assessment methods for existing buildings , 2012 .

[2]  Chang Liu,et al.  Evaluation of Classical Operators and Fuzzy Logic Algorithms for Edge Detection of Panels at Exterior Cladding of Buildings , 2019, Buildings.

[3]  Andrey Dimitrov,et al.  Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections , 2014, Adv. Eng. Informatics.

[4]  Frédéric Bosché,et al.  Automating surface flatness control using terrestrial laser scanning and building information models , 2014 .

[5]  Peter E. D. Love,et al.  Automated detection of workers and heavy equipment on construction sites: A convolutional neural network approach , 2018, Adv. Eng. Informatics.

[6]  FathiHabib,et al.  Automated as-built 3D reconstruction of civil infrastructure using computer vision , 2015 .

[7]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[9]  Mani Golparvar-Fard,et al.  Segmentation of building point cloud models including detailed architectural/structural features and MEP systems , 2015 .

[10]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  H. Opower.  Multiple view geometry in computer vision , 2002 .

[12]  Renato Pajarola,et al.  Automatic room detection and reconstruction in cluttered indoor environments with complex room layouts , 2014, Comput. Graph..

[13]  Jana Kosecka,et al.  Multi-view Superpixel Stereo in Urban Environments , 2010, International Journal of Computer Vision.

[14]  Soteris A. Kalogirou,et al.  Application of infrared thermography for the determination of the overall heat transfer coefficient (U-Value) in building envelopes , 2011 .

[15]  Lamine Mahdjoubi,et al.  Providing real-estate services through the integration of 3D laser scanning and building information modelling , 2013, Comput. Ind..

[16]  Burcu Akinci,et al.  Automatic Creation of Semantically Rich 3D Building Models from Laser Scanner Data , 2013 .

[17]  A. Adán,et al.  Fusion of thermal imagery and LiDAR data for generating TBIM models , 2017, 2017 IEEE SENSORS.

[18]  Ming-Der Yang,et al.  Fusion of Infrared Thermal Image and Visible Image for 3D Thermal Model Reconstruction Using Smartphone Sensors , 2018, Sensors.

[19]  Jan Hensen,et al.  Building Performance Simulation for Design and Operation , 2019 .

[20]  Mani Golparvar-Fard,et al.  Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques , 2011 .

[21]  Manolis I. A. Lourakis,et al.  Toward automated generation of parametric BIMs based on hybrid video and laser scanning data , 2010, Adv. Eng. Informatics.

[22]  Pascal Monasse,et al.  OpenMVG: Open Multiple View Geometry , 2016, RRPR@ICPR.

[23]  Slobodan Ribaric,et al.  A knowledge-based system for the non-destructive diagnostics of façade isolation using the information fusion of visual and IR images , 2009, Expert Syst. Appl..

[24]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Brenda McCabe,et al.  Automated computer vision-based detection of components of under-construction indoor partitions , 2017 .

[26]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[27]  Enrique Valero,et al.  Automatic Method for Building Indoor Boundary Models from Dense Point Clouds Collected by Laser Scanners , 2012, Sensors.

[28]  Fabio Fatiguso,et al.  Historic Building Information Modelling: performance assessment for diagnosis-aided information modelling and management , 2018 .

[29]  Kinam Kim,et al.  Data-driven scene parsing method for recognizing construction site objects in the whole image , 2016 .

[30]  Diego González-Aguilera,et al.  Novel approach to 3D thermography and energy efficiency evaluation , 2012 .

[31]  Michela Chiogna,et al.  A comprehensive experimental approach for the validation of quantitative infrared thermography in the evaluation of building thermal transmittance , 2015 .

[32]  Antonio Costanzo,et al.  Combined Use of Terrestrial Laser Scanning and IR Thermography Applied to a Historical Building , 2014, Sensors.

[33]  Andreas Kroll,et al.  Data fusion for 3D thermal imaging using depth and stereo camera for robust self-localization , 2018, 2018 IEEE Sensors Applications Symposium (SAS).

[34]  Ioannis Brilakis,et al.  Progressive 3D reconstruction of infrastructure with videogrammetry , 2011 .

[35]  Hyeran Byun,et al.  Detecting Construction Equipment Using a Region-Based Fully Convolutional Network and Transfer Learning , 2018, J. Comput. Civ. Eng..

[36]  Chao Wang,et al.  Automatic BIM component extraction from point clouds of existing buildings for sustainability applications , 2015 .

[37]  Long Chen,et al.  Image-driven fuzzy-based system to construct as-is IFC BIM objects , 2018, Automation in Construction.

[38]  Hanqing Lu,et al.  Fast and Accurate Image Matching with Cascade Hashing for 3D Reconstruction , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[40]  Andrew Parker,et al.  The OpenStudio Software Development Kit , 2018 .

[41]  Pedro Arias,et al.  Automation of thermographic 3D modelling through image fusion and image matching techniques , 2011 .

[42]  Markus König,et al.  Interior construction state recognition with 4D BIM registered image sequences , 2018 .

[43]  Mohamed Al-Hussein,et al.  A vision-based system for pre-inspection of steel frame manufacturing , 2019, Automation in Construction.

[44]  Dario Ambrosini,et al.  U-value assessment by infrared thermography: A comparison of different calculation methods in a Guarded Hot Box , 2016 .

[45]  Constantinos A. Balaras,et al.  Infrared thermography for building diagnostics , 2002 .

[46]  S. Umeyama,et al.  Least-Squares Estimation of Transformation Parameters Between Two Point Patterns , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Youngjib Ham,et al.  An automated vision-based method for rapid 3D energy performance modeling of existing buildings using thermal and digital imagery , 2013, Adv. Eng. Informatics.

[48]  Manolis I. A. Lourakis,et al.  Automated as-built 3D reconstruction of civil infrastructure using computer vision: Achievements, opportunities, and challenges , 2015, Adv. Eng. Informatics.

[49]  Frédéric Bosché,et al.  Tracking the Built Status of MEP Works: Assessing the Value of a Scan-vs-BIM System , 2014, J. Comput. Civ. Eng..

[50]  Ardeshir Mahdavi.  Simulation-based control of building systems operation , 2001 .

[51]  Burcu Akinci,et al.  A formalism for utilization of sensor systems and integrated project models for active construction quality control , 2006 .

[52]  David S. Citrin,et al.  An analogy between various machine-learning techniques for detecting construction materials in digital images , 2015, KSCE Journal of Civil Engineering.

[53]  Antonio Adán,et al.  Semantic 3D Reconstruction of Furnished Interiors Using Laser Scanning and RFID Technology , 2016 .

[54]  Jürgen Döllner,et al.  Concepts and techniques for integration, analysis and visualization of massive 3D point clouds , 2014, Comput. Environ. Urban Syst..

[55]  Pascal Monasse,et al.  Adaptive Structure from Motion with a Contrario Model Estimation , 2012, ACCV.

[56]  Antonio Adán,et al.  Scan-to-BIM for 'secondary' building components , 2018, Adv. Eng. Informatics.

[57]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Burcu Akinci,et al.  Automatic Reconstruction of As-Built Building Information Models from Laser-Scanned Point Clouds: A Review of Related Techniques | NIST , 2010 .

[59]  Daniele Perissin,et al.  Bringing Information to the Field: Automated Photo Registration and 4D BIM , 2018 .

[60]  Enrique Valero,et al.  Automatic segmentation of 3D point clouds of rubble masonry walls, and its application to building surveying, repair and maintenance , 2018, Automation in Construction.

[61]  Jaehoon Jung,et al.  Productive modeling for development of as-built BIM of existing indoor structures , 2014 .

[62]  Renzhong Guo,et al.  Fast and Automatic Reconstruction of Semantically Rich 3D Indoor Maps from Low-quality RGB-D Sequences , 2019, Sensors.

[63]  Frank Schultmann,et al.  Building Information Modeling (BIM) for existing buildings — Literature review and future needs , 2014 .

[64]  Nashwan Dawood,et al.  On-demand monitoring of construction projects through a game-like hybrid application of BIM and machine learning , 2020, Automation in Construction.

[65]  Antonio Adán,et al.  Geometric and Colour Data Fusion for Outdoor 3D Models , 2012, Sensors.