Improvement of the inspection-repair process with building information modelling and image classification

Purpose Automated technologies have been applied to facility management (FM) practices to address labour demands of, and time consumed by, inputting and processing manual data. Less attention has been focussed on automation of visual information, such as images, when improving timely maintenance decisions. This study aims to develop image classification algorithms to improve information flow in the inspection-repair process through building information modelling (BIM). Design/methodology/approach To improve and automate the inspection-repair process, image classification algorithms were used to connect images with a corresponding image database in a BIM knowledge repository. Quick response (QR) code decoding and Bag of Words were chosen to classify images in the system. Graphical user interfaces (GUIs) were developed to facilitate activity collaboration and communication. A pilot case study in an inspection-repair process was applied to demonstrate the applications of this system. Findings The system developed in this study associates the inspection-repair process with a digital three-dimensional (3D) model, GUIs, a BIM knowledge repository and image classification algorithms. By implementing the proposed application in a case study, the authors found that improvement of the inspection-repair process and automated image classification with a BIM knowledge repository (such as the one developed in this study) can enhance FM practices by increasing productivity and reducing time and costs associated with ecision-making. Originality/value This study introduces an innovative approach that applies image classification and leverages a BIM knowledge repository to enhance the inspection-repair process in FM practice. The system designed provides automated image-classifying data from a smart phone, eliminates time required to input image data manually and improves communication and collaboration between FM personnel for maintenance in the decision-making process.

[1]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[2]  Junjie Wu,et al.  Advances in K-means Clustering , 2012, Springer Theses.

[3]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[4]  Chih-Yung Chen,et al.  QR codes & GPS functions - New applications in TAIWAN , 2014, 2014 International Conference on Machine Learning and Cybernetics.

[5]  Michael Knight,et al.  Delivery of pedestrian real-time location and routing information to mobile architectural guide , 2010 .

[6]  O. Moselhi,et al.  An Automated Barcode System for Tracking and Control of Engineering Deliverables , 2005 .

[7]  Ru-Xi Ding,et al.  Variational Feature Representation-based Classification for face recognition with single sample per person , 2015, J. Vis. Commun. Image Represent..

[8]  Laurence Marsh,et al.  Electronic document management in construction using auto-ID , 1995 .

[9]  Lansford C. Bell,et al.  Bar Code Applications in Construction , 1988 .

[10]  C. Chen,et al.  Benefits of Implementing Building Information Modeling for Healthcare Facility Commissioning , 2011 .

[11]  Martin Kampel,et al.  A Bag of Visual Words Approach for Symbols-Based Coarse-Grained Ancient Coin Classification , 2013, ArXiv.

[12]  Burcin Becerik-Gerber,et al.  Imaged-based verification of as-built documentation of operational buildings , 2012 .

[13]  Sanghoon Lee,et al.  Image-Based Technologies for Constructing As-Is Building Information Models for Existing Buildings , 2017, J. Comput. Civ. Eng..

[14]  Jenny Benois-Pineau,et al.  Fusion in Computer Vision , 2014, Advances in Computer Vision and Pattern Recognition.

[15]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[16]  Fei Dai,et al.  Comparison of Image-Based and Time-of-Flight-Based Technologies for 3D Reconstruction of Infrastructure , 2012 .

[17]  Weiguo Fan,et al.  A new image classification method using CNN transfer learning and web data augmentation , 2018, Expert Syst. Appl..

[18]  C. Pintavirooj,et al.  QR codes application for reversible watermarking algorithm in biomedical images , 2013, The 6th 2013 Biomedical Engineering International Conference.

[19]  Nurdan Öncel Takran,et al.  Handbook of Research on Effective Advertising Strategies in the Social Media Age , 2015 .

[20]  Akif Durdu,et al.  Robotic hand grasping of objects classified by using support vector machine and bag of visual words , 2017, 2017 International Artificial Intelligence and Data Processing Symposium (IDAP).

[21]  Manolis I. A. Lourakis,et al.  Automated as-built 3D reconstruction of civil infrastructure using computer vision: Achievements, opportunities, and challenges , 2015, Adv. Eng. Informatics.

[22]  El Houssine Bouyakhf,et al.  A fast object recognition and categorization technique for robot grasping using the visual bag of words , 2016, 2016 5th International Conference on Multimedia Computing and Systems (ICMCS).

[23]  Amin Hammad,et al.  Knowledge-assisted BIM-based visual analytics for failure root cause detection in facilities management , 2014 .

[24]  Ming-Syan Chen,et al.  A General Scheme for Extracting QR Code from a Non-uniform Background in Camera Phones and Applications , 2007, Ninth IEEE International Symposium on Multimedia (ISM 2007).

[25]  P M Panchal,et al.  A Comparison of SIFT and SURF , 2013 .

[26]  George Stukhart,et al.  Bar‐Code Standardization in Industrial Construction , 1990 .

[27]  Antonio Grilo,et al.  Bibliometric analysis and review of Building Information Modelling literature published between 2005 and 2015 , 2017 .

[28]  Frank Schultmann,et al.  Building Information Modeling (BIM) for existing buildings — Literature review and future needs , 2014 .

[29]  Fouad Khelifi,et al.  Improving a bag of words approach for skin cancer detection in dermoscopic images , 2016, 2016 International Conference on Control, Decision and Information Technologies (CoDIT).

[30]  Andrey Dimitrov,et al.  Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections , 2014, Adv. Eng. Informatics.

[31]  Tatsuya Wakisaka,et al.  Automated construction system for high-rise reinforced concrete buildings , 1997 .

[32]  Raja R. A. Issa,et al.  Design for maintenance accessibility using BIM tools , 2014 .

[33]  Eric M. Wetzel,et al.  The use of a BIM-based framework to support safe facility management processes , 2015 .

[34]  Luc Van Gool,et al.  HPAT Indexing for Fast Object/Scene Recognition Based on Local Appearance , 2003, CIVR.

[35]  Frédéric Bosché,et al.  The value of integrating Scan-to-BIM and Scan-vs-BIM techniques for construction monitoring using laser scanning and BIM: The case of cylindrical MEP components , 2015 .

[36]  Lewis H. Blakey.  Bar Codes: Prescription for Precision, Performance, and Productivity , 1990 .

[37]  Abdul Kareem Almarshad,et al.  A knowledge-based BIM system for building maintenance , 2013 .

[38]  Brenda McCabe,et al.  IFC-Based Development of As-Built and As-Is BIMs Using Construction and Facility Inspection Data: Site-to-BIM Data Transfer Automation , 2018, J. Comput. Civ. Eng..

[39]  Nicolas Rivière,et al.  Ordered minimum distance bag-of-words approach for aerial object identification , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[40]  Martin Kampel,et al.  A rotation-invariant bag of visual words model for symbols based ancient coin classification , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[41]  Trani Marco Lorenzo,et al.  BIM and QR-code. A synergic application in construction site management , 2014 .

[42]  Jochen Teizer,et al.  Development of a Navigational Algorithm in BIM for Effective Utility Maintenance Management of Facilities Equipped with Passive RFID , 2013 .

[43]  Kushal Vyas,et al.  Using Bag of Visual Words and Spatial Pyramid Matching for Object Classification Along with Applications for RIS , 2016 .

[44]  Ronie Navon,et al.  Development and On-Site Evaluation of an Automated Materials Management and Control Model , 2005 .

[45]  David J. Edwards,et al.  The building information modelling trajectory in facilities management: A review , 2017 .

[46]  Hyun-Sang Choi,et al.  BIM perspective definition metadata for interworking facility management data , 2015, Adv. Eng. Informatics.

[47]  Raja R. A. Issa,et al.  Survey: Common Knowledge in BIM for Facility Maintenance , 2016 .

[48]  Nursabillilah Mohd Ali,et al.  Object classification and recognition using Bag-of-Words (BoW) model , 2016, 2016 IEEE 12th International Colloquium on Signal Processing & Its Applications (CSPA).

[49]  Yu-Cheng Lin,et al.  Developing mobile 2D barcode/RFID-based maintenance management system , 2014 .

[50]  Nan Li,et al.  Application Areas and Data Requirements for BIM-Enabled Facilities Management , 2012 .

[51]  Junjie Wu,et al.  Advances in K-means clustering: a data mining thinking , 2012 .

[52]  Fernanda Leite,et al.  Evaluation of accuracy of as-built 3D modeling from photos taken by handheld digital cameras , 2012 .

[53]  Rita Lavikka,et al.  Co-creating digital services with and for facilities management , 2017 .

[54]  Suleiman Wassim,et al.  Buildings Recognition and Camera Localization Using Image Texture Description , 2011 .

[55]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[56]  Daniele Perissin,et al.  Bringing Information to the Field: Automated Photo Registration and 4D BIM , 2018 .

[57]  Hsiang-Cheh Huang,et al.  Reversible data hiding with histogram-based difference expansion for QR code applications , 2011, IEEE Transactions on Consumer Electronics.

[58]  Vinh-Tiep Nguyen,et al.  Object retrieval in past video using bag-of-words model , 2017, 2017 International Conference on Control, Automation and Information Sciences (ICCAIS).

[59]  Philippe De Smedt,et al.  On introducing an image-based 3D reconstruction method in archaeological excavation practice , 2014 .

[60]  Hermann Ney,et al.  Bag-of-visual-words models for adult image classification and filtering , 2008, 2008 19th International Conference on Pattern Recognition.

[61]  Brad Williams,et al.  Resources and Field Technology for Sign Management System Implementation , 2003 .

[62]  Pardis Pishdad-Bozorgi,et al.  Future Smart Facilities: State-of-the-Art BIM-Enabled Facility Management , 2017 .

[63]  Abdul Hadi Nawawi,et al.  Building Information Modelling (BIM) in Facilities Management: Opportunities to be Considered by Facility Managers , 2016 .

[64]  David J. Edwards,et al.  Conceptualising the FinDD API plug-in: A study of BIM-FM integration , 2017 .

[65]  Zhang Xin,et al.  A TV logo detection and recognition method based on SURF feature and bag-of-words model , 2016, 2016 2nd IEEE International Conference on Computer and Communications (ICCC).