BIM-Based Digital Twin and XR Devices to Improve Maintenance Procedures in Smart Buildings: A Literature Review

In recent years, the use of digital twins (DT) to improve maintenance procedures has increased in various industrial sectors (e.g., manufacturing, energy industry, aerospace) but is more limited in the construction industry. However, the operation and maintenance (O&M) phase of a building’s life cycle is the most expensive. Smart buildings already use BIM (Building Information Modeling) for facility management, but they lack the predictive capabilities of DT. On the other hand, the use of extended reality (XR) technologies to improve maintenance operations has been a major topic of academic research in recent years, both through data display and remote collaboration. In this context, this paper focuses on reviewing projects using a combination of these technologies to improve maintenance operations in smart buildings. This review uses a combination of at least three of the terms “Digital Twin”, “Maintenance”, “BIM” and “Extended Reality”. Results show how a BIM can be used to create a DT and how this DT use combined with XR technologies can improve maintenance operations in a smart building. This paper also highlights the challenges for the correct implementation of a BIM-based DT combined with XR devices. An example of use is also proposed using a diagram of the possible interactions between the user, the DT and the application framework during maintenance operations.

[1]  P. Love,et al.  The ‘how’ of benefits management for digital technology: From engineering to asset management , 2019, Automation in Construction.

[2]  André Loibl,et al.  System-based concept for a mixed reality supported maintenance phase of an industrial plant , 2020 .

[3]  Vladimir Badenko,et al.  Digital twins of complex technical systems for management of built environment , 2020, IOP Conference Series: Materials Science and Engineering.

[4]  Sakdirat Kaewunruen,et al.  Digital Twin Aided Sustainability and Vulnerability Audit for Subway Stations , 2020, Sustainability.

[5]  Jack Chin Pang Cheng,et al.  Data-driven predictive maintenance planning framework for MEP components based on BIM and IoT using machine learning algorithms , 2020 .

[6]  Berardo Naticchia,et al.  Augmented Reality and Deep Learning towards the Management of Secondary Building Assets , 2019, Proceedings of the 36th International Symposium on Automation and Robotics in Construction (ISARC).

[7]  Xiang Xie,et al.  Developing a Digital Twin at Building and City Levels: Case Study of West Cambridge Campus , 2020, Journal of Management in Engineering.

[8]  Sakdirat Kaewunruen,et al.  Digital Twin for Sustainability Evaluation of Railway Station Buildings , 2018, Front. Built Environ..

[9]  Xiang Xie,et al.  An openBIM Approach to IoT Integration with Incomplete As-Built Data , 2020 .

[10]  Andrea Giovanni Mainini,et al.  The Effect of Real-Time Sensing of a Window on Energy Efficiency, Comfort, Health and User Behavior , 2020 .

[11]  Abdul Kareem Almarshad,et al.  A knowledge-based BIM system for building maintenance , 2013 .

[12]  Xun Xu,et al.  Visualisation of the Digital Twin data in manufacturing by using Augmented Reality , 2019, Procedia CIRP.

[13]  Maria Grazia Gnoni,et al.  A classification proposal of digital twin applications in the safety domain , 2021, Comput. Ind. Eng..

[14]  Philip Woodall,et al.  Developing a Dynamic Digital Twin at a Building Level: using Cambridge Campus as Case Study , 2019, International Conference on Smart Infrastructure and Construction 2019 (ICSIC).

[15]  Ray Y. Zhong,et al.  Digital Twin as a Service (DTaaS) in Industry 4.0: An Architecture Reference Model , 2021, Adv. Eng. Informatics.

[16]  Wilfried Sihn,et al.  Digital Twin in manufacturing: A categorical literature review and classification , 2018 .

[17]  Andrew A. West,et al.  Developing augmented reality capabilities for industry 4.0 small enterprises: Lessons learnt from a content authoring case study , 2020, Comput. Ind..

[18]  Samad M. E. Sepasgozar,et al.  Numerical Analysis of the Creep and Shrinkage Experienced in the Sydney Opera House and the Rise of Digital Twin as Future Monitoring Technology , 2019, Buildings.

[19]  William Derigent,et al.  Intelligent products through a SOHOMA prism , 2020 .

[20]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[21]  Jon Martin Fordal,et al.  Operator 4.0 – Emerging Job Categories in Manufacturing , 2018, Advanced Manufacturing and Automation VIII.

[22]  Marc Wenner,et al.  smartBridge Hamburg – prototypische Pilotierung eines digitalen Zwillings , 2020, Bautechnik.

[23]  Nick Dunn,et al.  Digital twins of cities and evasive futures , 2020 .

[24]  Lee Margetts,et al.  Use of gaming technology to bring bridge inspection to the office , 2019, Structure and Infrastructure Engineering.

[25]  Siavash H. Khajavi,et al.  Digital Twin: Vision, Benefits, Boundaries, and Creation for Buildings , 2019, IEEE Access.

[26]  J. Döllner,et al.  Towards The Generation of Digital Twins for Facility Management Based on 3D Point Clouds , 2018 .

[27]  Andreas Gerndt,et al.  Augmented Reality for Remote Collaboration in Aircraft Maintenance Tasks , 2019, 2019 IEEE Aerospace Conference.

[28]  Edward H. Glaessgen,et al.  The Digital Twin Paradigm for Future NASA and U.S. Air Force Vehicles , 2012 .

[29]  V Nyvlt BIM within current building facilities and infrastructure , 2020 .

[30]  Marco Gianinetto,et al.  Digital Transformation of the Design, Construction and Management Processes of the Built Environment , 2019, Research for Development.

[31]  C. U. Grosse Monitoring and Inspection Techniques Supporting a Digital Twin Concept in Civil Engineering , 2019 .

[32]  Dimitris Mourtzis,et al.  Augmented Reality Application to Support Remote Maintenance as a Service in the Robotics Industry , 2017 .

[33]  Shuang Zhang,et al.  Prediction of Highway Tunnel Pavement Performance Based on Digital Twin and Multiple Time Series Stacking , 2020 .

[34]  Xiang Xie,et al.  From BIM Towards Digital Twin: Strategy and Future Development for Smart Asset Management , 2019, SOHOMA.

[35]  Itxaro Errandonea,et al.  Digital Twin for maintenance: A literature review , 2020, Comput. Ind..

[36]  Guillaume Ducellier,et al.  Designing a unique Digital Twin for linear infrastructures lifecycle management , 2019, Procedia CIRP.

[37]  Ying Zuo,et al.  A enhanced interaction framework based on VR, AR and MR in digital twin , 2019, Procedia CIRP.

[38]  J. Erkoyuncu,et al.  Augmented reality assisted calibration of digital twins of mobile robots , 2020 .

[39]  N. Jones,et al.  Top 10 Strategic Technology Trends for 2019: A Gartner Trend Insight Report , 2018 .

[40]  Mohammad Nahangi,et al.  Use Cases for Subcontractors and Fabricators , 2020 .

[41]  Chris Turner,et al.  A Framework for Next Generation Interactive and Immersive DES Models , 2019, 2019 IEEE 17th International Conference on Industrial Informatics (INDIN).

[42]  Xiang Xie,et al.  Visualised inspection system for monitoring environmental anomalies during daily operation and maintenance , 2020 .

[43]  Giuliana Cardani,et al.  Development of the simulation model for Digital Twin applications in historical masonry buildings: The integration between numerical and experimental reality , 2020 .

[44]  F. Farzan,et al.  Improving Building Energy Footprint and Asset Performance Using Digital Twin Technology , 2021, IFAC-PapersOnLine.

[45]  Ngoc-Son Dang,et al.  Bridge assessment for PSC Girder Bridge using Digital Twins Model , 2019, Lecture Notes in Civil Engineering.

[46]  Kari Tammi,et al.  Towards Integrated Digital Twins for Industrial Products: Case Study on an Overhead Crane , 2021 .

[47]  Patrícia Pereira Oliveira,et al.  Digital twin development for airport management , 2020 .

[48]  Roland Wüchner,et al.  IIoT-based Fatigue Life Indication using Augmented Reality , 2019, 2019 IEEE 17th International Conference on Industrial Informatics (INDIN).

[49]  Diogo Vinícius João,et al.  Virtual and Augmented Reality Applied in Power Electric Utilities for Human Interface Improvement – A Study Case for Best Practices , 2020, 2020 IEEE Technology & Engineering Management Conference (TEMSCON).

[50]  Xifan Yao,et al.  Design and management of digital manufacturing and assembly systems in the Industry 4.0 era , 2019, The International Journal of Advanced Manufacturing Technology.

[51]  Eleonora Bottani,et al.  Digital Twin Reference Model Development to Prevent Operators’ Risk in Process Plants , 2020 .

[52]  Kristýna Prušková,et al.  BIM Technology and Changes in Traditional Design Process, Reliability of Data from Related Registers , 2020, IOP Conference Series: Materials Science and Engineering.

[53]  Hakdaoui Sofia,et al.  Mobile Mapping, Machine Learning and Digital Twin for Road Infrastructure Monitoring and Maintenance: Case Study of Mohammed VI Bridge in Morocco , 2020, 2020 IEEE International conference of Moroccan Geomatics (Morgeo).

[54]  Małgorzata Jasiulewicz-Kaczmarek,et al.  Maintenance 4.0 technologies – new opportunities for sustainability driven maintenance , 2023, Management and Production Engineering Review.

[55]  Miguel Oliveira,et al.  Industry Focused in Data Collection: How Industry 4.0 is Handled by Big Data , 2019, DSIT.

[56]  David A. Joy,et al.  Processing Asset Data at the Intelligent Edge: Implementation of an Industrial IoT Architecture to Drive Business Value , 2019 .

[57]  David Baudry,et al.  Inoovas - Industrial ontology for operation in virtual and augmented scene: The architecture , 2017, CoDIT.

[58]  Antonio Padovano,et al.  Ubiquitous knowledge empowers the Smart Factory: The impacts of a Service-oriented Digital Twin on enterprises' performance , 2022, Annu. Rev. Control..

[59]  Yuansong Qiao,et al.  Digital Twin: Origin to Future , 2021, Applied System Innovation.

[60]  Sakdirat Kaewunruen,et al.  Digital twin aided sustainability-based lifecycle management for railway turnout systems , 2019, Journal of Cleaner Production.

[61]  Abdul Kareem Almarshad,et al.  Case-based reasoning and BIM systems for asset management , 2015 .

[62]  Chang-Su Shim,et al.  Development of a bridge maintenance system for prestressed concrete bridges using 3D digital twin model , 2019, Structure and Infrastructure Engineering.

[63]  Baresi Luciano,et al.  Office building occupancy monitoring through image recognition sensors , 2019, International Journal of Safety and Security Engineering.

[64]  Maria Grazia Gnoni,et al.  Is Digital Twin Technology Supporting Safety Management? A Bibliometric and Systematic Review , 2021 .

[65]  Michael W. Grieves,et al.  Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems , 2017 .

[66]  Andreas Witzig,et al.  BIM use-case : model-based performance optimization , 2018 .

[67]  Sisi Zlatanova,et al.  INTEGRATION OF 3D OBJECTS AND TERRAIN FOR 3D MODELLING SUPPORTING THE DIGITAL TWIN , 2019, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[68]  Patrick Morilhat,et al.  Digital Twins, a New Step for Long Term Operation of Nuclear Power Plants , 2020 .

[69]  Xiang Xie,et al.  Digital twin-enabled anomaly detection for built asset monitoring in operation and maintenance , 2020, Automation in Construction.

[70]  Charles Eastman An Outline of the Building Description System. Research Report No. 50. , 1974 .

[71]  Chang-Su Shim,et al.  Digital Twin Models for Maintenance of Cable-Supported Bridges , 2019 .

[72]  Sudeep Tanwar,et al.  Blockchain for Industry 4.0: A Comprehensive Review , 2020, IEEE Access.

[73]  Francesco Longo,et al.  Value-Oriented and Ethical Technology Engineering in Industry 5.0: A Human-Centric Perspective for the Design of the Factory of the Future , 2020, Applied Sciences.

[74]  Ajith Kumar Parlikad,et al.  Design and development of BIM models to support operations and maintenance , 2019, Comput. Ind..

[75]  Arnaud Prouzeau,et al.  Corsican Twin: Authoring In Situ Augmented Reality Visualisations in Virtual Reality , 2020, AVI.

[76]  Lars Windelband Veränderungen in der industriellen Produktion – Notwendige Kompetenzen auf dem Weg vom Internet der Dinge zu Industrie 4.0 , 2015 .

[77]  J. Broekhuijsen,et al.  Lifecycle Governance for Effective Digital Twins: A Joint Systems Engineering and IT Perspective , 2020, 2020 IEEE International Systems Conference (SysCon).

[78]  Valéry Bourny,et al.  Towards improving the future of manufacturing through digital twin and augmented reality technologies , 2018 .

[79]  R.G.J. Damgrave,et al.  Smart Industry Testbed , 2019, Procedia CIRP.

[80]  T. P. Caudell,et al.  Augmented reality: an application of heads-up display technology to manual manufacturing processes , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.

[81]  Peng Yang,et al.  Digital Twin Hospital Buildings: An Exemplary Case Study through Continuous Lifecycle Integration , 2020 .