Monitoring Ancient Buildings: Real Deployment of an IoT System Enhanced by UAVs and Virtual Reality

The historical buildings of a nation are the tangible signs of its history and culture. Their preservation deserves considerable attention, being of primary importance from a historical, cultural, and economic point of view. Having a scalable and reliable monitoring system plays an important role in the Structural Health Monitoring (SHM): therefore, this paper proposes an Internet of Things (IoT) architecture for a remote monitoring system that is able to integrate, through the Virtual Reality (VR) paradigm, the environmental and mechanical data acquired by a wireless sensor network set on three ancient buildings with the images and context information acquired by an Unmanned Aerial Vehicle UAV. Moreover, the information provided by the UAV allows to promptly inspect the critical structural damage, such as the patterns of cracks in the structural components of the building being monitored. Our approach opens new scenarios to support SHM activities, because an operator can interact with real-time data retrieved from a Wireless Sensor Network (WSN) by means of the VR environment.

[1]  BhuiyanMd Zakirul Alam,et al.  Sensor Placement with Multiple Objectives for Structural Health Monitoring , 2014 .

[2]  Henrique Lorenzo,et al.  Monitoring and Assessing Structural Damage in Historic Buildings , 2008 .

[3]  Jerome P. Lynch,et al.  Electrical Impedance Tomographic Methods for Sensing Strain Fields and Crack Damage in Cementitious Structures , 2009 .

[4]  Chen-Ming Kuo,et al.  Unmanned Aircraft Systems for Remote Building Inspection and Monitoring , 2012 .

[5]  Davide Moroni,et al.  Long-Term Monitoring of Crack Patterns in Historic Structures Using UAVs and Planar Markers: A Preliminary Study , 2018, J. Imaging.

[6]  Mohamed S. Shehata,et al.  Structural Health Monitoring Using Wireless Sensor Networks: A Comprehensive Survey , 2017, IEEE Communications Surveys & Tutorials.

[7]  Erina Ferro,et al.  Environmental Monitoring for Smart Cities , 2017, IEEE Sensors Journal.

[8]  Davide Moroni,et al.  Towards Structural Monitoring and 3D Documentation of Architectural Heritage Using UAV , 2018, MISSI.

[9]  Christoforos Kanellakis,et al.  Survey on Computer Vision for UAVs: Current Developments and Trends , 2017, Journal of Intelligent & Robotic Systems.

[10]  F C Sham,et al.  FLASH THERMOGRAPHY Surface crack detection by flash thermography on concrete surface , 2008 .

[11]  Tran Hiep Dinh,et al.  System Architecture for Real-Time Surface Inspection Using Multiple UAVs , 2019, IEEE Systems Journal.

[12]  Vincenzo Gattulli,et al.  Structural health monitoring of the Basilica S: Maria di Collemaggio , 2013 .

[13]  Paolo Barsocchi,et al.  Sensing a City's State of Health: Structural Monitoring System by Internet-of-Things Wireless Sensing Devices , 2018, IEEE Consumer Electronics Magazine.

[14]  Andrea Berton,et al.  Architectural Heritage: 3D Documentation and Structural Monitoring Using UAV , 2019, VIPERC@IRCDL.

[15]  Mani Maran Ratnam,et al.  Novel moiré‐based crack monitoring system with smartphone interface and cloud processing , 2019, Structural Control and Health Monitoring.

[16]  Anu Pradhan,et al.  Masonry Crack Detection Application of an Unmanned Aerial Vehicle , 2014 .

[17]  Michael Forde,et al.  Application of infrared thermography to the non-destructive testing of concrete and masonry bridges , 2003 .

[18]  Mingchu Li,et al.  Initial Validation of Mobile-Structural Health Monitoring Method Using Smartphones , 2015, Int. J. Distributed Sens. Networks.

[19]  Pietro Cassarà,et al.  Modeling Reliable M2M/IoT Traffic Over Random Access Satellite Links in Non-Saturated Conditions , 2018, IEEE Journal on Selected Areas in Communications.

[20]  Bin Xu,et al.  Global Navigation Satellite System‐based positioning technology for structural health monitoring: a review , 2019, Structural Control and Health Monitoring.

[21]  Satoshi Nishiyama,et al.  Improved digital photogrammetry technique for crack monitoring , 2015, Adv. Eng. Informatics.

[22]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[23]  Hao Wu,et al.  Environmental Monitoring of Ancient Buildings Based on a Wireless Sensor Network , 2018, Sensors.

[24]  Yan Yu,et al.  Design of a wireless measurement system based on WSNs for large bridges , 2014 .

[25]  Jerome P. Lynch,et al.  Structural health monitoring: technological advances to practical implementations [scanning the issue] , 2016, Proc. IEEE.

[26]  Fatih Kucuksubasi,et al.  Transfer Learning-Based Crack Detection by Autonomous UAVs , 2018, Proceedings of the 35th International Symposium on Automation and Robotics in Construction (ISARC).

[27]  N. Rajic,et al.  In situ Monitoring of Crack Growth in Mild Steel under Closure Conditions Using a Piezotransducer Array , 2000 .

[28]  Guido Morgenthal,et al.  Quality Assessment of Unmanned Aerial Vehicle (UAV) Based Visual Inspection of Structures , 2014 .

[29]  Pietro Cassarà,et al.  Real-Time Multipath Multimedia Traffic in Cellular Networks for Command and Control Applications , 2019, 2019 IEEE 90th Vehicular Technology Conference (VTC2019-Fall).

[30]  Bin Gao,et al.  Structural Health Monitoring Framework Based on Internet of Things: A Survey , 2017, IEEE Internet of Things Journal.

[31]  Maria Q. Feng,et al.  Citizen Sensors for SHM: Use of Accelerometer Data from Smartphones , 2015, Sensors.

[32]  Rafael Muñoz-Salinas,et al.  Mapping and Localization from Planar Markers , 2016, Pattern Recognit..

[33]  Stefano Chessa,et al.  UAVs and UAV Swarms for Civilian Applications: Communications and Image Processing in the SCIADRO Project , 2017, WISATS.

[34]  Soojin Cho,et al.  Automated Vision-Based Detection of Cracks on Concrete Surfaces Using a Deep Learning Technique , 2018, Sensors.

[35]  Xiaoou Li,et al.  Structural Health Monitoring of Building Structures With Online Data Mining Methods , 2016, IEEE Systems Journal.

[36]  Houbing Song,et al.  Internet of Things and Big Data Analytics for Smart and Connected Communities , 2016, IEEE Access.

[37]  Ivan Roselli,et al.  3D Photogrammetric Reconstruction by Drone Scanning for FE Analysis and Crack Pattern Mapping of the “Bridge of the Towers”, Spoleto , 2017 .

[38]  Lucian Busoniu,et al.  Vision and Control for UAVs: A Survey of General Methods and of Inexpensive Platforms for Infrastructure Inspection , 2015, Sensors.

[39]  Krisada Chaiyasarn,et al.  CRACK DETECTION IN HISTORICAL STRUCTURES BASED ON CONVOLUTIONAL NEURAL NETWORK , 2018, International Journal of GEOMATE.

[40]  Maria Girardi,et al.  The NOSA-ITACA code for the safety assessment of ancient constructions: A case study in Livorno , 2015, Adv. Eng. Softw..

[41]  Liyan Zhang,et al.  Application of Internet of Things Technology and Convolutional Neural Network Model in Bridge Crack Detection , 2018, IEEE Access.

[42]  Mario Marchese,et al.  IoT Applications and Services in Space Information Networks , 2019, IEEE Wireless Communications.

[43]  Ovidio Salvetti,et al.  Marine virtual environment to access underwater cultural heritage , 2016, 2016 International Workshop on Computational Intelligence for Multimedia Understanding (IWCIM).

[44]  Sung-Han Sim,et al.  Concrete Crack Identification Using a UAV Incorporating Hybrid Image Processing , 2017, Sensors.

[45]  Amin Hammad,et al.  High Level Framework for Bridge Inspection Using LiDAR-Equipped UAV , 2017 .

[46]  Eduardo Júlio,et al.  Automatic crack monitoring using photogrammetry and image processing , 2013 .