A survey of automation-enabled human-in-the-loop systems for infrastructure visual inspection

Abstract Routine inspection and maintenance are critical for the proper functioning of civil infrastructures such as bridges, pavements and underground structures. Civil infrastructures are being inspected less frequently because of the high cost and long duration of current inspection procedures. Furthermore, conventional inspection procedures often interrupt the routine functioning of the infrastructure, are inspector-dependent and expose the inspectors to complex and unsafe working environments. Visual inspection technologies play a crucial role in the inspection and maintenance of civil infrastructures. Automation-assisted technologies such as drones and underwater vehicles equipped with multiple imaging and sensing systems have been developed to address some of these issues with the conventional visual inspection processes. This paper reviews peer-reviewed research publications investigating automated visual inspection technologies following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Specifically, 53 publications satisfying a set of inclusion criteria were reviewed, its results highlighting the application domain, the level of autonomy of the automated systems, the sensor technologies used for the inspection process and navigation, the navigation and control technologies and the algorithms used. The review of the articles revealed that the data collected by automation is used to augment the qualitative assessment. Several types of algorithms such as target detection and image enhancing have been developed to reduce the inspector bias in these automated technologies. Path planning algorithms reduce the workload on the inspector by automating the navigation and control tasks. Remotely operated systems reduce the risk to the inspectors by minimizing their exposure to the inspection environment. However, only a limited number of studies investigated the human factors aspects of the automation-assisted inspection process. It is important to understand the cognitive, physical, and temporal demands these technologies place on inspectors to improve the design of systems assisting in the inspection process. Moreover, factors such as automation bias, trust in the system and communication between the automation and the operator need to be investigated. Furthermore, it is important to incorporate appropriate decision aids that support adequate situation awareness in the interface design. Based on these findings this review proposes directions for future research. This review concludes by highlighting the need for human-centered research to develop better solutions for infrastructure inspection problems.