Deep learning-based automatic volumetric damage quantification using depth camera

Abstract A depth camera or 3-dimensional scanner was used as a sensor for traditional methods to quantify the identified concrete spalling damage in terms of volume. However, to quantify the concrete spalling damage automatically, the first step is to detect (i.e., identify) the concrete spalling. The multiple spots of spalling can be possible within a single structural element or in multiple structural elements. However, there is, as of yet, no method to detect concrete spalling automatically using deep learning methods. Therefore, in this paper, a faster region-based convolutional neural network (Faster R-CNN)-based concrete spalling damage detection method is proposed with an inexpensive depth sensor to quantify multiple instances of spalling simultaneously in the same surface separately and consider multiple surfaces in structural elements. A database composed of 1091 images (with 853 × 1440 pixels) labeled for volumetric damage is developed, and the deep learning network is then modified, trained, and validated using the proposed database. The damage quantification is automatically performed by processing the depth data, identifying surfaces, and isolating the damage after merging the output from the Faster R-CNN with the depth stream of the sensor. The trained Faster R-CNN presented an average precision (AP) of 90.79%. Volume quantifications show a mean precision error (MPE) of 9.45% when considering distances from 100 cm to 250 cm between the element and the sensor. Also, an MPE of 3.24% was obtained for maximum damage depth measurements across the same distance range.