Deep learning-based sewer defect classification for highly imbalanced dataset

Abstract Sanitary sewer systems play a fundamental role in protecting water quality and the public well-being. Structural, civil, and functional operations of any sewer network can deteriorate at accelerated levels due to harsh environments inside the sewer pipes. The existing maintenance procedures are usually deemed inefficient in terms of the assessment accuracy, reliability, safety, and the cost due to the difficulty of detecting and diagnosing defects inside the sewer network. As a result, this paper proposes a robust and efficient deep learning-based framework that can detect and evaluate the defects automatically with high accuracy. The main contributions of the work include (1) a fine-tuned deep learning-based sewer defect detection framework that is based on the block-based architecture, which contains a series of convolutional layers that can efficiently extract the abstract features from the defective regions, (2) hybrid extensions of the proposed model that apply the ensemble-based approach and the cost-sensitive learning-based method in order to cope with the imbalanced data problem (IDP) efficiently, and (3) a novel frame reduction algorithm that is based on analyzing the contextual information of the closed-circuit television (CCTV) videos. The experimental results indicated that the proposed framework obtained a state-of-the-art performance compared to the previous sewer defect detection systems, and it was robust against the IDP. The benefits of the proposed defect detection framework are that it motivates more efficient defect analysis algorithms and promotes a complete integration of deep learning-based approaches in real-world sewer defect analysis applications.