Vision-based nonintrusive context documentation for earthmoving productivity simulation

Abstract Although video surveillance systems have shown potential for analyzing jobsite contexts, the necessity of a complex multi-camera surveillance system or workers' privacy issues remain as substantive hurdles to adopt such systems in practice. To address such issues, this study presents a non-intrusive earthmoving productivity analysis method using imaging and simulation. The site access log of dump trucks is used to infer earthmoving contexts, which is produced by analyzing videos recorded at the entrance and the exit of a construction site. An algorithm for license plate detection and recognition in an uncontrolled environment is developed to automatically produce the site access log, by leveraging video deinterlacing, a deep convolutional network, and rule-based post-processing. The experimental results show the effectiveness of the proposed method for producing the site access log. Based on the site access log, simulation-based productivity analysis is conducted to produce a daily productivity report, which can provide the basis for earthmoving resource planning. It is expected that the resulting daily productivity report promotes data-driven decision-making for earthmoving resource allocation, thereby improving potential for saving cost and time for earthworks with an updated resource allocation plan.