Detecting and mapping traffic signs from Google Street View images using deep learning and GIS

Abstract Street traffic sign infrastructure remains an extremely difficult asset for local government to manage due to its diverse physical structure and geographical distribution. A spatial registrar of traffic infrastructure is currently a required component of local government councils' mandatory road management plans. Recent advancements of object detection technology in machine learning have presented an automated approach for the detection and classification of street signage captured by Google's Street View (GSV) imagery. This paper explores the possibility of using deep learning to produce an autonomous system for detecting traffic signs on GSV images to assist in traffic assets monitoring and maintenance. By leveraging Google's Street View API, this research offers an economic approach of building purposeful street sign computer vision datasets. A custom object detection model was trained to detect and classify Stop and Give Way signs from images captured at intersection approaches. Considering the output detected bounding box coordinates, photogrammetry approach was applied to calculate the approximate location of each detected sign in two-dimensional geographical space. The newly located and classified street signs can be combined with relevant spatial data for implementation into an asset management system. By combining GIS and the GSV API, the process is completely scalable to any level of street sign classification scope. The experiments conducted on the road network of study area recorded a detection accuracy of 95.63% and classification accuracy of 97.82%. Our proposed automated approach to the detection and localisation of street sign infrastructure has displayed a promising potential for its use by local government authorities. Our workflow can be used to detect other traffic signs and applied to other road sections and other cities. Of primary importance, this approach takes an entirely free and open-source approach throughout. The continuation of Google's Street View program will account for the spatiotemporal representation of street sign infrastructure for the ongoing maintenance and renewal programs of this valuable asset.