Quantifying what goes unseen in instrumented and autonomous vehicle perception sensor data – A case study

Abstract It is becoming common to use instrumented probe vehicles (IPV) equipped with perception sensors (e.g., LIDAR or radar) to monitor ambient traffic for empirical studies of traffic dynamics and driver behavior, while the rapidly advancing field of autonomous vehicles (AV) uses perception sensors to identify and mitigate hazards. Yet the perception sensors are far from perfect, so targets of interest go undetected. The “unseen” targets might lead to inaccurate theories of traffic flow or worse, autonomous vehicle crashes. The challenge with the unseen targets is that it is difficult to detect the absence. There is little study of how vehicle-mounted perception sensors perform “in the wild” because often the perception sensors themselves usually provide the best measurements available. The focus of this work is IPV data for offline study, but the general findings are also relevant for perception sensors used in AV applications. While there is a growing number of IPV data sets, to date most suffer from the lack of independent validation of the perception sensors. This paper fuses individual vehicle actuations from loop detectors and concurrent ambient vehicle trajectories collected from an IPV to provide a rare opportunity to assess the perception sensor performance in situ, using the loop detector data to “see” what is missed by the perception sensors. While the specific results are unique to the IPV, one should expect similar biases from other perception sensor data; though without an independent measure like the loop detectors in this study, it will be difficult to quantify what goes unseen in those data sets. These findings also highlight the need to develop means to actively calibrate mobile perception sensors for AV's. It is important to capture the behavior of these systems in situ, as maintained by private owners, though we envision in the case of an AV the role of the loop detectors might be replaced with localization information from other AV's. The data fusion has added benefits for the study of traffic dynamics, with the loop detectors providing continuous monitoring at the detector locations and the IPV providing spatially rich information about the ambient traffic as it travels between detector stations. This fused data set will be available upon publication.