Adaptive Automatic Ground Truth Generation for Testing of Vehicle Detectors

Field testing of vehicle presence detectors using non-aggregated data and metrics requires the generation of an accurate ground-truth record including the time of arrival of every individual vehicle at a roadway test site. Performance results for detectors under test are assessed by comparison with the ground truth record. Ground truth has traditionally been determined either by the use of a trusted reference detector, or by human observation, often using time-coded recorded video. But for meaningful multiple detector testing over statistically significant time periods, the volume of required observations becomes unwieldy, defying accurate human verification. Further, no detector can truly be considered “trusted” for use as an absolute ground truth reference – all detectors have some limitations or potential sources of error. This paper describes a self-optimizing method by which a weighted consensus of all detectors under test is used to automatically generate a reliable ground truth record against which all detectors are compared. The method employs a discrete adaptive (learning) algorithm that continuously adjusts a ‘reliability coefficient’ for each detector under test during successive samples to assure the most accurate possible consensus result. Limitations and convergence issues are discussed. The algorithm is currently in experimental use in the California Department of Transportation ATMS Detector Test Bed, in which multiple detectors of various technologies are being tested concurrently for six lanes of high-volume traffic.