A survey of performance measures to evaluate ego-lane estimation and a novel sensor-independent measure along with its applications

Lane estimation plays a central role for Driver Assistance Systems, therefore many approaches have been proposed to measure its performance. However, no commonly agreed metric exists. In this work, we first present a detailed survey of the current measures. Most of them apply pixel-level benchmarks on camera images and require a time-consuming and fault-prone labeling process. Moreover, these metrics cannot be used to assess other sources such as the detected guardrails, curbs or other vehicles. Therefore, we introduce an efficient and sensor-independent metric, which provides an objective and intuitive self-assessment for the entire road estimation process at multiple levels: individual detectors, lane estimation itself, and the target applications (e.g., lane keeping system). Our metric does not require a high labeling effort and can be used both online and offline. By selecting the evaluated points in specific distances, it can be applied to any road model representation. By comparing in 2D vehicle coordinate system, two possibilities exist to generate the ground-truth: the human-driven path or the expensive alternative with DGPS and detailed maps. This paper applies both methods and reveals that the human-driven path also qualifies for this task and it is applicable to scenarios without GPS signal, e.g., tunnel. Although the lateral offset between reference and detection is widely used in the majority of works, this paper shows that another criterion, the angle deviation, is more appropriate. Finally, we compare our metric with other state-of-the-art metrics using real data recordings from different scenarios.