Dynamic Door Modeling for Monocular 3D Vehicle Detection

The precise 3D localization of non-ego vehicles is a crucial task for the long-term goal of autonomous driving. In urban scenarios, where pedestrians frequently interact with vehicles, this task also requires a precise modeling of dynamic vehicle parts, e.g., doors. Current state-of-the-art computer vision algorithms are in fact able to estimate a vehicle pose but do not model doors by any means. To provide a solution solely based on a monocular camera, our proposed pipeline first performs a six degree-of-freedom pose estimation and then predicts the respective states of the vehicle doors. For both problems we utilize a perspective-n-point fitting method based on key points. To this end, we jointly detect the two required sets of correspondences for the vehicle body and the doors with a neural network. Since little insight is published for the application of key point based vehicle detection in the literature, we compare different implementations of the key point prediction module and investigate algorithm details, i.e., the role of a key point visibility analysis and two differing key point layouts. Results for the body estimation and the door detection with respect to these implementation details are presented on a proprietary dataset, in which we utilize an exact vehicle model to receive precise ground truth.