A Fusion Approach for Pre-Crash Scenarios based on Lidar and Camera Sensors

The use and development of new Advanced Driver Assistance Systems (ADAS) based on fusion of various sensors, aiming to achieve a higher robustness, are becoming more common. In pre-crash scenarios, the extraction of information from an incoming vehicle should be done as near as possible to the moment of impact "t0", where relevant issues such as disturbances in the sensor signal and partial occlusion of the vehicle play an important role. The presented work introduces a novel fusion approach between a Lidar and a camera to extract the distance and the approach angle of an oncoming vehicle. It detects the bullet vehicle’s license plate using a deep learning neural network and fuses the Lidar information to estimate its distance and approach angle. The proposed architecture was evaluated on data from a benchmark dataset - KITTI, and with a pre-crash scenario dataset collected inside the CARISSMA Research Center indoor hall facilities. The algorithm reached an accuracy of 0.16m±0.26m for the distance and 5.48°±4.99° for the angle estimation on the benchmark dataset. On the pre-crash dataset, the experimental results yielded an accuracy of 0.022m ± 0.022m and 2.82°± 1.98° for the distance and angle, respectively. The system can extract parameters until 0.4m before a collision, and can overcome blurriness and partial occlusion conditions, being also easily reproducible in other sensor setups.