OLIMP: A Heterogeneous Multimodal Dataset for Advanced Environment Perception

A reliable environment perception is a crucial task for autonomous driving, especially in dense traffic areas. Recent improvements and breakthroughs in scene understanding for intelligent transportation systems are mainly based on deep learning and the fusion of different modalities. In this context, we introduce OLIMP: A heterOgeneous Multimodal Dataset for Advanced EnvIronMent Perception. This is the first public, multimodal and synchronized dataset that includes UWB radar data, acoustic data, narrow-band radar data and images. OLIMP comprises 407 scenes and 47,354 synchronized frames, presenting four categories: pedestrian, cyclist, car and tram. The dataset includes various challenges related to dense urban traffic such as cluttered environment and different weather conditions. To demonstrate the usefulness of the introduced dataset, we propose a fusion framework that combines the four modalities for multi object detection. The obtained results are promising and spur for future research.

[1]  Imen Jegham,et al.  MDAD: A Multimodal and Multiview in-Vehicle Driver Action Dataset , 2019, CAIP.

[2]  Ruigang Yang,et al.  The ApolloScape Open Dataset for Autonomous Driving and Its Application , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Jianru Xue,et al.  A Survey of Scene Understanding by Event Reasoning in Autonomous Driving , 2018, Int. J. Autom. Comput..

[4]  David Vázquez,et al.  On-Board Object Detection: Multicue, Multimodal, and Multiview Random Forest of Local Experts , 2017, IEEE Transactions on Cybernetics.

[5]  Gaël Richard,et al.  Acoustic Features for Environmental Sound Analysis , 2018 .

[6]  Hang Yin,et al.  Test Your Self-Driving Algorithm: An Overview of Publicly Available Driving Datasets and Virtual Testing Environments , 2019, IEEE Transactions on Intelligent Vehicles.

[7]  Nacer K. M'Sirdi,et al.  Perception and characterization of materials using signal processing techniques , 2001, IEEE Trans. Instrum. Meas..

[8]  In So Kweon,et al.  KAIST Multi-Spectral Day/Night Data Set for Autonomous and Assisted Driving , 2018, IEEE Transactions on Intelligent Transportation Systems.

[9]  Michael Ying Yang,et al.  Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection , 2018, Inf. Fusion.

[10]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Klaus C. J. Dietmayer,et al.  Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges , 2019, IEEE Transactions on Intelligent Transportation Systems.

[12]  Bryan Reimer,et al.  MIT Advanced Vehicle Technology Study: Large-Scale Naturalistic Driving Study of Driver Behavior and Interaction With Automation , 2017, IEEE Access.

[13]  Imen Jegham,et al.  Vision-based human action recognition: An overview and real world challenges , 2020, Digit. Investig..

[14]  Najoua Essoukri Ben Amara,et al.  Pedestrian detection using a moving camera: A novel framework for foreground detection , 2020, Cognitive Systems Research.

[15]  Chengyang Li,et al.  Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detection , 2018, Pattern Recognit..

[16]  Mohak Shah,et al.  Is it Safe to Drive? An Overview of Factors, Challenges, and Datasets for Driveability Assessment in Autonomous Driving , 2018, ArXiv.

[17]  Paul Newman,et al.  1 year, 1000 km: The Oxford RobotCar dataset , 2017, Int. J. Robotics Res..

[18]  Hang-Bong Kang,et al.  Object Detection and Classification by Decision-Level Fusion for Intelligent Vehicle Systems , 2017, Sensors.

[19]  Hyeonjoon Moon,et al.  A Survey on Internet of Things and Cloud Computing for Healthcare , 2019, Electronics.

[20]  Abdelmalik Taleb-Ahmed,et al.  Medical Video Coding Based on 2nd-Generation Wavelets: Performance Evaluation , 2019, Electronics.

[21]  Omar Y. Al-Jarrah,et al.  A Survey on 3D Object Detection Methods for Autonomous Driving Applications , 2019, IEEE Transactions on Intelligent Transportation Systems.

[22]  Nanning Zheng,et al.  On-Road Vehicle Detection and Tracking Using MMW Radar and Monovision Fusion , 2016, IEEE Transactions on Intelligent Transportation Systems.