The UMA-SAR Dataset: Multimodal data collection from a ground vehicle during outdoor disaster response training exercises

This article presents a collection of multimodal raw data captured from a manned all-terrain vehicle in the course of two realistic outdoor search and rescue (SAR) exercises for actual emergency responders conducted in Málaga (Spain) in 2018 and 2019: the UMA-SAR dataset. The sensor suite, applicable to unmanned ground vehicles (UGVs), consisted of overlapping visible light (RGB) and thermal infrared (TIR) forward-looking monocular cameras, a Velodyne HDL-32 three-dimensional (3D) lidar, as well as an inertial measurement unit (IMU) and two global positioning system (GPS) receivers as ground truth. Our mission was to collect a wide range of data from the SAR domain, including persons, vehicles, debris, and SAR activity on unstructured terrain. In particular, four data sequences were collected following closed-loop routes during the exercises, with a total path length of 5.2 km and a total time of 77 min. In addition, we provide three more sequences of the empty site for comparison purposes (an extra 4.9 km and 46 min). Furthermore, the data is offered both in human-readable format and as rosbag files, and two specific software tools are provided for extracting and adapting this dataset to the users’ preference. The review of previously published disaster robotics repositories indicates that this dataset can contribute to fill a gap regarding visual and thermal datasets and can serve as a research tool for cross-cutting areas such as multispectral image fusion, machine learning for scene understanding, person and object detection, and localization and mapping in unstructured environments. The full dataset is publicly available at: www.uma.es/robotics-and-mechatronics/sar-datasets.

[1]  Wolfram Burgard,et al.  Efficient and robust deep networks for semantic segmentation , 2018, Int. J. Robotics Res..

[2]  Robin R. Murphy,et al.  Disaster Robotics , 2014, Springer Handbook of Robotics, 2nd Ed..

[3]  Solvi Arnold,et al.  An image recognition system aimed at search activities using cyber search and rescue dogs , 2018, J. Field Robotics.

[4]  Sven Behnke,et al.  Continuous mapping and localization for autonomous navigation in rough terrain using a 3D laser scanner , 2017, Robotics Auton. Syst..

[5]  Robin R. Murphy,et al.  Autonomous Capabilities for Small Unmanned Aerial Systems Conducting Radiological Response: Findings from a High‐fidelity Discovery Experiment , 2014, J. Field Robotics.

[6]  Renaud Dubé,et al.  3D registration of aerial and ground robots for disaster response: An evaluation of features, descriptors, and transformation estimation , 2017, 2017 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR).

[7]  Alex Bewley,et al.  Large-scale outdoor scene reconstruction and correction with vision , 2020, Int. J. Robotics Res..

[8]  Cyrill Stachniss,et al.  Bonnet: An Open-Source Training and Deployment Framework for Semantic Segmentation in Robotics using CNNs , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[9]  Ricardo Vázquez-Martín,et al.  Object Detection from Thermal Infrared and Visible Light Cameras in Search and Rescue Scenes , 2020, 2020 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[10]  Martial Hebert,et al.  Learning Shape-based Representation for Visual Localization in Extremely Changing Conditions , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Vipul Mishra,et al.  Drone-surveillance for search and rescue in natural disaster , 2020, Comput. Commun..

[12]  Gerald Steinbauer,et al.  The RoboCup Rescue Victim Dataset , 2018, 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[13]  Michal Reinstein,et al.  Tracked Robot Odometry for Obstacle Traversal in Sensory Deprived Environment , 2019, IEEE/ASME Transactions on Mechatronics.

[14]  A. Birk,et al.  3D data collection at Disaster City at the 2008 NIST Response Robot Evaluation Exercise (RREE) , 2009, 2009 IEEE International Workshop on Safety, Security & Rescue Robotics (SSRR 2009).

[15]  John J. Leonard,et al.  Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age , 2016, IEEE Transactions on Robotics.

[16]  Roland Siegwart,et al.  Integrated Data Management for a Fleet of Search‐and‐rescue Robots , 2017, J. Field Robotics.

[17]  Raúl Páez,et al.  The RAP Net: A Geodetic Positioning Network for Andalusia (South Spain) , 2006 .

[18]  Dietrich Paulus,et al.  Real-time 3D mapping of rough terrain: A field report from Disaster City , 2010, 2010 IEEE Safety Security and Rescue Robotics.

[19]  Karel Zimmermann,et al.  Simultaneous exploration and segmentation for search and rescue , 2018, J. Field Robotics.

[20]  Paul Newman,et al.  1 year, 1000 km: The Oxford RobotCar dataset , 2017, Int. J. Robotics Res..

[21]  In So Kweon,et al.  KAIST Multi-Spectral Day/Night Data Set for Autonomous and Assisted Driving , 2018, IEEE Transactions on Intelligent Transportation Systems.

[22]  Alexander Ferrein,et al.  Evaluation of sensors and mapping approaches for disasters in tunnels , 2013, 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[23]  Marina Ivasic-Kos,et al.  Thermal Imaging Dataset for Person Detection , 2019, 2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[24]  Renaud Dubé,et al.  An online multi-robot SLAM system for 3D LiDARs , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Alexander Ferrein,et al.  Evaluation of Sensors and Mapping Approaches for Disasters in Tunnels , 2016, J. Field Robotics.

[26]  In So Kweon,et al.  DISC: A Large-scale Virtual Dataset for Simulating Disaster Scenarios , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[27]  Satish Kumar,et al.  Data-Driven Skin Detection in Cluttered Search and Rescue Environments , 2020, IEEE Sensors Journal.

[28]  Guangming Xiong,et al.  Real-Time 6D Lidar SLAM in Large Scale Natural Terrains for UGV , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[29]  Zhenxue Chen,et al.  Real-time Semantic Segmentation for Road Scene , 2018, 2018 3rd International Conference on Advanced Robotics and Mechatronics (ICARM).

[30]  Robin R. Murphy,et al.  Use of a Small Unmanned Aerial System for the SR‐530 Mudslide Incident near Oso, Washington , 2016, J. Field Robotics.

[31]  Keiji Nagatani,et al.  Trials of 3-D map construction using the tele-operated tracked vehicle kenaf at disaster city , 2010, 2010 IEEE International Conference on Robotics and Automation.

[32]  Ayoung Kim,et al.  Sparse Depth Enhanced Direct Thermal-Infrared SLAM Beyond the Visible Spectrum , 2019, IEEE Robotics and Automation Letters.

[33]  Andreas Birk,et al.  Online three‐dimensional SLAM by registration of large planar surface segments and closed‐form pose‐graph relaxation , 2010, J. Field Robotics.

[34]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Shaowu Peng,et al.  Benchmarking a large-scale FIR dataset for on-road pedestrian detection , 2019, Infrared Physics & Technology.

[36]  Hannes Sommer,et al.  SegMap: Segment-based mapping and localization using data-driven descriptors , 2019, Int. J. Robotics Res..

[37]  Roland Siegwart,et al.  The current state and future outlook of rescue robotics , 2019, J. Field Robotics.

[38]  Michael Bosse,et al.  Large-scale, real-time visual–inertial localization revisited , 2019, Int. J. Robotics Res..