Early Recall, Late Precision: Multi-Robot Semantic Object Mapping under Operational Constraints in Perceptually-Degraded Environments

Semantic object mapping in uncertain, perceptually degraded environments during long-range multi-robot autonomous exploration tasks such as search-and-rescue is important and challenging. During such missions, high recall is desirable to avoid missing true target objects and high precision is also critical to avoid wasting valuable operational time on false positives. Given recent advancements in visual perception algorithms, the former is largely solvable autonomously, but the latter is difficult to address without the supervision of a human operator. However, operational constraints such as mission time, computational requirements and mesh network bandwidth can make the operator's task infeasible unless properly managed. We propose the Early Recall, Late Precision (EaRLaP) semantic object mapping pipeline to solve this problem. EaRLaP was used by Team CoSTAR in DARPA Subterranean Challenge, where it successfully detected all the artifacts encountered by the team of robots. We will discuss these results and the performance of the EaRLaP on various datasets.

[1]  Jay L. Gao,et al.  NeBula: TEAM CoSTAR's Robotic Autonomy Solution that Won Phase II of DARPA Subterranean Challenge , 2022, Field Robotics.

[2]  Roland Siegwart,et al.  CERBERUS: Autonomous Legged and Aerial Robotic Exploration in the Tunnel and Urban Circuits of the DARPA Subterranean Challenge , 2022, Field Robotics.

[3]  Eric W. Frew,et al.  Multi-Agent Autonomy: Advancements and Challenges in Subterranean Exploration , 2021, Field Robotics.

[4]  Katrina Lo Surdo,et al.  Heterogeneous Ground and Air Platforms, Homogeneous Sensing: Team CSIRO Data61's Approach to the DARPA Subterranean Challenge , 2021, Field Robotics.

[5]  Geoffrey A. Hollinger,et al.  Resilient and Modular Subterranean Exploration with a Team of Roving and Flying Robots , 2022 .

[6]  S. Bhattacharyya,et al.  Plug-and-Play Deblurring for Robust Object Detection , 2021, 2021 International Conference on Visual Communications and Image Processing (VCIP).

[7]  Geoffrey A. Hollinger,et al.  Adversarial Training on Point Clouds for Sim-to-Real 3D Object Detection , 2021, IEEE Robotics and Automation Letters.

[8]  Jay L. Gao,et al.  NeBula: Quest for Robotic Autonomy in Challenging Environments; TEAM CoSTAR at the DARPA Subterranean Challenge , 2021, ArXiv.

[9]  Amanda Bouman,et al.  Autonomous Spot: Long-Range Autonomous Exploration of Extreme Environments with Legged Locomotion , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Eric Heiden,et al.  LAMP: Large-Scale Autonomous Mapping and Positioning for Exploration of Perceptually-Degraded Subterranean Environments , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Christos Papachristos,et al.  Multi-modal Visual-Thermal Saliency-based Object Detection in Visually-degraded Environments , 2020, 2020 IEEE Aerospace Conference.

[12]  Abel Gawel,et al.  Learning Densities in Feature Space for Reliable Segmentation of Indoor Scenes , 2019, IEEE Robotics and Automation Letters.

[13]  Quoc V. Le,et al.  Learning Data Augmentation Strategies for Object Detection , 2019, ECCV.

[14]  Shreyansh Daftry,et al.  Object and Gas Source Detection with Robotic Platforms in Perceptually-Degraded Environments , 2020 .

[15]  Martin Saska,et al.  DARPA Subterranean Challenge: Multi-robotic Exploration of Underground Environments , 2019, MESAS.

[16]  Jeremy Kepner,et al.  Survey and Benchmarking of Machine Learning Accelerators , 2019, 2019 IEEE High Performance Extreme Computing Conference (HPEC).

[17]  Arash Vahdat,et al.  A Robust Learning Approach to Domain Adaptive Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[18]  Gabriel J. Brostow,et al.  Digging Into Self-Supervised Monocular Depth Estimation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[20]  Lars Petersson,et al.  DeNet: Scalable Real-Time Object Detection with Directed Sparse Sampling , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  G. Raj,et al.  Blur image detection using Laplacian operator and Open-CV , 2016, 2016 International Conference System Modeling & Advancement in Research Trends (SMART).

[22]  F. Dellaert Factor Graphs and GTSAM: A Hands-on Introduction , 2012 .

[23]  William Whittaker,et al.  Robotic introspection for exploration and mapping of subterranean environments , 2007 .

[24]  Fredric C. Gey,et al.  The Relationship between Recall and Precision , 1994, J. Am. Soc. Inf. Sci..