Integration of CNN into a Robotic Architecture to Build Semantic Maps of Indoor Environments

In robotics, semantic mapping refers to the construction of a rich representation of the environment that includes high level information needed by the robot to accomplish its tasks. Building a semantic map requires algorithms to process sensor data at different levels: geometric, topological and object detections/categories, which must be integrated into an unified model. This paper describes a robotic architecture that successfully builds such semantic maps for indoor environments. For this purpose, within a ROS-based ecosystem, we apply a state-of-the-art Convolutional Neural Network (CNN), concretely YOLOv3, for detecting objects in images. The detection results are placed within a geometric map of the environment making use of a number of modules of the architecture: robot localization, camera extrinsic calibration, data form a depth camera, etc. We demonstrate the suitability of the proposed framework by building semantic maps of several home environments from the Robot@Home dataset, using Unity 3D as a tool to visualize the maps as well as to provide future robotic developments.