A novel fusing semantic- and appearance-based descriptors for visual loop closure detection

Abstract Loop-closure detection plays an important role in visual simultaneous localisation and mapping;it is an independent part of the visual odometer and can effectively reduce its accumulated error, in addition to helping with loop-closure detection for relocalisation. With the development of deep learning methods in recent years, the training models of convolutional neural networks for major data sets have been improved for loop-closure detection. Presently, some high-level engineering problems still rely on auxiliary equipment, such as panoramic cameras and radar lasers, which greatly increase the expensive extra cost; however, owing to the extreme appearance and viewpoint changes involved in such problems, loop-closure detection that relies on two-dimensional images is not applicable. Based on the two nearest neighbour vector of locally aggregated descriptors (TNNVLAD) method, a novel feature descriptor called two nearest neighbour local sensor tensor(TNNLoST) is proposed herein by combining the semantic features of high-level neural networks with dense descriptors. This approach introduces a semantic concept similar to human cognition for the surrounding environment, thus enabling better understanding of the environment. The proposed method was applied to publicly available benchmark datasets to show its performance.