Loop closure detection using CNN words

Loop closure detection (LCD) is crucial for the simultaneous localization and mapping system of an autonomous robot. Image features from a convolution neural network (CNN) have been widely used for LCD in recent years. Instead of directly using the feature vectors to compute the image similarity, we propose a novel and easy-to-implement method that manages features from a CNN via a novel approach to improve the performance. In this method, the elements of feature maps from the higher layer of the CNN are clustered to generate CNN words (CNNW). To encode spatial information of CNNW, we create word pairs (CNNWP) that are based on single words to improve the performance. In addition, traditional tricks that are used in methods that are based on bag of words (BoW) are integrated into our approach. We also demonstrate that the feature maps from lower layers can be used as descriptors to conduct local region matching between images. Via this approach, we can perform geometric verification for possible loop closures, similar to BoW methods, in our approach. The experimental results demonstrate that our method substantially outperforms state-of-the-art methods that directly use CNN features for LCD.