Content-based medical image retrieval by spatial matching of visual words

Abstract Content-Based Image Retrieval (CBIR) systems have recently emerged as one of the most promising and best image retrieval paradigms. To pacify the semantic gap associated with CBIR systems, the Bag of Visual Words (BoVW) techniques are now increasingly used. However, existing BoVW techniques fail to capture the location information of visual words effectively. This paper proposes an unsupervised Content-Based Medical Image Retrieval (CBMIR) framework based on the spatial matching of the visual words. The proposed method efficiently computes the spatial similarity of visual words using a novel similarity measure called the Skip Similarity Index. Experiments on three large medical datasets reveal promising results. The location-based correlation of visual words assists in more accurate and efficient retrieval of anatomically diverse and multimodal medical images than the state-of-the-art CBMIR systems.