Warp-centric K-Nearest Neighbor Graphs construction on GPU

Recent advances and applications of machine learning algorithms are becoming more common in different fields. It is expected that some applications require the processing of large datasets with those algorithms, which leads to high computational costs. Massively parallel GPU methods can be applied to surpass this limitation and reduce the execution time of these algorithms. The construction of approximate K-Nearest Neighbor Graphs (K-NNG) is frequently required for similarity search or other applications such as the t-SNE dimensionality reduction technique. The K-NNG represents the K closest points (neighbors) for each point in a set. In this paper, we propose and analyze an all-points K-Nearest Neighbor Graph construction algorithm on GPU called Warp-centric K-NNG (w-KNNG), which is based on the Random Projection Forest method. Usually, the construction or search for k-NN sets for high dimensional points presents challenges for its implementation on many-core processing units, due to the space limitation in maintaining these sets in high speed shared memory. We present three warp-centric approaches for our algorithm that efficiently search and maintain the k-NN high dimensional point sets in global memory. In our experiments, the new methods allows the algorithm to achieve up to 639% faster execution when compared to the state-of-the-art FAISS library, considering an equivalent accuracy of approximate K-NNG. One of the new strategies (w-KNNG atomic) is more successful when applied to a smaller number of dimensions, while the tiled w-KNNG approach was successful in general scenarios for higher dimensional points.