A hardware accelerated Scale Invariant Feature detector for real-time visual localization and mapping

Scale Invariant Feature Transform (SIFT) has drawn attention in the field of computer vision, recently. SIFT has been adopted in many visual localization and mapping applications, for its robustness to scale, rotation and illumination changes. However, the high computational cost limits its use in practical scenarios. In this paper, we present a real-time FPGA-based hardware accelerator of SIFT. The design is composed of two main parts: key-point detection component and feature generation component. The key-point detection component applies an octave-interleaved scale-parallel pipeline structure, as a tradeoff between frame rate and resource consumption. The feature generation component works in task-level burst mode for each key-point. The buffer together with buffer management logic enables quasi-parallelism between the two components, and also enables task-level quasi-parallelism between main orientation generation and local descriptor generation in the feature generation component. Our proposal can perform feature extraction of 720p video with real-time efficiency of 42fps at a clock frequency of 100MHz.