FPGA-based architecture of a real-time SIFT matcher and RANSAC algorithm for robotic vision applications

A fundamental problem in computer vision is finding correspondences between features in pairs of similar images. By comparing feature descriptors instead of pixel intensities, the matching capability is significantly increased. Keypoints extracted by Scale-Invariant Feature Transform (SIFT) provide superior matching ability, however, a small proportion of false corresponcences is always inevitable. The exemption of false matches is achieved using robust fitting algorithms, with RANSAC (random sample consensus) being a popular one. SIFT and RANSAC are computationally demanding and time consuming algorithms. When the target application operates in real-time, conventional approaches based on personal computers usually fail to meet the requirements. In this paper, an FPGA-based architecture for real-time SIFT matching and RANSAC algorithm is presented. The proposed scheme is applied to identify the correspondences between point features across consecutive video frames and reject the false matches. The architecture is verified using the DE2i-150 development board. Using Cyclone IV technology, the system supports a processing rate of 40fps for VGA resolution and therefore meets real-time requirements.

[1]  R. Hartley,et al.  Multiple View Geometry in Computer Vision: Estimation – 2D Projective Transformations , 2004 .

[2]  Han Xiao,et al.  A fast stereovision measurement algorithm based on SIFT keypoints for mobile robot , 2013, 2013 IEEE International Conference on Mechatronics and Automation.

[3]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[4]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[5]  U. U. Sheikh,et al.  FPGA implementation of RANSAC algorithm for real-time image geometry estimation , 2013, 2013 IEEE Student Conference on Research and Developement.

[6]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[7]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, CVPR 2004.

[8]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[9]  Liu Hu.  Application of Improved SIFT Algorithm on Stitching of UAV Remote Sensing Image , 2014 .

[10]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Yuichiro Shibata,et al.  Deep-pipelined FPGA implementation of ellipse estimation for eye tracking , 2012, 22nd International Conference on Field Programmable Logic and Applications (FPL).

[12]  Mohammed Boulekchour,et al.  Robust L∞ convex optimisation for monocular visual odometry trajectory estimation , 2014, Robotica.

[13]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[14]  Rafal Kapela,et al.  Embedded platform for local image descriptor based object detection , 2015, Appl. Math. Comput..

[15]  John N. Lygouras,et al.  FPGA accelerator for real-time SIFT matching with RANSAC support , 2017, Microprocess. Microsystems.

[16]  Zhiguo Cao,et al.  An Embedded System-on-Chip Architecture for Real-time Visual Detection and Matching , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Miguel A. Vega-Rodríguez,et al.  A Comparative Study of Parallel RANSAC Implementations in 3D Space , 2014, International Journal of Parallel Programming.

[18]  John N. Lygouras,et al.  Fully pipelined FPGA-based architecture for real-time SIFT extraction , 2016, Microprocess. Microsystems.

[19]  Lan-Rong Dung,et al.  Implementation of RANSAC Algorithm for Feature-Based Image Registration , 2013 .

[20]  Li Haiyang,et al.  A Fast Image Matching Algorithm Based on GPU Parallel Computing , 2013 .

[21]  Vincent Lepetit,et al.  BRIEF: Computing a Local Binary Descriptor Very Fast , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Hannes Fassold,et al.  A real-time GPU implementation of the SIFT algorithm for large-scale video analysis tasks , 2015, Electronic Imaging.

[23]  James E. Gentle,et al.  Matrix Transformations and Factorizations , 2017 .

[24]  Stefano Di Carlo,et al.  SA-FEMIP: A Self-Adaptive Features Extractor and Matcher IP-Core Based on Partially Reconfigurable FPGAs for Space Applications , 2015, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[25]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[26]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[27]  John Vourvoulakis,et al.  Acceleration of RANSAC algorithm for images with affine transformation , 2016, 2016 IEEE International Conference on Imaging Systems and Techniques (IST).

[28]  Guangjun Zhang,et al.  SIFT Hardware Implementation for Real-Time Image Feature Extraction , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Mariagiovanna Sami,et al.  An OpenCL-based feature matcher , 2013, Signal Process. Image Commun..