FPGA Design of Enhanced Scale-Invariant Feature Transform with Finite-Area Parallel Feature Matching for Stereo Vision

In this paper, we propose an FPGA-based enhanced-SIFT with feature matching for stereo vision. Gaussian blur and difference of Gaussian pyramids are realized in parallel to accelerate the processing time required for multiple convolutions. As for the feature descriptor, a simple triangular identification approach with a look-up table is proposed to efficiently determine the direction and gradient of the feature points. Thus, the dimension of the feature descriptor in this paper is reduced by half compared to conventional approaches. As far as feature detection is concerned, the condition for high-contrast detection is simplified by moderately changing a threshold value, which also benefits the reduction of the resulting hardware in realization. The proposed enhanced-SIFT not only accelerates the operational speed but also reduces the hardware cost. The experiment results show that the proposed enhanced-SIFT reaches a frame rate of 205 fps for 640 × 480 images. Integrated with two enhanced-SIFT, a finite-area parallel checking is also proposed without the aid of external memory to improve the efficiency of feature matching. The resulting frame rate by the proposed stereo vision matching can be as high as 181 fps with good matching accuracy as demonstrated in the experimental results.

[1]  Yung-Chang Chen,et al.  High-Performance SIFT Hardware Accelerator for Real-Time Image Feature Extraction , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Massimo Bertozzi,et al.  GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection , 1998, IEEE Trans. Image Process..

[3]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[4]  John N. Lygouras,et al.  FPGA accelerator for real-time SIFT matching with RANSAC support , 2017, Microprocess. Microsystems.

[5]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[6]  Luca Di Nunzio,et al.  Energy Consumption Saving in Embedded Microprocessors Using Hardware Accelerators , 2018 .

[7]  John N. Lygouras,et al.  A complete processor for SIFT feature matching in video sequences , 2017, 2017 9th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS).

[8]  Chenyang Li,et al.  Ground Control Point Automatic Extraction for Spaceborne Georeferencing Based on FPGA , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[9]  Cheng-Hung Lin,et al.  Heterogeneous Implementation of a Novel Indirect Visual Odometry System , 2019, IEEE Access.

[10]  Dongxiang Zhou,et al.  Robust Visual Compass Using Hybrid Features for Indoor Environments , 2019, Electronics.

[11]  Jacky Baltes,et al.  Adaptive computational SLAM incorporating strategies of exploration and path planning , 2019, Knowl. Eng. Rev..

[12]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Hyuk-Jae Lee,et al.  A Hardware Architecture for the Affine-Invariant Extension of SIFT , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Chiang-Ju Chien,et al.  Hardware-Software Co-Design of an Image Feature Extraction and Matching Algorithm , 2019, 2019 2nd International Conference on Intelligent Autonomous Systems (ICoIAS).

[15]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[16]  Alan D. George,et al.  A power-efficient real-time architecture for SURF feature extraction , 2014, 2014 International Conference on ReConFigurable Computing and FPGAs (ReConFig14).

[17]  Wei-Yen Wang,et al.  Indirect Visual Simultaneous Localization and Mapping Based on Linear Models , 2020, IEEE Sensors Journal.

[18]  Yingping Huang,et al.  Ego-Motion Estimation Using Recurrent Convolutional Neural Networks through Optical Flow Learning , 2021 .

[19]  Hyuk-Jae Lee,et al.  A Novel Hardware Architecture With Reduced Internal Memory for Real-Time Extraction of SIFT in an HD Video , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Jack E. Volder.  The CORDIC Trigonometric Computing Technique , 1959, IRE Trans. Electron. Comput..

[21]  Xiao Wang,et al.  Efficient parallel optimizations of a high-performance SIFT on GPUs , 2019, J. Parallel Distributed Comput..

[22]  Tian-Sheuan Chang,et al.  Fast SIFT Design for Real-Time Visual Feature Extraction , 2013, IEEE Transactions on Image Processing.

[23]  John N. Lygouras,et al.  Fully pipelined FPGA-based architecture for real-time SIFT extraction , 2016, Microprocess. Microsystems.

[24]  John N. Lygouras,et al.  FPGA-based architecture of a real-time SIFT matcher and RANSAC algorithm for robotic vision applications , 2018, Multimedia Tools and Applications.

[25]  8 Epipolar Geometry and the Fundamental Matrix , .

[26]  Ema Utami,et al.  Comparison of Scale Invariant Feature Transform and Speed Up Robust Feature for Image Forgery Detection Copy Move , 2019, 2019 4th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE).

[27]  George A. Constantinides,et al.  A Parallel Hardware Architecture for Scale and Rotation Invariant Feature Detection , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Chen-Chien James Hsu,et al.  FPGA-Based Hardware Design for Scale-Invariant Feature Transform , 2018, IEEE Access.