Hardware Based Scale- and Rotation-Invariant Feature Extraction: A Retrospective Analysis and Future Directions

Computer Vision techniques represent a class of algorithms that are highly computation and data intensive in nature. Generally, performance of these algorithms in terms of execution speed on desktop computers is far from real-time. Since real-time performance is desirable in many applications, special-purpose hardware is required in most cases to achieve this goal. Scale- and rotation-invariant local feature extraction is a low level computer vision task with very high computational complexity. The state-of-the-art algorithms that currently exist in this domain, like SIFT and SURF, suffer from slow execution speeds and at best can only achieve rates of 2-3 Hz on modern desktop computers. Hardware-based scale- and rotation-invariant local feature extraction is an emerging trend enabling real-time performance for these computationally complex algorithms. This paper takes a retrospective look at the advances made so far in this field, discusses the hardware design strategies employed and results achieved, identifies current research gaps and suggests future research directions.

[1]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[2]  Karsten Berns,et al.  SoPC Architecture for a Key Point Detector , 2007, 2007 International Conference on Field Programmable Logic and Applications.

[3]  David Nistér,et al.  Linear Time Maximally Stable Extremal Regions , 2008, ECCV.

[4]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[5]  Piotr Jasiobedzki,et al.  Stereo-Vision Based 3D Modeling and Localization for Unmanned Vehicles , 2008 .

[6]  W. James MacLean,et al.  Real-Time Extraction of Maximally Stable Extremal Regions on an FPGA , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[7]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[8]  F. Attneave.  Some informational aspects of visual perception. , 1954, Psychological review.

[9]  W. James MacLean,et al.  Implementation of an Affine-Covariant Feature D etector in Field-Programmable Gate Arrays , 2007, ICVS 2007.

[10]  S. Se,et al.  VISION BASED MODELING AND LOCALIZATION FOR PLANETARY EXPLORATION ROVERS , 2004 .

[11]  George A. Constantinides,et al.  A Parallel Hardware Architecture for Scale and Rotation Invariant Feature Detection , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Binoy Pinto,et al.  Speeded Up Robust Features , 2011 .

[13]  Donghyun Kim,et al.  An 81.6 GOPS Object Recognition Processor Based on NoC and Visual Image Processing Memory , 2007, 2007 IEEE Custom Integrated Circuits Conference.

[14]  Christophe Bobda,et al.  Optimizing Logarithmic Arithmetic on FPGAs , 2007, 15th Annual IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM 2007).

[15]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[16]  Karsten Berns,et al.  Hardware/Software co-design of a key point detector on FPGA , 2007, 15th Annual IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM 2007).

[17]  Harm J. W. Belt.  Word length reduction for the integral image , 2008, 2008 15th IEEE International Conference on Image Processing.

[18]  B. Kisacanin,et al.  Integral Image Optimizations for Embedded Vision Applications , 2008, 2008 IEEE Southwest Symposium on Image Analysis and Interpretation.

[19]  L. Petersson,et al.  Online stereo calibration using FPGAs , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[20]  Satoru Yamamoto,et al.  Systolic Architecture for Computational Fluid Dynamics on FPGAs , 2007 .

[21]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[22]  Jens H. Krüger,et al.  A Survey of General‐Purpose Computation on Graphics Hardware , 2007, Eurographics.

[23]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..