A fast and efficient sift detector using the mobile GPU

Emerging mobile applications, such as augmented reality, demand robust feature detection at high frame rates. We present an implementation of the popular Scale-Invariant Feature Transform (SIFT) feature detection algorithm that incorporates the powerful graphics processing unit (GPU) in mobile devices. Where the usual GPU methods are inefficient on mobile hardware, we propose a heterogeneous dataflow scheme. By methodically partitioning the computation, compressing the data for memory transfers, and taking into account the unique challenges that arise out of the mobile GPU, we are able to achieve a speedup of 4-7x over an optimized CPU version, and a 6.4x speedup over a published GPU implementation. Additionally, we reduce energy consumption by 87 percent per image. We achieve near-realtime detection without compromising the original algorithm.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Zhen Fang,et al.  Performance characterization and optimization of mobile augmented reality on handheld platforms , 2009, 2009 IEEE International Symposium on Workload Characterization (IISWC).

[3]  Yongdong Zhang,et al.  GPU-based fast scale invariant interest point detector , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  Guy-Richard Kayombya,et al.  SIFT feature extraction on a Smartphone GPU using OpenGL ES2.0 , 2010 .

[5]  Joseph R. Cavallaro,et al.  Accelerating computer vision algorithms using OpenCL framework on the mobile GPU - A case study , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Kyle Dunn,et al.  Adaptive OpenCL (ACL) execution in GPU architectures , 2013, ADAPT '13.

[7]  Matt Pharr,et al.  Gpu gems 2: programming techniques for high-performance graphics and general-purpose computation , 2005 .

[8]  Olli Silvén,et al.  Accelerating image recognition on mobile devices using GPGPU , 2011, Electronic Imaging.

[9]  Zhen Fang,et al.  Accelerating mobile augmented reality on a handheld platform , 2009, 2009 IEEE International Conference on Computer Design.

[10]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[11]  Jan-Michael Frahm,et al.  Feature tracking and matching in video using programmable graphics hardware , 2007, Machine Vision and Applications.

[12]  T. Lindeberg,et al.  Scale-Space Theory : A Basic Tool for Analysing Structures at Different Scales , 1994 .

[13]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Thomas Wiegand,et al.  SIFT Implementation and Optimization for General-Purpose GPU , 2007 .

[15]  Kwang-Ting Cheng,et al.  Using mobile GPU for general-purpose computing – a case study of face recognition on smartphones , 2011, Proceedings of 2011 International Symposium on VLSI Design, Automation and Test.

[16]  Seth Hall,et al.  GPU-based Image Analysis on Mobile Devices , 2011, ArXiv.

[17]  Amy W. Apon,et al.  Accelerating SIFT on parallel architectures , 2009, 2009 IEEE International Conference on Cluster Computing and Workshops.