Image feature extraction for mobile processors

High-quality cameras are a standard feature of mobile platforms, but the computational capabilities of mobile processors limit the applications capable of exploiting them. Emerging mobile application domains, for example Mobile Augmented Reality (MAR), rely heavily on techniques from computer vision, requiring sophisticated analyses of images followed by higher-level processing. An important class of image analyses is the detection of sparse localized interest points. The Scale Invariant Feature Transform (SIFT), the most popular such analysis, is computationally representative of many other feature extractors. Using a novel code-generation framework, we demonstrate that a small set of optimizations produce high-performance SIFT implementations for three very different architectures: a laptop CPU (Core 2 Duo), a low-power CPU (Intel Atom), and a low-power GPU (GMA X3100). We improve the runtime of SIFT by more than 5X on our low-power architectures, enabling a low-power mobile device to extract SIFT features up to 63% as fast as the laptop CPU.

[1]  G. Gerosa,et al.  A sub 2W low power IA Processor for Mobile Internet Devices in 45nm Hi-K metal gate CMOS , 2008, 2008 IEEE Asian Solid-State Circuits Conference.

[2]  Samuel Williams,et al.  Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures , 2008, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis.

[3]  Cordelia Schmid,et al.  A sparse texture representation using local affine regions , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Qi Zhang,et al.  SIFT implementation and optimization for multi-core systems , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[5]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[6]  Steve Mann,et al.  OpenVIDIA: parallel GPU computer vision , 2005, ACM Multimedia.

[7]  Cordelia Schmid,et al.  Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[8]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[9]  Franz Franchetti,et al.  SPIRAL: Code Generation for DSP Transforms , 2005, Proceedings of the IEEE.

[10]  Jack J. Dongarra,et al.  Automatically Tuned Linear Algebra Software , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[11]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Belliappa Kuttanna,et al.  A Sub-1W to 2W Low-Power IA Processor for Mobile Internet Devices and Ultra-Mobile PCs in 45nm Hi-Κ Metal Gate CMOS , 2008, 2008 IEEE International Solid-State Circuits Conference - Digest of Technical Papers.

[13]  Katherine Yelick,et al.  OSKI: A library of automatically tuned sparse matrix kernels , 2005 .

[14]  Kurt Keutzer,et al.  Efficient, high-quality image contour detection , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Steven G. Johnson,et al.  The Design and Implementation of FFTW3 , 2005, Proceedings of the IEEE.

[16]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[17]  Samuel Williams,et al.  Optimization of sparse matrix-vector multiplication on emerging multicore platforms , 2009, Parallel Comput..

[18]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[19]  Kurt Keutzer,et al.  Building ASIPs: The Mescal Methodology , 2006 .

[20]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..