Computer Vision Accelerators for Mobile Systems based on OpenCL GPGPU Co-Processing

In this paper, we present an OpenCL-based heterogeneous implementation of a computer vision algorithm – image inpainting-based object removal algorithm – on mobile devices. To take advantage of the computation power of the mobile processor, the algorithm workflow is partitioned between the CPU and the GPU based on the profiling results on mobile devices, so that the computationally-intensive kernels are accelerated by the mobile GPGPU (general-purpose computing using graphics processing units). By exploring the implementation trade-offs and utilizing the proposed optimization strategies at different levels including algorithm optimization, parallelism optimization, and memory access optimization, we significantly speed up the algorithm with the CPU-GPU heterogeneous implementation, while preserving the quality of the output images. Experimental results show that heterogeneous computing based on GPGPU co-processing can significantly speed up the computer vision algorithms and makes them practical on real-world mobile devices.

[1]  Jyrki Leskela,et al.  OpenCL embedded profile prototype in mobile device , 2009, 2009 IEEE Workshop on Signal Processing Systems.

[2]  Seth Hall,et al.  GPU-based Image Analysis on Mobile Devices , 2011, ArXiv.

[3]  Juan M. Orduña,et al.  Performance Characterization of Mobile Phones in Augmented Reality Marker Tracking , 2012 .

[4]  Jason Clemons,et al.  Computer Architectures for Mobile Computer Vision Systems , 2013 .

[5]  Yingen Xiong,et al.  Effective gradient domain object editing on mobile devices , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[6]  Patrick Pérez,et al.  Object removal by exemplar-based inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[7]  Joseph R. Cavallaro,et al.  Workload analysis and efficient OpenCL-based implementation of SIFT algorithm on a smartphone , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[8]  Matthew Turk,et al.  Location-based augmented reality on mobile phones , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[9]  Dieter Schmalstieg,et al.  Pose tracking from natural features on mobile phones , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[10]  Xin Yang,et al.  LDB: An ultra-fast feature for scalable Augmented Reality on mobile devices , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[11]  Olli Silvén,et al.  Accelerating image recognition on mobile devices using GPGPU , 2011, Electronic Imaging.

[12]  Timothy G. Mattson,et al.  OpenCL Programming Guide , 2011 .

[13]  Robert Hofmann,et al.  Extraction of Natural Feature Descriptors on Mobile GPUs , 2012 .

[14]  Gerhard Reitmayr,et al.  A GPGPU accelerated descriptor for mobile devices , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[15]  Zhen Fang,et al.  Accelerating mobile augmented reality on a handheld platform , 2009, 2009 IEEE International Conference on Computer Design.

[16]  Kwang-Ting Cheng,et al.  Using mobile GPU for general-purpose computing – a case study of face recognition on smartphones , 2011, Proceedings of 2011 International Symposium on VLSI Design, Automation and Test.

[17]  Xin Yang,et al.  Accelerating SURF detector on mobile devices , 2012, ACM Multimedia.

[18]  Tack-Don Han,et al.  MobiRT: an implementation of OpenGL ES-based CPU-GPU hybrid ray tracer for mobile devices , 2010, SIGGRAPH ASIA.

[19]  Sungdae Cho,et al.  Implementation and optimization of image processing algorithms on handheld GPU , 2010, 2010 IEEE International Conference on Image Processing.

[20]  Joseph R. Cavallaro,et al.  Accelerating computer vision algorithms using OpenCL framework on the mobile GPU - A case study , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Joseph R. Cavallaro,et al.  A fast and efficient sift detector using the mobile GPU , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[22]  Kwang-Ting Cheng,et al.  Energy and Performance Characterization of Mobile Heterogeneous Computing , 2012, 2012 IEEE Workshop on Signal Processing Systems.

[23]  Kari Pulli,et al.  Real-time computer vision with OpenCV , 2012, Commun. ACM.