Differentiated Handling of Physical Scenes and Virtual Objects for Mobile Augmented Reality

Mobile devices running augmented reality applications consume considerable energy for graphics-intensive workloads. This paper presents a scheme for the differentiated handling of camera-captured physical scenes and computer-generated virtual objects according to different perceptual quality metrics. We propose online algorithms and their real-time implementations to reduce energy consumption through dynamic frame rate adaptation while maintaining the visual quality required for augmented reality applications. To evaluate system efficacy, we integrate our scheme into Android and conduct extensive experiments on a commercial smartphone with various application scenarios. The results show that the proposed scheme can achieve energy savings of up to 39.1% in comparison to the native graphics system in Android while maintaining satisfactory visual quality.

[1]  Yunxin Liu,et al.  RAVEN: Perception-aware Optimization of Power Consumption for Mobile Games , 2017, MobiCom.

[2]  Yiran Chen,et al.  Scope - quality retaining display rendering workload scaling based on user-smartphone distance , 2016, 2016 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[3]  Kajal T. Claypool,et al.  On frame rate and player performance in first person shooter games , 2007, Multimedia Systems.

[4]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[5]  Keith Bugeja,et al.  Frame Rate vs Resolution: A Subjective Evaluation of Spatiotemporal Perceived Quality Under Varying Computational Budgets , 2018, Comput. Graph. Forum.

[6]  Paramvir Bahl,et al.  Energy characterization and optimization of image sensing toward continuous mobile vision , 2013, MobiSys '13.

[7]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[8]  Hojung Cha,et al.  Content-centric display energy management for mobile devices , 2014, 2014 51st ACM/EDAC/IEEE Design Automation Conference (DAC).

[9]  Osvaldo Simeone,et al.  Energy-Efficient Resource Allocation for Mobile Edge Computing-Based Augmented Reality Applications , 2016, IEEE Wireless Communications Letters.

[10]  Yafeng Yin,et al.  Tracking Human Motions in Photographing , 2017, ACM Trans. Sens. Networks.

[11]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[12]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[13]  Yiran Chen,et al.  Mobile GPU Power Consumption Reduction via Dynamic Resolution and Frame Rate Scaling , 2014, HotPower.

[14]  Petru Eles,et al.  Perception-aware power management for mobile games via dynamic resolution scaling , 2015, 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[15]  Luca Benini,et al.  Visual quality analysis for dynamic backlight scaling in LCD systems , 2009, 2009 Design, Automation & Test in Europe Conference & Exhibition.

[16]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..

[17]  Tei-Wei Kuo,et al.  A user-centric CPU-GPU governing framework for 3D games on mobile devices , 2015, 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[18]  Ali E. Kayaalp,et al.  A Pipeline Architecture For Near Real-Time Stereo Range Detection , 1989, Other Conferences.

[19]  Adam Wolisz,et al.  EvalVid - A Framework for Video Transmission and Quality Evaluation , 2003, Computer Performance Evaluation / TOOLS.

[20]  Minglu Li,et al.  E3: energy-efficient engine for frame rate adaptation on smartphones , 2013, SenSys '13.

[21]  Yunxin Liu,et al.  Optimizing Smartphone Power Consumption through Dynamic Resolution Scaling , 2015, MobiCom.

[22]  José C. López-Ardao,et al.  Combining the effects of frame rate, bit rate, display size and video content in a parametric video quality model , 2011, LANC.