Optimization Techniques for 3D Graphics Deployment on Mobile Devices

Abstract3D Internet technologies are becoming essential enablers in many application areas including games, education, collaboration, navigation and social networking. The use of 3D Internet applications with mobile devices provides location-independent access and richer use context, but also performance issues. Therefore, one of the important challenges facing 3D Internet applications is the deployment of 3D graphics on mobile devices. In this article, we present an extensive survey on optimization techniques for 3D graphics deployment on mobile devices and qualitatively analyze the applicability of each technique from the standpoints of visual quality, performance and energy consumption. The analysis focuses on optimization techniques related to data-driven 3D graphics deployment, because it supports off-line use, multi-user interaction, user-created 3D graphics and creation of arbitrary 3D graphics. The outcome of the analysis facilitates the development and deployment of 3D Internet applications on mobile devices and provides guidelines for future research.Graphical Abstract

[1]  Jürgen Döllner,et al.  Server-based rendering of large 3D scenes for mobile devices using G-buffer cube maps , 2012, Web3D '12.

[2]  Jürgen Döllner,et al.  Service-oriented interactive 3D visualization of massive 3D city models on thin clients , 2011, COM.Geo.

[3]  Steve Benford,et al.  MobGeoSen: facilitating personal geosensor data collection and visualization using mobile phones , 2007, Personal and Ubiquitous Computing.

[4]  魏瑨,et al.  Feature Preserving Mesh Simplification Using Feature Sensitive Metric , 2010 .

[5]  Pierre Alliez,et al.  Progressive compression for lossless transmission of triangle meshes , 2001, SIGGRAPH.

[6]  Norbert Wehn,et al.  DRAM power management and energy consumption: a critical assessment , 2009, SBCCI.

[7]  John D. Owens,et al.  Resolution-matched shadow maps , 2007, TOGS.

[8]  Krste Asanovic,et al.  Energy Aware Lossless Data Compression , 2003, MobiSys.

[9]  Leif Kobbelt,et al.  Character animation from 2D pictures and 3D motion data , 2007, TOGS.

[10]  Hojung Cha,et al.  Runtime power estimation of mobile AMOLED displays , 2013, 2013 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[11]  Paolo Fiorini,et al.  Energy autonomous systems : future trends in devices, technology, and systems , 2009 .

[12]  Lin Zhong,et al.  Power modeling of graphical user interfaces on OLED displays , 2009, 2009 46th ACM/IEEE Design Automation Conference.

[13]  Peter Deutsch,et al.  GZIP file format specification version 4.3 , 1996, RFC.

[14]  Andrea Sanna,et al.  A Streaming-Based Solution for Remote Visualization of 3D Graphics on Mobile Devices , 2007, IEEE Transactions on Visualization and Computer Graphics.

[15]  Guillaume Lavoué,et al.  Rate-distortion optimization for progressive compression of 3D mesh with color attributes , 2011, The Visual Computer.

[16]  Ian H. Witten,et al.  Data Compression Using Adaptive Coding and Partial String Matching , 1984, IEEE Trans. Commun..

[17]  Sixto Ortiz,et al.  Is 3D Finally Ready for the Web? , 2010, Computer.

[18]  S. Kiss.  Book review: Virtual Clothing: Theory and Practice by P. Volino and N. Magnenat-Thalmann, Springer, 2000, ISBN 3-540-67600-7 , 2002 .

[19]  George Drettakis,et al.  Perspective shadow maps , 2002, ACM Trans. Graph..

[20]  Timo Koskela,et al.  Adaptive content management for collaborative 3D virtual spaces , 2013, 2013 13th Conference of Open Innovations Association (FRUCT).

[21]  László Böszörményi,et al.  A survey of Web cache replacement strategies , 2003, CSUR.

[22]  George Drettakis,et al.  C‐LOD: Context‐aware Material Level‐of‐Detail applied to Mobile Graphics , 2014, Comput. Graph. Forum.

[23]  C.-C. Jay Kuo,et al.  Technologies for 3D mesh compression: A survey , 2005, J. Vis. Commun. Image Represent..

[24]  Paula Savioja,et al.  Augmented reality for plant lifecycle management , 2007, 2007 IEEE International Technology Management Conference (ICE).

[25]  Seungyong Lee,et al.  Mesh Geometry Compression for Mobile Graphics , 2010, 2010 7th IEEE Consumer Communications and Networking Conference.

[26]  Shree K. Nayar,et al.  A practical analytic single scattering model for real time rendering , 2005, SIGGRAPH '05.

[27]  Jun Shingu,et al.  The virtual factory: Exploring 3D worlds as industrial collaboration and control environments , 2010, 2010 IEEE Virtual Reality Conference (VR).

[28]  Peter-Pike J. Sloan,et al.  Importance Driven Texture Coordinate Optimization , 1998, Comput. Graph. Forum.

[29]  Dieter W. Fellner,et al.  A scalable rendering framework for generative 3D content , 2014, Web3D '14.

[30]  Robert L. Cook,et al.  A Reflectance Model for Computer Graphics , 1987, TOGS.

[31]  Pedro V. Sander,et al.  Signal-Specialized Parametrization , 2002, Rendering Techniques.

[32]  Hugues Hoppe,et al.  Progressive meshes , 1996, SIGGRAPH.

[33]  Matt Pharr,et al.  Gpu gems 2: programming techniques for high-performance graphics and general-purpose computation , 2005 .

[34]  Manuel Menezes de Oliveira Neto,et al.  Relief mapping of non-height-field surface details , 2006, I3D '06.

[35]  Timo Koskela,et al.  Power Consumption Model of a Mobile GPU Based on Rendering Complexity , 2013, 2013 Seventh International Conference on Next Generation Mobile Apps, Services and Technologies.

[36]  Dmitry A. Shkarin,et al.  PPM: one step to practicality , 2002, Proceedings DCC 2002. Data Compression Conference.

[37]  Martin Pettersson,et al.  ETC2: texture compression using invalid combinations , 2007, GH '07.

[38]  Andrew Willmott,et al.  Rapid simplification of multi-attribute meshes , 2011, HPG '11.

[39]  S. Tachi,et al.  Detailed Shape Representation with Parallax Mapping , 2001 .

[40]  Marc Alexa,et al.  The POP Buffer: Rapid Progressive Clustering by Geometry Quantization , 2013, Comput. Graph. Forum.

[41]  Luca Chittaro,et al.  Rendering of X3D content on mobile devices with OpenGL ES , 2006, Web3D '06.

[42]  Matti Siekkinen,et al.  On the energy efficiency of proxy-based traffic shaping for mobile audio streaming , 2011, 2011 IEEE Consumer Communications and Networking Conference (CCNC).

[43]  Adam Hunter,et al.  Uniform frequency images: adding geometry to images to produce space-efficient textures , 2000 .

[44]  Heidrun Schumann,et al.  Demand-driven image transmission with levels of detail and regions of interest , 1999, Comput. Graph..

[45]  Steve Marschner,et al.  Microfacet Models for Refraction through Rough Surfaces , 2007, Rendering Techniques.

[46]  Jacob Ström,et al.  Lossless compression of already compressed textures , 2011, HPG '11.

[47]  Josep Blat,et al.  3D graphics on the web: A survey , 2014, Comput. Graph..

[48]  Hanan Samet,et al.  Applications of spatial data structures - computer graphics, image processing, and GIS , 1990 .

[49]  Jack J. Purdum,et al.  C programming guide , 1983 .

[50]  Mohamed Hefeeda,et al.  Energy-efficient gaming on mobile devices using dead reckoning-based power management , 2010, 2010 9th Annual Workshop on Network and Systems Support for Games.

[51]  Feng Qian,et al.  An in-depth study of LTE: effect of network protocol and application behavior on performance , 2013, SIGCOMM.

[52]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[53]  Yanghee Choi,et al.  Traffic characteristics of a massively multi-player online role playing game , 2005, NetGames '05.

[54]  Alexander Zipf,et al.  Generating web-based 3D City Models from OpenStreetMap: The current situation in Germany , 2010, Comput. Environ. Urban Syst..

[55]  Erno Kuusela,et al.  Methods and network architecture for modifying extensible virtual environment to support mobility , 2011, MindTrek.

[56]  Steve Wheeler,et al.  Second Life: an overview of the potential of 3-D virtual worlds in medical and health education. , 2007, Health information and libraries journal.

[57]  Matti Siekkinen,et al.  Streaming over 3G and LTE: how to save smartphone energy in radio access network-friendly way , 2013, MoVid '13.

[58]  Matti Siekkinen,et al.  A System-Level Model for Runtime Power Estimation on Mobile Devices , 2010, 2010 IEEE/ACM Int'l Conference on Green Computing and Communications & Int'l Conference on Cyber, Physical and Social Computing.

[59]  Terry A. Welch,et al.  A Technique for High-Performance Data Compression , 1984, Computer.

[60]  Chun-Fa Chang,et al.  Enhancing 3D Graphics on Mobile Devices by Image-Based Rendering , 2002, IEEE Pacific Rim Conference on Multimedia.

[61]  Paulo Trezentos,et al.  Impacts of data interchange formats on energy consumption and performance in smartphones , 2011, OSDOC '11.

[62]  Moez Limayem,et al.  3D Social Virtual Worlds: Research Issues and Challenges , 2008, IEEE Internet Computing.

[63]  Athanasios V. Vasilakos,et al.  Measurement and analysis of World of Warcraft in mobile WiMAX networks , 2009, 2009 8th Annual Workshop on Network and Systems Support for Games (NetGames).

[64]  Bren Mochocki,et al.  Power Analysis of Mobile 3D Graphics , 2006, Proceedings of the Design Automation & Test in Europe Conference.

[65]  Tomas Akenine-Möller,et al.  iPACKMAN: high-quality, low-complexity texture compression for mobile phones , 2005, HWWS '05.

[66]  Muhammad Hussain.  Efficient Simplification Methods for Generating High Quality LODs of 3D Meshes , 2009, Journal of Computer Science and Technology.

[67]  Dave Shreiner,et al.  The OpenGL ES 2.0 programming guide , 2008 .

[68]  Tomas Akenine-Möller,et al.  The State of the Art in Mobile Graphics Research , 2008, IEEE Computer Graphics and Applications.

[69]  Hugues Hoppe,et al.  Efficient implementation of progressive meshes , 1998, Comput. Graph..

[70]  Khe Foon Hew,et al.  Use of three-dimensional (3-D) immersive virtual worlds in K-12 and higher education settings: A review of the research , 2010, Br. J. Educ. Technol..

[71]  David Murphy,et al.  From 2D Web Map to Mobile 3D Mirror World: A Live Virtual Advertising Use Case , 2012, 2012 Sixth International Conference on Next Generation Mobile Applications, Services and Technologies.

[72]  Antti Nurminen,et al.  Mobile, hardware-accelerated urban 3D maps in 3G networks , 2007, Web3D '07.

[73]  Mohammad Hosseini,et al.  Energy-aware adaptations in mobile 3d graphics , 2012, ACM Multimedia.

[74]  Lin Zhong,et al.  Power-saving color transformation of mobile graphical user interfaces on OLED-based displays , 2009, ISLPED.

[75]  Albert Mo Kim Cheng,et al.  Priority-driven coding of progressive JPEG images for transmission in real-time applications , 2005, 11th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA'05).

[76]  Luigi Ciminiera,et al.  An Adaptive Control System to Deliver Interactive Virtual Environment Content to Handheld Devices , 2011, Mob. Networks Appl..

[77]  Pierre Alliez,et al.  Progressive compression of manifold polygon meshes , 2012, Comput. Graph..

[78]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[79]  Ning Gu,et al.  An Agent Approach to Supporting Collaborative Design in 3D Virtual Worlds , 2005, eCAADe proceedings.

[80]  Y. Wang,et al.  Video plus depth compression for mobile 3D services , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[81]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[82]  Gauthier Lafruit,et al.  A model for adapting 3D graphics based on scalable coding, real-time simplification and remote rendering , 2008 .

[83]  Anselmo Lastra,et al.  Precision selection for energy-efficient pixel shaders , 2011, HPG '11.

[84]  Kirill Dmitriev,et al.  Generating displacement from normal map for use in 3D games , 2011, SIGGRAPH '11.

[85]  Paolo Cignoni,et al.  BDAM — Batched Dynamic Adaptive Meshes for High Performance Terrain Visualization , 2003, Comput. Graph. Forum.

[86]  A. Venkataramani,et al.  Energy Consumption in Mobile Phones: Measurement, Design Implications, and Algorithms , 2009 .

[87]  Arun Venkataramani,et al.  Energy consumption in mobile phones: a measurement study and implications for network applications , 2009, IMC '09.

[88]  Touradj Ebrahimi,et al.  The JPEG 2000 still image compression standard , 2001, IEEE Signal Process. Mag..

[89]  Pierre Alliez,et al.  Recent advances in compression of 3D meshes , 2005, 2005 13th European Signal Processing Conference.

[90]  Dieter Schmalstieg,et al.  Fast Projected Area Computation for Three-Dimensional Bounding Boxes , 1999, J. Graphics, GPU, & Game Tools.

[91]  Tomas Akenine-Möller,et al.  Real-time rendering , 1997 .

[92]  Gabriel Taubin,et al.  Space-Optimized Texture Maps (Guenter Enderle [Best Paper] Award , 2002, Comput. Graph. Forum.

[93]  Jörg Widmer,et al.  A generic proxy system for networked computer games , 2002, NetGames '02.

[94]  Gozde Bozdagi Akar,et al.  Three-Dimensional Media for Mobile Devices , 2011, Proceedings of the IEEE.

[95]  Toni Alatalo,et al.  An Entity-Component Model for Extensible Virtual Worlds , 2011, IEEE Internet Computing.

[96]  Michael Garland,et al.  Simplifying surfaces with color and texture using quadric error metrics , 1998, Proceedings Visualization '98 (Cat. No.98CB36276).

[97]  Chandra S. Bontu,et al.  DRX mechanism for power saving in LTE , 2009, IEEE Communications Magazine.

[98]  D. J. Wheeler,et al.  A Block-sorting Lossless Data Compression Algorithm , 1994 .

[99]  Guillaume Lavoué,et al.  Streaming compressed 3D data on the web using JavaScript and WebGL , 2013, Web3D '13.

[100]  Mika Ylianttila,et al.  Energy consumption model for mobile devices in 3G and WLAN networks , 2012, 2012 IEEE Consumer Communications and Networking Conference (CCNC).

[101]  Simin Nadjm-Tehrani,et al.  EnergyBox: A Trace-Driven Tool for Data Transmission Energy Consumption Studies , 2013, EE-LSDS.