C‐LOD: Context‐aware Material Level‐of‐Detail applied to Mobile Graphics

Attention‐based Level‐Of–Detail (LOD) managers downgrade the quality of areas that are expected to go unnoticed by an observer to economize on computational resources. The perceptibility of lowered visual fidelity is determined by the accuracy of the attention model that assigns quality levels. Most previous attention based LOD managers do not take into account saliency provoked by context, failing to provide consistently accurate attention predictions. In this work, we extend a recent high level saliency model with four additional components yielding more accurate predictions: an object‐intrinsic factor accounting for canonical form of objects, an object‐context factor for contextual isolation of objects, a feature uniqueness term that accounts for the number of salient features in an image, and a temporal context that generates recurring fixations for objects inconsistent with the context. We conduct a perceptual experiment to acquire the weighting factors to initialize our model. We design C‐LOD, a LOD manager that maintains a constant frame rate on mobile devices by dynamically re‐adjusting material quality on secondary visual features of non‐attended objects. In a proof of concept study we establish that by incorporating C‐LOD, complex effects such as parallax occlusion mapping usually omitted in mobile devices can now be employed, without overloading GPU capability and, at the same time, conserving battery power.

[1]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[2]  Kurt Debattista,et al.  A GPU based saliency map for high-fidelity selective rendering , 2006, AFRIGRAPH '06.

[3]  Marina Daecher,et al.  Level Of Detail For 3d Graphics , 2016 .

[4]  Sungkil Lee,et al.  Real-Time Tracking of Visually Attended Objects in Virtual Environments and Its Application to LOD , 2009, IEEE Transactions on Visualization and Computer Graphics.

[5]  Adam Finkelstein,et al.  Perceptual models of viewpoint preference , 2011, TOGS.

[6]  Katerina Mania,et al.  A schema-based selective rendering framework , 2009, APGV '09.

[7]  Erik Reinhard,et al.  A psychophysical study of fixation behavior in a computer game , 2008, APGV '08.

[8]  Hans-Peter Seidel,et al.  On-the-fly Point Clouds through Histogram Pyramids , 2006 .

[9]  J. Movshon,et al.  The statistical reliability of signals in single neurons in cat and monkey visual cortex , 1983, Vision Research.

[10]  Henrik I. Christensen,et al.  Computational visual attention systems and their cognitive foundations: A survey , 2010, TAP.

[11]  Mark W. Becker,et al.  Object-intrinsic oddities draw early saccades. , 2007, Journal of experimental psychology. Human perception and performance.

[12]  J. Theeuwes,et al.  Irrelevant singletons capture attention: Evidence from inhibition of return , 2002, Perception & psychophysics.

[13]  Lester C. Loschky,et al.  User performance with gaze contingent multiresolutional displays , 2000, ETRA.

[14]  Tomas Akenine-Möller,et al.  The State of the Art in Mobile Graphics Research , 2008, IEEE Computer Graphics and Applications.

[15]  Michael S. Ambinder,et al.  Change blindness , 1997, Trends in Cognitive Sciences.

[16]  Ronald A. Rensink The Dynamic Representation of Scenes , 2000 .

[17]  M. Eckstein The Lower Visual Search Efficiency for Conjunctions Is Due to Noise and not Serial Attentional Processing , 1998 .

[18]  John Owens,et al.  Streaming architectures and technology trends , 2005, SIGGRAPH Courses.

[19]  James H. Clark,et al.  Hierarchical geometric models for visible surface algorithms , 1976, CACM.

[20]  W. Brewer,et al.  Role of schemata in memory for places , 1981, Cognitive Psychology.

[21]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[22]  M. Posner,et al.  Components of visual orienting , 1984 .

[23]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[24]  Natalya Tatarchuk,et al.  Dynamic parallax occlusion mapping with approximate soft shadows , 2006, I3D '06.

[25]  Jonathan D. Cohen,et al.  Level of Detail for 3D Graphics , 2012 .

[26]  M J Tarr,et al.  What Object Attributes Determine Canonical Views? , 1999, Perception.

[27]  Anatole Lécuyer,et al.  A real-time visual attention model for predicting gaze point during first-person exploration of virtual environments , 2010, VRST '10.

[28]  J. Henderson,et al.  The effects of semantic consistency on eye movements during complex scene viewing , 1999 .

[29]  Erik Lindholm,et al.  A user-programmable vertex engine , 2001, SIGGRAPH.

[30]  Alan Chalmers,et al.  Detail to Attention: Exploiting Visual Tasks for Selective Rendering , 2003, Rendering Techniques.

[31]  Andrew T. Duchowski,et al.  Eye Tracking Methodology: Theory and Practice , 2003, Springer London.

[32]  S. Tachi,et al.  Detailed Shape Representation with Parallax Mapping , 2001 .

[33]  Chris Wyman An approximate image-space approach for interactive refraction , 2005, SIGGRAPH 2005.

[34]  H. Pashler,et al.  Evidence for split attentional foci. , 2000, Journal of experimental psychology. Human perception and performance.

[35]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[36]  George Drettakis,et al.  An Automated High-Level Saliency Predictor for Smart Game Balancing , 2014, ACM Trans. Appl. Percept..

[37]  Mel Slater,et al.  Simulating virtual environments within virtual environments as the basis for a psychophysics of presence , 2010, SIGGRAPH 2010.