A Programmable Display Layer for Virtual Reality System Architectures

Display systems typically operate at a minimum rate of 60 Hz. However, existing VR-architectures generally produce application updates at a lower rate. Consequently, the display is not updated by the application every display frame. This causes a number of undesirable perceptual artifacts. We describe an architecture that provides a programmable display layer (PDL) in order to generate updated display frames. This replaces the default display behavior of repeating application frames until an update is available. We will show three benefits of the architecture typical to VR. First, smooth motion is provided by generating intermediate display frames by per-pixel depth-image warping using 3D motion fields. Smooth motion eliminates various perceptual artifacts due to judder. Second, we implement fine-grained latency reduction at the display frame level using a synchronized prediction of simulation objects and the viewpoint. This improves the average quality and consistency of latency reduction. Third, a crosstalk reduction algorithm for consecutive display frames is implemented, which improves the quality of stereoscopic images. To evaluate the architecture, we compare image quality and latency to that of a classic level-of-detail approach.

[1]  Ronald Pose,et al.  Priority rendering with a virtual reality address recalculation pipeline , 1994, SIGGRAPH.

[2]  Bernd Fröhlich,et al.  Non-Uniform Crosstalk Reduction for Dynamic Scenes , 2007, 2007 IEEE Virtual Reality Conference.

[3]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[4]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[5]  P. Debevec,et al.  Image-based modeling, rendering, and lighting , 2002, IEEE Computer Graphics and Applications.

[6]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[7]  Mark R. Mine.  Characterization of End-to-End Delays in Head-Mounted Display Systems , 1993 .

[8]  Ronald Pose,et al.  techniques for Reducing Virtual Reality Latency with Architectural Support and Consideration on Human Factors , 1994, MHVR.

[9]  Bernd Fröhlich,et al.  An Image-Warping Architecture for VR: Low Latency versus Image Quality , 2009, 2009 IEEE Virtual Reality Conference.

[10]  Bernd Fröhlich,et al.  Three extensions to subtractive crosstalk reduction , 2007, EGVE'07.

[11]  Willem Bles,et al.  Appropriate use of virtual environments to minimize motion sickness , 2000 .

[12]  Peter J. Bex,et al.  MULTIPLE IMAGES APPEAR WHEN MOTION ENERGY DETECTION FAILS , 1995 .

[13]  Joyce E. Farrell,et al.  The visible persistence of stimuli in stroboscopic motion , 1990, Vision Research.

[14]  Jay Torborg,et al.  Talisman: commodity realtime 3D graphics for the PC , 1996, SIGGRAPH.

[15]  Leonard McMillan,et al.  PixelView: a view-independent graphics rendering architecture , 2004, Graphics Hardware.

[16]  Bernd Fröhlich,et al.  A shared-scene-graph image-warping architecture for VR: Low latency versus image quality , 2010, Comput. Graph..

[17]  William R. Mark,et al.  Post-Rendering 3D Image Warping: Visibility, Reconstruction, and Performance for Depth-Image Warping , 1999 .

[18]  Bernd Fröhlich,et al.  Multi-Frame Rate Rendering and Display , 2007, 2007 IEEE Virtual Reality Conference.

[19]  Andrew J. Woods,et al.  Characterizing sources of ghosting in time-sequential stereoscopic video displays , 2002, IS&T/SPIE Electronic Imaging.

[20]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[21]  Chris Shaw,et al.  Decoupled simulation in virtual reality with the MR toolkit , 1993, TOIS.

[22]  Ryugo Kijima,et al.  Reflex HMD to compensate lag and correction of derivative deformation , 2002, Proceedings IEEE Virtual Reality 2002.

[23]  Marc Olano,et al.  Combatting rendering latency , 1995, I3D '95.

[24]  Paolo Cignoni,et al.  MeshLab: an Open-Source 3D Mesh Processing System , 2008, ERCIM News.

[25]  Anthony Steed.  A simple method for estimating the latency of interactive, real-time graphics simulations , 2008, VRST '08.