An image-warping VR-architecture: design, implementation and applications

We describe an architecture that provides a programmable display layer in order to allow the execution of custom programs on consecutive display frames. This replaces the default display behavior of repeating application frames until an update is available. The architecture is implemented using a multi-GPU system. We will show three applications of the architecture typical to VR. First, smooth motion is provided by generating intermediate display frames by per-pixel depth-image warping using 3D motion fields. Smooth motion can be beneficial for walk-throughs of large scenes. Second, we implement fine-grained latency reduction at the display frame level using a synchronized prediction of simulation objects and the viewpoint. This improves the average quality and consistency of latency reduction. Third, a crosstalk reduction algorithm for consecutive display frames is implemented, which improves the quality of stereoscopic images.

[1]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[2]  Mark R. Mine.  Characterization of End-to-End Delays in Head-Mounted Display Systems , 1993 .

[3]  Bernd Fröhlich,et al.  Three extensions to subtractive crosstalk reduction , 2007, EGVE'07.

[4]  Bernd Fröhlich,et al.  The design and implementation of a VR-architecture for smooth motion , 2007, VRST '07.

[5]  Marc Olano,et al.  Combatting rendering latency , 1995, I3D '95.

[6]  Renato Pajarola,et al.  REAL-TIME 3 D GRAPHICS STREAMING USING MPEG 4 , 2004 .

[7]  Willem Bles,et al.  Appropriate use of virtual environments to minimize motion sickness , 2000 .

[8]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[9]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[10]  Peter J. Bex,et al.  MULTIPLE IMAGES APPEAR WHEN MOTION ENERGY DETECTION FAILS , 1995 .

[11]  Ryugo Kijima,et al.  Reflex HMD to compensate lag and correction of derivative deformation , 2002, Proceedings IEEE Virtual Reality 2002.

[12]  Chris Shaw,et al.  Decoupled simulation in virtual reality with the MR toolkit , 1993, TOIS.

[13]  A. Lastra,et al.  Efficient warping for architectural walkthroughs using layered depth images , 1998, Proceedings Visualization '98 (Cat. No.98CB36276).

[14]  Bernd Fröhlich,et al.  Non-Uniform Crosstalk Reduction for Dynamic Scenes , 2007, 2007 IEEE Virtual Reality Conference.

[15]  Dan S. Wallach,et al.  Accelerated MPEG compression of dynamic polygonal scenes , 1994, SIGGRAPH.

[16]  Leonard McMillan,et al.  PixelView: a view-independent graphics rendering architecture , 2004, Graphics Hardware.

[17]  Bernd Fröhlich,et al.  Multi-Frame Rate Rendering and Display , 2007, 2007 IEEE Virtual Reality Conference.

[18]  Andrew J. Woods,et al.  Characterizing sources of ghosting in time-sequential stereoscopic video displays , 2002, IS&T/SPIE Electronic Imaging.

[19]  Joyce E. Farrell,et al.  The visible persistence of stimuli in stroboscopic motion , 1990, Vision Research.