An Image-Warping Architecture for VR: Low Latency versus Image Quality

Designing low end-to-end latency system architectures for virtual reality is still an open and challenging problem. We describe the design, implementation and evaluation of a client-server depth-image warping architecture that updates and displays the scene graph at the refresh rate of the display. Our approach works for scenes consisting of dynamic and interactive objects. The end-to-end latency is minimized as well as smooth object motion generated. However, this comes at the expense of image quality inherent to warping techniques. We evaluate the architecture and its design trade-offs by comparing latency and image quality to a conventional rendering system. Our experience with the system confirms that the approach facilitates common interaction tasks such as navigation and object manipulation.

[1]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[2]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[3]  Mark R. Mine Characterization of End-to-End Delays in Head-Mounted Display Systems , 1993 .

[4]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[5]  Marc Olano,et al.  Combatting rendering latency , 1995, I3D '95.

[6]  Ryugo Kijima,et al.  Reflex HMD to compensate lag and correction of derivative deformation , 2002, Proceedings IEEE Virtual Reality 2002.

[7]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[8]  Bernd Fröhlich,et al.  An image-warping VR-architecture: design, implementation and applications , 2008, VRST '08.

[9]  Peter J. Bex,et al.  MULTIPLE IMAGES APPEAR WHEN MOTION ENERGY DETECTION FAILS , 1995 .

[10]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[11]  Anthony Steed A simple method for estimating the latency of interactive, real-time graphics simulations , 2008, VRST '08.

[12]  Leonard McMillan,et al.  PixelView: a view-independent graphics rendering architecture , 2004, Graphics Hardware.

[13]  Renato Pajarola,et al.  Multi-view point splatting , 2006, GRAPHITE '06.

[14]  William R. Mark,et al.  Post-Rendering 3D Image Warping: Visibility, Reconstruction, and Performance for Depth-Image Warping , 1999 .

[15]  Jay Torborg,et al.  Talisman: commodity realtime 3D graphics for the PC , 1996, SIGGRAPH.

[16]  Bernd Fröhlich,et al.  The design and implementation of a VR-architecture for smooth motion , 2007, VRST '07.