Explorations for real-time point cloud rendering of natural scenes in virtual reality

This work is a proof of concept, where we explore the possibility of rendering natural scenes in a head mounted display device without meshing. Real-time, stereoscopic full-HD rendering is obtained for a 14 million points scene, using a low end graphics card for virtual reality (Nvidia GeForce GTX 970), within an Oculus Rift DK2. High quality is achieved by using splatting, while real-time rendering is made possible by the means of a good data structure and a complexity reduction of the scene with techniques such as Optimized Sub-Sampling, Level of Detail and Frustum Culling. Altogether, those techniques lead to a good virtual reality immersion. Choices and limitations of the proposed techniques are discussed.

[1]  Helmut Mayer,et al.  Incremental Division of Very Large Point Clouds for Scalable 3D Surface Reconstruction , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[2]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[3]  Jon M. Kleinberg,et al.  Two algorithms for nearest-neighbor search in high dimensions , 1997, STOC '97.

[4]  Michael M. Kazhdan,et al.  Reconstruction of solid models from oriented point sets , 2005, SGP '05.

[5]  Leif Kobbelt,et al.  Optimized Sub‐Sampling of Point Sets for Surface Splatting , 2004, Comput. Graph. Forum.

[6]  Matthias Zwicker,et al.  Surfels: surface elements as rendering primitives , 2000, SIGGRAPH.

[7]  Marina Daecher,et al.  Level Of Detail For 3d Graphics , 2016 .

[8]  Donald Meagher,et al.  Geometric modeling using octree encoding , 1982, Computer Graphics and Image Processing.

[9]  Tomas Akenine-Möller,et al.  Optimized View Frustum Culling Algorithms for Bounding Boxes , 2000, J. Graphics, GPU, & Game Tools.

[10]  Leif Kobbelt,et al.  Phong Splatting , 2004, PBG.

[11]  Giovanna Sansoni,et al.  State-of-The-Art and Applications of 3D Imaging Sensors in Industry, Cultural Heritage, Medicine, and Criminal Investigation , 2009, Sensors.

[12]  Tom Davis,et al.  Opengl programming guide: the official guide to learning opengl , 1993 .

[13]  Markus Gross,et al.  Point-Based Graphics , 2007 .

[14]  James F. O'Brien,et al.  Spectral surface reconstruction from noisy point clouds , 2004, SGP '04.

[15]  R. Fabio.  From point cloud to surface the modeling and visualization problem , 2003 .

[16]  Fabio Remondino.  From point cloud to surface , 2003 .

[17]  Martin Reddy,et al.  Specification and evaluation of level of detail selection criteria , 1998, Virtual Reality.

[18]  Nassir Navab,et al.  Adaptive neighborhood selection for real-time surface normal estimation from organized point cloud data using integral images , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Youngho Chai,et al.  Analysis of Efficient Visualization and Interactive Editing of 3D Point Cloud Data , 2016 .

[20]  Yoshihiko Kuroki,et al.  3.4: Improvement of Motion Image Quality by High Frame Rate , 2006 .