GPU‐Based Spherical Light Field Rendering with Per‐Fragment Depth Correction

Image‐based rendering techniques are a powerful alternative to traditional polygon‐based computer graphics. This paper presents a novel light field rendering technique which performs per‐pixel depth correction of rays for high‐quality reconstruction. Our technique stores combined RGB and depth values in a parabolic 2D texture for every light field sample acquired at discrete positions on a uniform spherical setup. Image synthesis is implemented on the GPU as a fragment program which extracts the correct image information from adjacent cameras for each fragment by applying per‐pixel depth correction of rays.

[1]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[2]  Andrea Fusiello.  Image-based Rendering * , 2003 .

[3]  Andreas Kolb,et al.  Lateral and Depth Calibration of PMD-Distance Sensors , 2006, ISVC.

[4]  Christian Vogelgsang,et al.  Adaptive Lumigraph Rendering with Depth Maps , 2000 .

[5]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[6]  Stephen Lin,et al.  View-dependent displacement mapping , 2003, ACM Trans. Graph..

[7]  S. E. Ghobadi,et al.  First steps in enhancing 3D vision technique using 2D/3D sensors , 2006 .

[8]  J. M. Gerzso,et al.  Computer graphics and interactive techniques: 15th-17th July 1974. Boulder, Colorado, USA. Sponsored by the University of Colorado Computing Centre and ACM/SIGGRAPH , 1975, Comput. Aided Des..

[9]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[10]  Insung Ihm,et al.  Rendering of spherical light fields , 1997, Proceedings The Fifth Pacific Conference on Computer Graphics and Applications.

[11]  James F. Blinn,et al.  Texture and reflection in computer generated images , 1976, CACM.

[12]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.

[13]  G. Yahav,et al.  3D Imaging Camera for Gaming Application , 2007, 2007 Digest of Technical Papers International Conference on Consumer Electronics.

[14]  Ned Greene,et al.  Environment Mapping and Other Applications of World Projections , 1986, IEEE Computer Graphics and Applications.

[15]  Michael S. Landy,et al.  Computational models of visual processing , 1991 .

[16]  C. R. Hoffman,et al.  Illumination and Reflection Maps : Simulated Objects in Simulated and Real Environments Gene , 1984 .

[17]  Hans-Peter Seidel,et al.  Realistic, hardware-accelerated shading and lighting , 1999, SIGGRAPH.

[18]  Andreas Kolb,et al.  Sub-pixel data fusion and edge-enhanced distance refinement for 2D/3D images , 2008, Int. J. Intell. Syst. Technol. Appl..

[19]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[20]  Xavier Décoret,et al.  Rendering geometry with relief textures , 2006, Graphics Interface.

[21]  S. Burak Gokturk,et al.  A Time-Of-Flight Depth Sensor - System Description, Issues and Solutions , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[22]  Donald S. Fussell,et al.  Uniformly Sampled Light Fields , 1998, Rendering Techniques.

[23]  Pere-Pau Vázquez,et al.  Omni‐directional Relief Impostors , 2007, Comput. Graph. Forum.

[24]  K. Hartmann,et al.  Data-Fusion of PMD-Based Distance-Information and High-Resolution RGB-Images , 2007, 2007 International Symposium on Signals, Circuits and Systems.

[25]  Hans-Peter Seidel,et al.  Efficient Free Form Light Field Rendering , 2001, VMV.

[26]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[27]  Manuel Menezes de Oliveira Neto,et al.  Real-time relief mapping on arbitrary polygonal surfaces , 2005, I3D '05.

[28]  Leonard McMillan,et al.  Head-tracked stereoscopic display using image warping , 1995, Electronic Imaging.

[29]  Baining Guo,et al.  Capturing and rendering geometry details for BTF-mapped surfaces , 2005, The Visual Computer.

[30]  Manuel Menezes de Oliveira Neto,et al.  Relief texture mapping , 2000, SIGGRAPH.

[31]  Harry Shum,et al.  Image-based rendering , 2006, Found. Trends Comput. Graph. Vis..

[32]  Andreas Kolb,et al.  Real time fusion of range and light field images , 2005, SIGGRAPH '05.