Plenoptic sampling

This paper studies the problem of plenoptic sampling in image-based rendering (IBR). From a spectral analysis of light field signals and using the sampling theorem, we mathematically derive the analytical functions to determine the minimum sampling rate for light field rendering. The spectral support of a light field signal is bounded by the minimum and maximum depths only, no matter how complicated the spectral support might be because of depth variations in the scene. The minimum sampling rate for light field rendering is obtained by compacting the replicas of the spectral support of the sampled light field within the smallest interval. Given the minimum and maximum depths, a reconstruction filter with an optimal and constant depth can be designed to achieve anti-aliased light field rendering. Plenoptic sampling goes beyond the minimum number of images needed for anti-aliased light field rendering. More significantly, it utilizes the scene depth information to determine the minimum sampling curve in the joint image and geometry space. The minimum sampling curve quantitatively describes the relationship among three key elements in IBR systems: scene complexity (geometrical and textural information), the number of image samples, and the output resolution. Therefore, plenoptic sampling bridges the gap between image-based rendering and traditional geometry-based rendering. Experimental results demonstrate the effectiveness of our approach.

[1]  Donald S. Fussell,et al.  Uniformly Sampled Light Fields , 1998, Rendering Techniques.

[2]  Sing Bing Kang,et al.  Survey of image-based rendering techniques , 1998, Electronic Imaging.

[3]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[4]  Anselmo Lastra,et al.  LDI tree: a hierarchical representation for image-based rendering , 1999, SIGGRAPH.

[5]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[6]  Peter-Pike J. Sloan,et al.  Time critical lumigraph rendering , 1997, SI3D.

[7]  오승준.  [서평]「Digital Video Processing」 , 1996 .

[8]  B. Girod,et al.  Motion Compensation: Visual Aspects, Accuracy, and Fundamental Limits , 1993 .

[9]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[10]  Hans-Peter Seidel,et al.  Adaptive Acquisition of Lumigraphs from Synthetic Scenes , 1999, Comput. Graph. Forum.

[11]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[12]  Michael W. Halle.  Holographic stereograms as discrete imaging systems , 1994, Electronic Imaging.

[13]  Harry Shum,et al.  Parallel projections for stereo reconstruction , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Václav Hlavác,et al.  Automatic Selection of Reference Views for Image-Based Scene Representations , 1996, ECCV.

[15]  M. E. Cox.  Handbook of Optics , 1980 .

[16]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[17]  Harry Shum,et al.  On the number of samples needed in light field rendering with constant-depth assumption , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  Heung-Yeung Shum,et al.  An Optical Analysis of Light Field Rendering , 2002 .

[19]  Jed Lengyel,et al.  The Convergence of Graphics and Vision , 1998, Computer.

[20]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[21]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[22]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[23]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[24]  Harry Shum,et al.  Rendering with concentric mosaics , 1999, SIGGRAPH.