Image-assisted geometry simplification for the plenoptic sampling

In this paper, a new method is proposed to generate image-assisted geometry simplification for estimating minimum sampling rate of image-based rendering (IBR). If some geometry information (e.g., the shape of object surface and proxy geometry) on a scene is known, we can decompose the scene geometry into a collection of simpler structures on a block-by-block basis. Our framework predicts the characteristics of simpler structure such as irregular object. Predictions on the frequency content can then be used to control sampling rates for IBR. The new method allows the sampling of the IBR to be analyzed and estimated for the non-uniform sampling. Furthermore, the minimum sampling rate of the IBR necessary for alias-free rendering will be reduced as the number of simpler structures increases.

[1]  Tsuhan Chen,et al.  Spectral analysis for sampling image-based rendering data , 2003, IEEE Trans. Circuits Syst. Video Technol..

[2]  Mike Brookes,et al.  On the Spectrum of the Plenoptic Function , 2014, IEEE Transactions on Image Processing.

[3]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[4]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[5]  Long Quan,et al.  Image interpolation by joint view triangulation , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[6]  Harry Shum,et al.  Pop-up light field: An interactive image-based modeling and rendering system , 2004, TOGS.

[7]  Mike Brookes,et al.  Plenoptic Layer-Based Modeling for Image Based Rendering , 2013, IEEE Transactions on Image Processing.

[8]  Minh N. Do,et al.  Ieee Transactions on Image Processing on the Bandwidth of the Plenoptic Function , 2022 .

[9]  J. Berent,et al.  Plenoptic Manifolds , 2007, IEEE Signal Processing Magazine.

[10]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[11]  Wanqing Li,et al.  Objective evaluation of light field rendering methods using effective sampling density , 2011, 2011 IEEE 13th International Workshop on Multimedia Signal Processing.

[12]  Tsuhan Chen,et al.  A survey on image-based rendering - representation, sampling and compression , 2004, Signal Process. Image Commun..

[13]  Frédo Durand,et al.  A frequency analysis of light transport , 2005, SIGGRAPH '05.

[14]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.

[15]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[16]  Andrea Fusiello.  Image-based Rendering * , 2003 .

[17]  Ravi Ramamoorthi,et al.  A Fourier theory for cast shadows , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Tsuhan Chen,et al.  Light Field Sampling , 2006, Light Field Sampling.

[19]  Harry Shum,et al.  Rendering with concentric mosaics , 1999, SIGGRAPH.

[20]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[21]  Wanqing Li,et al.  Estimation of Signal Distortion Using Effective Sampling Density for Light Field-Based Free Viewpoint Video , 2015, IEEE Transactions on Multimedia.

[22]  George Drettakis,et al.  Silhouette‐Aware Warping for Image‐Based Rendering , 2011, Comput. Graph. Forum.

[23]  Boris Beizer,et al.  Black Box Testing: Techniques for Functional Testing of Software and Systems , 1996, IEEE Software.

[24]  Harry Shum,et al.  Image-based rendering , 2006, Found. Trends Comput. Graph. Vis..

[25]  S. B. Kang,et al.  Survey of image-based representations and compression techniques , 2003, IEEE Trans. Circuits Syst. Video Technol..

[26]  Wojciech Matusik,et al.  Moving gradients: a path-based method for plausible image interpolation , 2009, ACM Trans. Graph..

[27]  Tsuhan Chen,et al.  Nonuniform sampling of image-based rendering data with the position-interval-error (PIE) function , 2003, Visual Communications and Image Processing.

[28]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.