Frequency Estimation of the Plenoptic Function Using the Autocorrelation Theorem

The frequency estimation of the plenoptic function (POF) is an important research topic in spectral analysis for determining the minimum sampling rate of image-based rendering. In this paper, we mathematically derive a frequency estimation function of the POF using the autocorrelation theory. The autocorrelation function (ACF) of the POF is studied along both the spatial and image plane frequency axes. The influence of the scene's complexity and depth on the ACF of the POF is analyzed. Furthermore, we study the frequency estimation error to analyze the performance of the method. Existing techniques typically use Fourier transformation to determine the frequency of the POF. The technique presented herein simply starts from measurements of the light field in a finite number of positions, and the information from the light field is analyzed in the spatial domain. Finally, experimental results are presented to demonstrate that the proposed method can effectively estimate the frequency of the POF. The results are shown to be in good agreement with traditional methods.

[1]  S. B. Kang,et al.  Survey of image-based representations and compression techniques , 2003, IEEE Trans. Circuits Syst. Video Technol..

[2]  Masayuki Tanimoto,et al.  Multiview Imaging and 3DTV , 2007, IEEE Signal Processing Magazine.

[3]  David Salesin,et al.  Surface light fields for 3D photography , 2000, SIGGRAPH.

[4]  Steven Kay,et al.  A Fast and Accurate Single Frequency Estimator , 2022 .

[5]  Michael P. Fitz,et al.  Further results in the fast estimation of a single frequency , 1994, IEEE Trans. Commun..

[6]  L. C. Palmer,et al.  Coarse frequency estimation using the discrete Fourier transform (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[7]  Minh N. Do,et al.  Ieee Transactions on Image Processing on the Bandwidth of the Plenoptic Function , 2022 .

[8]  J. Berent,et al.  Plenoptic Manifolds , 2007, IEEE Signal Processing Magazine.

[9]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[10]  Toshiaki Fujii,et al.  Free-Viewpoint TV , 2011, IEEE Signal Processing Magazine.

[11]  P. Vaidyanathan.  Multirate Systems And Filter Banks , 1992 .

[12]  Ha T. Nguyen,et al.  Immersive Visual Communication , 2011, IEEE Signal Processing Magazine.

[13]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[14]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.

[15]  Mike Brookes,et al.  On the Spectrum of the Plenoptic Function , 2014, IEEE Transactions on Image Processing.

[16]  Pascal Frossard,et al.  Plenoptic spherical sampling , 2012, 2012 19th IEEE International Conference on Image Processing.

[17]  Gavin S. P. Miller,et al.  Lazy Decompression of Surface Light Fields for Precomputed Global Illumination , 1998, Rendering Techniques.

[18]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[19]  Steven A. Tretter,et al.  Estimating the frequency of a noisy sinusoid by linear regression , 1985, IEEE Trans. Inf. Theory.

[20]  Harry Shum,et al.  On the number of samples needed in light field rendering with constant-depth assumption , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[21]  George Drettakis,et al.  Silhouette‐Aware Warping for Image‐Based Rendering , 2011, Comput. Graph. Forum.

[22]  Homer H. Chen,et al.  Light Field Analysis for Modeling Image Formation , 2011, IEEE Transactions on Image Processing.

[23]  Michael Mao Wang,et al.  An iterative algorithm for single-frequency estimation , 2002, IEEE Trans. Signal Process..

[24]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[25]  Richard H. Sherman,et al.  Chaotic communications in the presence of noise , 1993, Optics & Photonics.

[26]  Donald S. Fussell,et al.  Uniformly Sampled Light Fields , 1998, Rendering Techniques.

[27]  Umberto Mengali,et al.  Data-aided frequency estimation for burst digital transmission , 1997, IEEE Trans. Commun..

[28]  Dan Schonfeld,et al.  Geometrical plenoptic sampling , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[29]  A. Khintchine.  Korrelationstheorie der stationären stochastischen Prozesse , 1934 .

[30]  Mike Brookes,et al.  Plenoptic Layer-Based Modeling for Image Based Rendering , 2013, IEEE Transactions on Image Processing.

[31]  M.N. Do,et al.  Error Analysis for Image-Based Rendering With Depth Information , 2006, IEEE Transactions on Image Processing.

[32]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[33]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[34]  Tsuhan Chen,et al.  Spectral analysis for sampling image-based rendering data , 2003, IEEE Trans. Circuits Syst. Video Technol..

[35]  Andrew J. Viterbi,et al.  Principles of coherent communication , 1966 .

[36]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[37]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[38]  Tsuhan Chen,et al.  A survey on image-based rendering - representation, sampling and compression , 2004, Signal Process. Image Commun..

[39]  Harry Shum,et al.  Rendering with concentric mosaics , 1999, SIGGRAPH.

[40]  Leonard McMillan,et al.  A new reconstruction filter for undersampled light fields , 2003 .

[41]  N. Wiener.  Generalized harmonic analysis , 1930 .