An RBF-based compression method for image-based relighting

In image-based relighting, a pixel is associated with a number of sampled radiance values. This paper presents a two-level compression method. In the first level, the plenoptic property of a pixel is approximated by a spherical radial basis function (SRBF) network. That means that the spherical plenoptic function of each pixel is represented by a number of SRBF weights. In the second level, we apply a wavelet-based method to compress these SRBF weights. To reduce the visual artifact due to quantization noise, we develop a constrained method for estimating the SRBF weights. Our proposed approach is superior to JPEG, JPEG2000, and MPEG. Compared with the spherical harmonics approach, our approach has a lower complexity, while the visual quality is comparable. The real-time rendering method for our SRBF representation is also discussed.

[1]  Gene H. Golub,et al.  Matrix computations , 1983 .

[2]  K. H. Barratt.  Digital Coding of Waveforms , 1985 .

[3]  Peter No,et al.  Digital Coding of Waveforms , 1986 .

[4]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[5]  Nelson L. Max,et al.  Bidirectional reflection functions from surface bump maps , 1987, SIGGRAPH.

[6]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[7]  Roy Hall,et al.  Illumination and Color in Computer Generated Imagery , 1988, Monographs in Visual Communication.

[8]  A. C. Tsoi.  Multilayer perceptron trained using radial basis functions , 1989 .

[9]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[10]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[11]  Stephen H. Westin,et al.  A global illumination solution for general reflectance distributions , 1991, SIGGRAPH.

[12]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[13]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1992 .

[14]  Russell A. Epstein,et al.  5/spl plusmn/2 eigenimages suffice: an empirical investigation of low-dimensional lighting models , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[15]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[16]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[17]  Rick L. Jenison,et al.  A Spherical Basis Function Neural Network for Modeling Auditory Space , 1996, Neural Computation.

[18]  Jyh-Shin Pan Jyh-Shin Pan,et al.  Video Encoder Architecture for MPEG2 Real Time Encoding , 1996, 1996. Digest of Technical Papers., International Conference on Consumer Electronics.

[19]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[20]  Tien-Tsin Wong,et al.  Image-based Rendering with Controllable Illumination , 1997, Rendering Techniques.

[21]  Eero P. Simoncelli,et al.  Progressive wavelet image coding based on a conditional probability model , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[22]  Tien-Tsin Wong,et al.  Sampling with Hammersley and Halton Points , 1997, J. Graphics, GPU, & Game Tools.

[23]  Asim Roy,et al.  A neural-network learning theory and a polynomial time RBF algorithm , 1997, IEEE Trans. Neural Networks.

[24]  Ah Chung Tsoi,et al.  Recurrent neural networks: A constructive algorithm, and its properties , 1997, Neurocomputing.

[25]  Harry Shum,et al.  Rendering with concentric mosaics , 1999, SIGGRAPH.

[26]  Wolfgang Straßer,et al.  The Wavelet Stream – Progressive Transmission of Compressed Light Field Data , 1999 .

[27]  Jin Li,et al.  Compression of Lumigraph with multiple reference frame (MRF) prediction and just-in-time rendering , 2000, Proceedings DCC 2000. Data Compression Conference.

[28]  Tsuhan Chen,et al.  Compression with mosaic prediction for image-based rendering applications , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[29]  Marcus A. Magnor,et al.  Data compression for light-field rendering , 2000, IEEE Trans. Circuits Syst. Video Technol..

[30]  Zixiang Xiong,et al.  Low bit-rate scalable video coding with 3-D set partitioning in hierarchical trees (3-D SPIHT) , 2000, IEEE Trans. Circuits Syst. Video Technol..

[31]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[32]  Bryan Usevitch,et al.  A tutorial on modern lossy wavelet image compression: foundations of JPEG 2000 , 2001, IEEE Signal Process. Mag..

[33]  Insung Ihm,et al.  3D RGB image compression for interactive applications , 2001, TOGS.

[34]  M.D. Adams,et al.  Wavelet transforms in the JPEG-2000 standard , 2001, 2001 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (IEEE Cat. No.01CH37233).

[35]  Chi-Wing Fu,et al.  Interactive Relighting of Panoramas , 2001, IEEE Computer Graphics and Applications.

[36]  Andrew Chi-Sing Leung,et al.  The plenoptic illumination function , 2002, IEEE Trans. Multim..

[37]  Pat Hanrahan,et al.  Frequency space environment map rendering , 2002, SIGGRAPH.

[38]  Jan Kautz,et al.  Fast Arbitrary BRDF Shading for Low-Frequency Lighting Using Spherical Harmonics , 2002, Rendering Techniques.

[39]  Paul E. Debevec.  Image-Based Lighting , 2002, IEEE Computer Graphics and Applications.

[40]  Peter-Pike J. Sloan,et al.  Clustered principal components for precomputed radiance transfer , 2003, ACM Trans. Graph..

[41]  Chi-Wing Fu,et al.  Real-time relighting of compressed panoramas , 2003 .

[42]  Harry Shum,et al.  Bi-scale radiance transfer , 2003, ACM Trans. Graph..

[43]  Randima Fernando,et al.  The CG Tutorial: The Definitive Guide to Programmable Real-Time Graphics , 2003 .

[44]  Andrew Chi-Sing Leung,et al.  Compression of illumination-adjustable images , 2003, IEEE Trans. Circuits Syst. Video Technol..

[45]  Jan Kautz,et al.  Hardware Lighting and Shading: a Survey , 2004, Comput. Graph. Forum.