Reconstruction of nonuniformly sampled images in spline spaces

This paper presents a novel approach to the reconstruction of images from nonuniformly spaced samples. This problem is often encountered in digital image processing applications. Nonrecursive video coding with motion compensation, spatiotemporal interpolation of video sequences, and generation of new views in multicamera systems are three possible applications. We propose a new reconstruction algorithm based on a spline model for images. We use regularization, since this is an ill-posed inverse problem. We minimize a cost function composed of two terms: one related to the approximation error and the other related to the smoothness of the modeling function. All the processing is carried out in the space of spline coefficients; this space is discrete, although the problem itself is of a continuous nature. The coefficients of regularization and approximation filters are computed exactly by using the explicit expressions of B-spline functions in the time domain. The regularization is carried out locally, while the computation of the regularization factor accounts for the structure of the nonuniform sampling grid. The linear system of equations obtained is solved iteratively. Our results show a very good performance in motion-compensated interpolation applications.

[1]  Thomas Strohmer,et al.  Computationally attractive reconstruction of bandlimited images from irregular samples , 1997, IEEE Trans. Image Process..

[2]  Farrokh Marvasti,et al.  Motion compensation using spatial transformations with forward mapping , 1999, Signal Process. Image Commun..

[3]  James F. O'Brien,et al.  Shape transformation using variational implicit functions , 1999, SIGGRAPH Courses.

[4]  Amir A. Amini,et al.  Fast LV motion estimation using subspace approximation techniques , 2001, IEEE Transactions on Medical Imaging.

[5]  Peyman Milanfar,et al.  Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement , 2001, IEEE Trans. Image Process..

[6]  Richard G. Baraniuk,et al.  Interpolation and denoising of nonuniformly sampled data using wavelet-domain processing , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[7]  M. Unser,et al.  Interpolation Revisited , 2000, IEEE Trans. Medical Imaging.

[8]  D. Suter,et al.  Using a fast multipole method to accelerate spline evaluations , 1998 .

[9]  Kalpathi R. Subramanian,et al.  Interpolating implicit surfaces from scattered surface data using compactly supported radial basis functions , 2001, Proceedings International Conference on Shape Modeling and Applications.

[10]  William A. Pearlman,et al.  A full-featured, error-resilient, scalable wavelet video codec based on the set partitioning in hierarchical trees (SPIHT) algorithm , 2002, IEEE Trans. Circuits Syst. Video Technol..

[11]  Demetri Terzopoulos,et al.  The Computation of Visible-Surface Representations , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Akram Aldroubi,et al.  Nonuniform Sampling and Reconstruction in Shift-Invariant Spaces , 2001, SIAM Rev..

[13]  Michael Unser,et al.  Multigrid image reconstruction from arbitrarily spaced samples , 2002, Proceedings. International Conference on Image Processing.

[14]  Carlos Vázquez Reconstruction d'images à partir d'échantillons irrégulièrement espacés , 2002 .

[15]  Carlos Vázquez,et al.  Reconstruction of irregularly-sampled images by regularization in spline spaces , 2002, Proceedings. International Conference on Image Processing.

[16]  Ryszard Stasinski,et al.  Improved POCS reconstruction of stereoscopic views , 2002, Signal Process. Image Commun..

[17]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[18]  Torbjørn Eltoft,et al.  Adaptive regularized constrained least squares image restoration , 1999, IEEE Trans. Image Process..

[19]  Janusz Konrad Enhancement of viewer comfort in stereoscopic viewing: parallax adjustment , 1999, Electronic Imaging.

[20]  Konrad View reconstruction for 3-D video entertainment: issues, algorithms and applications , 1999 .

[21]  Youming Liu Irregular Sampling for Spline Wavelet Subspaces , 1996, IEEE Trans. Inf. Theory.

[22]  M. Unser,et al.  Interpolation revisited [medical images application] , 2000, IEEE Transactions on Medical Imaging.

[23]  Hans G. Feichtinger,et al.  Theory and practice of irregular sampling , 2021, Wavelets.

[24]  Ryszard Stasinski,et al.  Linear shift-variant filtering for POCS of reconstruction irregularly sampled images , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[25]  Michael Unser,et al.  Splines: a perfect fit for signal and image processing , 1999, IEEE Signal Process. Mag..

[26]  Michael Unser,et al.  B-spline signal processing. I. Theory , 1993, IEEE Trans. Signal Process..

[27]  Eric L. Miller,et al.  Wavelet domain image restoration with adaptive edge-preserving regularization , 2000, IEEE Trans. Image Process..

[28]  John C. Davis,et al.  Contouring: A Guide to the Analysis and Display of Spatial Data , 1992 .

[29]  Andrew S. Glassner,et al.  Principles of Digital Image Synthesis , 1995 .

[30]  Peyman Milanfar,et al.  A wavelet-based interpolation-restoration method for superresolution (wavelet superresolution) , 2000 .

[31]  Kalpathi R. Subramanian,et al.  Interpolating implicit surfaces from scattered surface data using compactly supported radial basis functions , 2001, Proceedings International Conference on Shape Modeling and Applications.

[32]  A. C. Faul,et al.  Proof of convergence of an iterative technique for thin plate spline interpolation in two dimensions , 1999, Adv. Comput. Math..

[33]  Sung Yong Shin,et al.  Scattered Data Interpolation with Multilevel B-Splines , 1997, IEEE Trans. Vis. Comput. Graph..

[34]  David Suter,et al.  Fast Multipole Method for Accelerating the Evaluation of Splines , 1998 .

[35]  Jan P. Allebach,et al.  Iterative reconstruction of bandlimited images from nonuniformly spaced samples , 1987 .

[36]  Richard Szeliski,et al.  Fast Surface Interpolation Using Hierarchical Basis Functions , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  David S. Taubman,et al.  Highly scalable video compression using a lifting-based 3D wavelet transform with deformable mesh motion compensation , 2002, ICIP.