Fast Kernel Smoothing by a Low-Rank Approximation of the Kernel Toeplitz Matrix

Kernel smoothing methods, including the bilateral filter, are commonly used in data processing/modeling and edge-aware image smoothing. Due to their nonlinear nature, these filters require significant computational time. In this paper, we address this problem by studying a practical case in which the data to be processed are integers. The basic idea is to use eigendecomposition to approximate the kernel matrix which is a real symmetric Toeplitz matrix. This approximation leads to more efficient computation. We study the distribution of its eigenvalues and show that the upper bounds of the eigenvalues can be expressed analytically in terms of the Fourier transform of the kernel function. This result not only captures the relationship between the order of the low-rank approximation of the kernel matrix and the filtering quality, but also shows that among the three kernel functions considered in this work, the Gaussian kernel can be most efficiently approximated. We have applied the proposed fast algorithm to implement the bilateral filter. By taking advantage of a property of the Gaussian kernel, we have also proposed another algorithm with even faster speed. Experimental results show that the performance of the proposed algorithms is competitive with those state-of-the-art algorithms in terms of speed and quality.

[1]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[2]  W. Rey Introduction to Robust and Quasi-Robust Statistical Methods , 1983 .

[3]  Pierre Charbonnier,et al.  The Guided Bilateral Filter: When the Joint/Cross Bilateral Filter Becomes Robust , 2015, IEEE Transactions on Image Processing.

[4]  Leslie Greengard,et al.  The Fast Gauss Transform , 1991, SIAM J. Sci. Comput..

[5]  Larry S. Davis,et al.  Efficient Kernel Machines Using the Improved Fast Gauss Transform , 2004, NIPS.

[6]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[7]  Peyman Milanfar,et al.  A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical , 2013, IEEE Signal Processing Magazine.

[8]  Sei-ichiro Kamata,et al.  Compressive Bilateral Filtering , 2015, IEEE Transactions on Image Processing.

[9]  Bahadir K. Gunturk Fast bilateral filter with arbitrary range and domain kernels , 2010, ICIP.

[10]  Enhua Wu,et al.  Constant Time Weighted Median Filtering for Stereo Matching and Beyond , 2013, 2013 IEEE International Conference on Computer Vision.

[11]  Thomas Kailath,et al.  Fast reliable algorithms for matrices with structure , 1999 .

[12]  Pascal Getreuer,et al.  A Survey of Gaussian Convolution Algorithms , 2013, Image Process. Line.

[13]  Eugene E. Tyrtyshnikov,et al.  Spectra of multilevel toeplitz matrices: Advanced theory via simple matrix relationships , 1998 .

[14]  Michael Unser,et al.  Fast $O(1)$ Bilateral Filtering Using Trigonometric Range Kernels , 2011, IEEE Transactions on Image Processing.

[15]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[16]  Albrecht Böttcher,et al.  Spectral properties of banded Toeplitz matrices , 1987 .

[17]  F. Girosi Models of Noise and Robust Estimates , 1991 .

[18]  Qi Zhang,et al.  100+ Times Faster Weighted Median Filter (WMF) , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Alexander Belyaev,et al.  Fast Gauss Bilateral Filtering , 2010, Comput. Graph. Forum.

[20]  Toby P. Breckon,et al.  Constant-time bilateral filter using spectral decomposition , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[21]  Guang Deng Fast compressive bilateral filter , 2017 .

[22]  M. Kass,et al.  Smoothed local histogram filters , 2010, SIGGRAPH 2010.

[23]  Alan V. Oppenheim,et al.  Discrete-Time Signal Pro-cessing , 1989 .

[24]  F. Girosi Models of Noise and Robust Estimation , 1991 .

[25]  Jason Weston,et al.  Large-Scale Kernel Machines (Neural Information Processing) , 2007 .

[26]  Frédo Durand,et al.  Bilateral Filtering: Theory and Applications: Series: Foundations and Trends® in Computer Graphics and Vision , 2009 .

[27]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[28]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[29]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis: List of kernels , 2004 .

[30]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.

[31]  Michael Griebel,et al.  Fast Approximation of the Discrete Gauss Transform in Higher Dimensions , 2013, J. Sci. Comput..

[32]  Sei-ichiro Kamata,et al.  [Paper] Efficient Constant-time Gaussian Filtering with Sliding DCT/DST-5 and Dual-domain Error Minimization , 2015 .

[33]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, ECCV.

[34]  Xiaopeng Zhang,et al.  Accelerate bilateral filter using Hermite polynomials , 2014 .

[35]  Paulo J. S. G. Ferreira Localization of the Eigenvalues of Toeplitz Matrices using Additive Decomposition, Embedding in Circulants, and The Fourier Transform , 1994 .

[36]  E. E. Tyrtyshnikov A unifying approach to some old and new theorems on distribution and clustering , 1996 .

[37]  Ivan Markovsky,et al.  Low Rank Approximation - Algorithms, Implementation, Applications , 2018, Communications and Control Engineering.

[38]  Fatih Porikli,et al.  Constant time O(1) bilateral filtering , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.