High Performance Kernel Smoothing Library For Biomedical Imaging

of the Thesis High Performance Kernel Smoothing Library For Biomedical Imaging by Haofu Liao Master of Science in Electrical and Computer Engineering Northeastern University, May 2015 Dr. Deniz Erdogmus, Adviser The estimation of probability density and probability density derivatives has full potential for applications. In biomedical imaging, the estimation of the first and second derivatives of the density is crucial to extract tubular structures, such as blood vessels and neuron traces. Probability density and probability density derivatives are often estimated using nonparametric data-driven methods. One of the most popular nonparametric methods is the Kernel Density Estimation (KDE) and Kernel Density Derivative Estimation (KDDE). However, a very serious drawback of using KDE and KDDE is the intensive computational requirements, especially for large data sets. In this thesis, we develop a high performance kernel smoothing library to accelerate KDE and KDDE methods. A series of hardware optimizations are used to deliver a high performance code. On the host side, multi-core platforms and parallel programming frameworks are used to accelerate the execution of the library. For 2 or 3-dimensional data points, the Graphic Processing Unit (GPU) platform is used to provide high levels of performance to the kernel density estimators, kernel gradient estimators as well as the kernel curvature estimators. Several Compute Unified Device Architecture (CUDA) based techniques are used to optimize their performances. What’s more, a graph-based algorithm is designed to calculate the derivatives efficiently and a fast k-nearest neighbor bandwidth selector is designed to speed up the variable bandwidth selection for image data on GPU.

[1]  B. Silverman,et al.  Algorithm AS 176: Kernel Density Estimation Using the Fast Fourier Transform , 1982 .

[2]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 1991 .

[3]  Jack J. Dongarra,et al.  A hybrid Hermitian general eigenvalue solver , 2012, ArXiv.

[4]  Y. Katznelson An Introduction to Harmonic Analysis: Interpolation of Linear Operators , 1968 .

[5]  M. Wand Fast Computation of Multivariate Kernel Estimators , 1994 .

[6]  Jeff M. Phillips,et al.  Є-Samples for Kernels , 2013, SODA.

[7]  Konstantinos G. Margaritis,et al.  Accelerating Kernel Density Estimation on the GPU Using the CUDA Framework , 2013 .

[8]  Jeffrey S. Racine,et al.  Parallel distributed kernel estimation , 2002 .

[9]  M. Rosenblatt Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[10]  D. W. Scott,et al.  Variable Kernel Density Estimation , 1992 .

[11]  A. Bowman,et al.  Applied smoothing techniques for data analysis : the kernel approach with S-plus illustrations , 1999 .

[12]  Konstantinos G. Margaritis,et al.  Parallel Computing of Kernel Density Estimation with Different Multi-core Programming Models , 2013, 2013 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing.

[13]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[14]  Tarn Dunong Bandwidth selectors for multivariate kernel density estimation , 2005, Bulletin of the Australian Mathematical Society.

[15]  Deniz Erdogmus,et al.  Locally Defined Principal Curves and Surfaces , 2011, J. Mach. Learn. Res..

[16]  Deniz Erdogmus,et al.  Computationally Efficient Exact Calculation of Kernel Density Derivatives , 2015, J. Signal Process. Syst..

[17]  T. Duong,et al.  Data-driven density derivative estimation, with applications to nonparametric clustering and bump hunting , 2012, 1204.6160.

[18]  T. Apostol Mathematical Analysis , 1957 .

[19]  Victor Y. Pan The Trade-Off Between the Additive Complexity and the Asynchronicity of Linear and Bilinear Algorithms , 1986, Inf. Process. Lett..

[20]  Ian Abramson On Bandwidth Variation in Kernel Estimates-A Square Root Law , 1982 .

[21]  Deniz Erdogmus,et al.  Principal Curves as Skeletons of Tubular Objects , 2011, Neuroinformatics.

[22]  Jeffrey S. Racine,et al.  Nonparametric Econometrics: The np Package , 2008 .

[23]  Larry S. Davis,et al.  Improved fast gauss transform and efficient kernel density estimation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  Szymon Łukasik,et al.  Parallel Computing of Kernel Density Estimates with MPI , 2007 .

[25]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[26]  Tarn Duong,et al.  ks: Kernel Density Estimation and Kernel Discriminant Analysis for Multivariate Data in R , 2007 .

[27]  Witold Andrzejewski,et al.  Graphics processing units in acceleration of bandwidth selection for kernel density estimation , 2013, Int. J. Appl. Math. Comput. Sci..

[28]  E. Parzen On Estimation of a Probability Density Function and Mode , 1962 .

[29]  Inge Koch,et al.  Feature significance for multivariate kernel density estimation , 2008, Comput. Stat. Data Anal..

[30]  Larry S. Davis,et al.  Efficient Kernel Density Estimation Using the Fast Gauss Transform with Applications to Color Modeling and Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Feifei Li,et al.  Quality and efficiency for kernel density estimates in large data , 2013, SIGMOD '13.

[32]  S. R. Searle,et al.  Vec and vech operators for matrices, with some uses in jacobians and multivariate statistics , 1979 .

[33]  M. Wand,et al.  ASYMPTOTICS FOR GENERAL MULTIVARIATE KERNEL DENSITY DERIVATIVE ESTIMATORS , 2011 .

[34]  Alejandro F. Frangi,et al.  Muliscale Vessel Enhancement Filtering , 1998, MICCAI.

[35]  Arnab Sinha,et al.  Fast Estimation of Nonparametric Kernel Density Through PDDP, and its Application in Texture Synthesis , 2008, BCS Int. Acad. Conf..

[36]  V. A. Epanechnikov Non-Parametric Estimation of a Multivariate Probability Density , 1969 .

[37]  L. Breiman,et al.  Variable Kernel Estimates of Multivariate Densities , 1977 .

[38]  M. C. Jones Variable kernel density estimates and variable kernel density estimates , 1990 .

[39]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.