Eigen-analysis of kernel operators for nonlinear dimension reduction and discrimination

There has been growing interest in kernel methods for classification, clustering and dimension reduction. For example, kernel linear discriminant analysis, spectral clustering and kernel principal component analysis are widely used in statistical learning and data mining applications. The empirical success of the kernel method is generally attributed to nonlinear feature mapping induced by the kernel, which in turn determines a low dimensional data embedding. It is important to understand the effect of a kernel and its associated kernel parameter(s) on the embedding in relation to data distributions. In this dissertation, we examine the geometry of the nonlinear embeddings for kernel PCA and kernel LDA through spectral analysis of the corresponding kernel operators. In particular, we carry out eigenanalysis of the polynomial kernel operator associated with data distributions and investigate the effect of the degree of polynomial on the data embedding. We also investigate the effect of centering kernels on the spectral property of both polynomial and Gaussian kernel operators. In addition, we extend the framework of the eigen-analysis of kernel PCA to kernel LDA by considering between-class and within-class variation operators for polynomial kernels. The results provide both insights into the geometry of nonlinear data embeddings given by kernel methods and practical guidelines for choosing an appropriate degree for dimension reduction and discrimination with polynomial kernels. ii This is dedicated to my parents, Yigui Liang and Yunmei Hou; my sister, Zhiyan Liang and my husband, Sungmin Kim.