Unsupervised Kernel Function Building Using Maximization of Information Potential Variability

We propose a kernel function estimation strategy to support machine learning tasks by analyzing the input samples using Renyi’s Information Metrics. Specifically, we aim to identify a Reproducing Kernel Hilbert Space spanning the most widely the information force among data points by the maximization of the information potential variability of Parzen-based pdf estimation. So, a Gaussian kernel bandwidth updating rule is obtained as a function of the forces induced by a given dataset. Our proposal is tested on synthetic and real-world datasets related to clustering and classification tasks. Obtained results show that presented approach allows to compute RKHS’s favoring data groups separability, attaining suitable learning performances in comparison with state of the art algorithms.

[1]  Weifeng Liu,et al.  Kernel Adaptive Filtering: A Comprehensive Introduction , 2010 .

[2]  Jose C. Principe,et al.  Information Theoretic Learning - Renyi's Entropy and Kernel Perspectives , 2010, Information Theoretic Learning.

[3]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[4]  Robert Jenssen,et al.  Information cut and information forces for clustering , 2003, 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No.03TH8718).

[5]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[6]  José Carlos Príncipe,et al.  Mixture kernel least mean square , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[7]  José Carlos Príncipe,et al.  Information Theoretic Learning with Infinitely Divisible Kernels , 2013, ICLR.

[8]  Badong Chen,et al.  An adaptive kernel width update for correntropy , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[9]  José Carlos Príncipe,et al.  Advanced search algorithms for information-theoretic learning with kernel-based estimators , 2004, IEEE Transactions on Neural Networks.

[10]  Weifeng Liu,et al.  Kernel Adaptive Filtering , 2010 .

[11]  Lluís A. Belanche Muñoz,et al.  Developments in kernel design , 2013, ESANN.

[12]  José Carlos Príncipe,et al.  Kernel width adaptation in information theoretic cost functions , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[14]  Hong Yu,et al.  Local density adaptive similarity measurement for spectral clustering , 2011, Pattern Recognit. Lett..