Small sets of random Fourier features by kernelized Matrix LVQ

Kernel based learning is very popular in machine learning but often quite costly with at least quadratic runtime complexity. Random Fourier features and related techniques have been proposed to provide an explicit kernel expansion such that standard techniques with low runtime and memory complexity can be used. This strategy leads to rather high dimensional datasets which is a drawback in many cases. Here, we combine a recently proposed unsupervised selection strategy for random Fourier features [1] with the very efficient supervised relevance learning given by Matrix LVQ. The suggested technique provides reasonable small but very discriminative features sets.1

[1]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[2]  Thomas Villmann,et al.  Limited Rank Matrix Learning, discriminative dimension reduction and visualization , 2012, Neural Networks.

[3]  Michael Biehl,et al.  Adaptive Relevance Matrices in Learning Vector Quantization , 2009, Neural Computation.

[4]  Frank-Michael Schleif,et al.  Metric and non-metric proximity transformations at linear costs , 2014, Neurocomputing.

[5]  James T. Kwok,et al.  Clustered Nyström Method for Large Scale Manifold Learning and Dimension Reduction , 2010, IEEE Transactions on Neural Networks.

[6]  Thomas Villmann,et al.  Prototype based fuzzy classification in clinical proteomics , 2008, Int. J. Approx. Reason..

[7]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[8]  Koby Crammer,et al.  Margin Analysis of the LVQ Algorithm , 2002, NIPS.

[9]  Barbara Hammer,et al.  Relevance determination in Learning Vector Quantization , 2001, ESANN.

[10]  Rong Jin,et al.  Efficient Kernel Clustering Using Random Fourier Features , 2012, 2012 IEEE 12th International Conference on Data Mining.

[11]  Thomas Villmann,et al.  Stationarity of Matrix Relevance LVQ , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[12]  Frank-Michael Schleif,et al.  Low-Rank Kernel Space Representations in Prototype Learning , 2016, WSOM.

[13]  Rasmus Pagh,et al.  Fast and scalable polynomial kernels via explicit feature maps , 2013, KDD.

[14]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[15]  Peter Tiño,et al.  Finding Small Sets of Random Fourier Features for Shift-Invariant Kernel Approximation , 2016, ANNPR.

[16]  Frank-Michael Schleif,et al.  Learning interpretable kernelized prototype-based models , 2014, Neurocomputing.

[17]  Inderjit S. Dhillon,et al.  Memory Efficient Kernel Approximation , 2014, ICML.

[18]  Teuvo Kohonen,et al.  Self-Organizing Maps, Second Edition , 1997, Springer Series in Information Sciences.

[19]  Thomas Villmann,et al.  Kernelized vector quantization in gradient-descent learning , 2015, Neurocomputing.

[20]  Thomas Villmann,et al.  Efficient Kernelized Prototype Based Classification , 2011, Int. J. Neural Syst..

[21]  Frank-Michael Schleif,et al.  Fast approximated relational and kernel clustering , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[22]  Thomas Villmann,et al.  Margin based Active Learning for LVQ Networks , 2007, ESANN.

[23]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.