Higher Order Functions for Kernel Regression

Kernel regression is a well-established nonparametric method, in which the target value of a query point is estimated using a weighted average of the surrounding training examples. The weights are typically obtained by applying a distance-based kernel function, which presupposes the existence of a distance measure. This paper investigates the use of Genetic Programming for the evolution of task-specific distance measures as an alternative to Euclidean distance. Results on seven real-world datasets show that the generalisation performance of the proposed system is superior to that of Euclidean-based kernel regression and standard GP.

[1]  Tina Yu,et al.  Hierarchical Processing for Evolving Recursive and Modular Programs Using Higher-Order Functions and Lambda Abstraction , 2001, Genetic Programming and Evolvable Machines.

[2]  Shiliang Sun,et al.  Kernel regression with sparse metric learning , 2013, J. Intell. Fuzzy Syst..

[3]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[4]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[5]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[6]  Peyman Milanfar,et al.  Robust Kernel Regression for Restoration and Reconstruction of Images from Sparse Noisy Data , 2006, 2006 International Conference on Image Processing.

[7]  Simon M. Lucas,et al.  Evolving Efficient Recursive Sorting Algorithms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[8]  Anthony Brabazon,et al.  Higher-order functions in aesthetic EC encodings , 2010, IEEE Congress on Evolutionary Computation.

[9]  Jan Larsen,et al.  Adaptive Metric Kernel Regression , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[10]  Kilian Q. Weinberger,et al.  Metric Learning for Kernel Regression , 2007, AISTATS.

[11]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[12]  Alexandros Agapitos,et al.  Adaptive Distance Metrics for Nearest Neighbour Classification Based on Genetic Programming , 2013, EuroGP.

[13]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[14]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.