Function approximation with LWPR and XCSF: a comparative study

Function approximation, also called regression, is an important tool in numerical mathematics and engineering. The most challenging approximation problems arise, when the function class is unknown and the surface has to be approximated online from incoming samples. One way to achieve good approximations of complex non-linear functions is to cluster the input space into small patches, apply linear models in each niche, and recombine these models via a weighted sum. While it is rather simple to optimally fit a linear model to given data, it is fairly complex to find a reasonable structuring of the input space in order to exploit linearities in the underlying function. We compare two non-parametric regression algorithms that are able to approximate multi-dimensional, non-linear functions online. The XCSF Learning Classifier System is a modified version of XCS, which is a genetics-based machine learning algorithm. Locally Weighted Projection Regression is a statistics-based machine learning technique that is mainly used for function approximation tasks in robotics. For both algorithms the relevant, conflicting performance criteria are accuracy and population size, that is, the number of local models. We explore the trade-off between those criteria on three benchmark problems by means of intense grid search for Pareto optimal solutions. Detailed learning behavior is investigated using selected Pareto optimal parameters. The illustration of final input space clusterings sheds light on the structuring capabilities. A discussion of advantages and drawbacks completes this comparative study.