Sparse kernel methods

Abstract A disadvantage of many statistical modelling techniques is that the resulting model is extremely difficult to interpret. A number of new concepts and algorithms have been introduced by researchers to address this problem. They focus primarily on determining which inputs arc relevant in predicting the output. This work describes a transparent, nonlinear modelling approach that enables the constructed predictive models to be visualised, allowing model validation and assisting in interpretation. The technique combines the representational advantage of a sparse ANOVA decomposition, with the good generalisation ability of a kernel machine. It achieves this by employing two forms of rcgularisation: a I-norm based structural rcgulariser to enforce transparency, and a 2-norm based rcgulariser to control smoothness. The resulting model structure can be visualised showing the overall effects of different inputs, their interactions, and the strength of the interactions.