Kernel Affine Projection Algorithm

The unknown system to be identified by an adaptive filter is usually assumed to be a linear system. Based on this assumption, we model the unknown system by a linear filter. In reality, however, there are cases where a linear filter is inadequate. In spite of this problem, using a very general nonlinear filter is not a good idea. To construct an adaptation algorithm for a general nonlinear filter is not simple. Moreover, an adaptive model that has too many free parameters is not desirable from a machine learning theoretic point of view, because such a model exhibits poor generalization. In this chapter, we make a review of a work that extends the APA by the kernel trick so that it is applicable to identification of a nonlinear system. We start with the kernel perceptron as a simple example to show how the kernel trick is used to extend the perceptron so that it can learn a nonlinear discriminant function without losing the simplicity of the original linear structure. Then, noting that the kernel trick replaces the inner product with the kernel function, we extend the APA to the kernel APA. It is seen that the kernel APA has a similar structure with the resource-allocating network. In the perceptron, the training data set is finite and fixed. In the APA, on the other hand, the set of training data, i.e., the set of regressors, is infinite. To keep the set of regressors actually used in adaptation to be finite, we sieve the regressors by the novelty criterion that checks if a newly arrived regressor is informative enough for adaptation.