Learning from Target Knowledge Approximation

The classical least squares error (LSE) learning method for pattern classification is to learn a classifier based on data density where the learning process (density-fitting error minimization) and the learning objective (classification error rate) do not find a good match. In this work, we propose to learn according to classification decision objectives directly. We shall work on two classification objectives namely, the total error rate and the receiver operating characteristics, and directly optimize the learning process according to these objectives. Using a learning model which is linear in its parameters, we propose two approximation methods to optimize these classification objectives. Our empirical results on biometrics fusion show comparable performances of the proposed methods with the widely used support vector machines (SVM), with one of the approaches having a clear advantage of fast single-step solution