An improvement of incremental recursive fisher linear discriminant for online feature extraction

This paper proposes a new online feature extraction method called the Incremental Recursive Fisher Linear Discriminant (IRFLD), whose batch learning algorithm, referred to as RFLD, was proposed by Xiang and colleagues. In the conventional Linear Discriminant Analysis (LDA), the number of discriminant vectors is limited to the number of classes minus one due to the rank of the between-class covariance matrix. However, RFLD and the proposed IRFLD can break this limit; that is, an arbitrary number of discriminant vectors can be obtained. In the proposed IRFLD, the Incremental Linear Discriminant Analysis (ILDA) of Pang and colleagues is extended in such a way that effective discriminant vectors are recursively searched for in the complementary space of a conventional discriminant subspace. In addition, to estimate a suitable number of effective discriminant vectors, the classification accuracy is evaluated using the cross-validation method in an online manner. For this purpose, validation data are obtained by performing k-means clustering on incoming training data and the previous validation data. The performance of IRFLD is evaluated for 16 benchmark data sets. The experimental results show that the final classification accuracies of IRFLD are always better than those of ILDA. We also show that this performance improvement is attained by adding discriminant vectors in a complementary LDA subspace. © 2013 Wiley Periodicals, Inc. Electron Comm Jpn, 96(4): 29–40, 2013; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/ecj.10430