Weighted feature extraction with a functional data extension

Dimensionality reduction has proved to be a beneficial tool in learning problems. Two of the main advantages provided by dimensionality reduction are interpretation and generalization. Typically, dimensionality reduction is addressed in two separate ways: variable selection and feature extraction. However, in the recent years there has been a growing interest in developing combined schemes such as feature extraction with built-in feature selection. In this paper, we look at dimensionality reduction as a rank-deficient problem that embraces variable selection and feature extraction, simultaneously. From our analysis, we derive a weighting algorithm that is able to select and linearly transform variables by fixing the dimensionality of the space where a relevance criterion is evaluated. This step enforces sparseness on the resulting weights. Our main goal is dimensionality reduction for classification problems. Namely, we introduce modified versions of principal component analysis (PCA) by expectation maximization (EM) and linear regularized discriminant analysis (RDA). Finally, we propose a simple extension of WRDA that deals with functional features. In this case, observations are described by a set of functions defined over the same domain. Methods were put to test on artificial and real data sets showing high levels of generalization even for small sized training samples.