Unsupervised Relevance Analysis for Feature Extraction and Selection - A Distance-based Approach for Feature Relevance

The aim of this paper is to propose a new generalized formulat ion for feature extraction based on distances from a feature relevance point of view. This is done within an unsupervised framework. To do so, it is first outlined the formal concept of feature relevance. Then, a no vel feature extraction approach is introduced. Such an approach employs the M-norm as a distance measure. It is de monstrated that under some conditions, this method can readily explain literature methods. As another c ontribution of this paper, we propose an elegant feature ranking approach for feature selection followed fr om the spectral analysis of the data variability. Also, we provide a weighted PCA scheme revealing the relationship between feature extraction and feature selection. To assess the behavior of the studied methods within a patter n recognition system, a clustering stage is carried out. Normalized mutual information is used to quantify the q uality of resultant clusters. Proposed methods reach comparable results with respect to literature method s.