Automated feature weighting in clustering with separable distances and inner product induced norms - A theoretical generalization

The feature weighting schemes for crisp and fuzzy clustering are generalized.Alternating optimization heuristics are derived for hard and fuzzy clustering.Existence of the minimizer of the proposed optimization tasks is proved.Prominent weighting schemes shown as special cases of the proposed generalization. For decades practitioners have been using the separable distance and inner product induced norms as the distance measures for k-means, Fuzzy C-Means (FCM), hard and fuzzy k-modes clustering algorithms. In this paper, we introduce a novel concept of automated feature weighting for general clustering algorithms (including both hard and fuzzy clustering) to amplify the effect of the discriminating features, which play a key role in identifying the naturally occurring groups in data with minimal computational overheads. We derive a Lloyd heuristic and an alternating optimization algorithm for solving the hard and the fuzzy clustering problems respectively. We also investigate the mathematical nature of the problems in sufficient details to guarantee the existence and feasibility of a solution at each iteration of the aforementioned algorithms. We show that majority of the automated feature weighting schemes existing in the literature turn out to be the special cases of this proposed generalization. A brief discussion on practical utility of the proposed generalization is also presented along with indication of the future applications of this new approach.