Discriminative deep semi-nonnegative matrix factorization network with similarity maximization for unsupervised feature learning

Abstract Deep Semi-NMF (DSN), which learns hierarchical representations by stacking multiple layers Semi-NMF, shows competitive performance in unsupervised data analysis. However, the features learned from DSN always lack of representativity and discriminativity. In this paper, we build a novel Deep Semi-NMF network (DSNnet) to address the issues of DSN. Specifically, DSNnet contains multiple fully-connected layers, in which the activation function of each layer adopts Smoothly Clipped Absolute Deviation (SCAD). The non-negative hidden features are computed forwardly, while the network parameters are updated by the stochastic gradient descent method. Moreover, to enhance the discriminativity of features, we suggest simultaneously minimizing the reconstruction error of input and output, and maximizing the similarity between input and learned features. The proposed similarity measurement, which consists of global geometric similarity and local pointwise similarity, encourages the compactness between similar points and separateness between dissimilar points in the feature space, and is beneficial to preserve intrinsic information of original data. Extensive experiments conducted on several datasets illustrate the superiority of the proposed approach in comparison with state-of-the-art methods.