A constrained least squares regression model

Abstract Least squares regression (LSR) is a widely used regression technique for multicategory classification. Conventional LSR model assumes that during the learning phase, the labeled samples can be exactly transformed into a discrete label matrix, which is too strict to learn a regression matrix for fitting the labels. To overcome this drawback, lots of LSR’s variants utilize the soft target label, which contains the continuous values, to replace this discrete label to improve the learning performance. Since the regression matrix can be learnt from these soft target labels, it is reasonable to assume that the samples in the same class have similar soft target labels. Nevertheless, most of existing LSR-based models don’t adequately consider this similarity assumption. In this paper, we propose a constrained least squares regression (CLSR) model for multicategory classification. The main motivation of CLSR is to force the samples in the same class to obtain the similar soft target labels. To effectively optimize CLSR, we propose a novel alternating algorithm, which can converge to the globally optimal solution. Extensive experiments results on face and digit data sets confirm the effectiveness of our proposed model.