Multi-stage multi-task feature learning via adaptive threshold

Multi-task feature learning aims to identify the shared features among tasks to improve generalization. Recent works have shown that the non-convex learning model often returns a better solution than the convex alternatives. Thus a non-convex model based on the capped-1, 1 regularization was proposed in [1], and the corresponding efficient multi-stage multi-task feature learning algorithm (MSMTFL) was presented. However, this method harnesses a fixed threshold in the capped-1, 1 regularization. The lack of adaptivity might result in suboptimal practical performance. In this paper we propose to employ an adaptive threshold in the capped-1, 1 regularized formulation, and the corresponding variant of MSMTFL will incorporate an additional scheme to adaptively determine the threshold. Considering that this threshold aims to distinguish true nonzero components of large magnitude from others, the heuristic of detecting the "first significant jump" proposed in [2] is applied here to adaptively determine its value. The preliminary theoretical analysis is provided to guarantee the feasibility of the proposed method. Several numerical experiments demonstrate the proposed method outperforms existing state-of-the-art feature learning approaches.