Domain Transfer via Multiple Sources Regularization

The common assumption that training and testing samples share the same distribution is often violated in practice. When this happens, traditional learning models may not generalize well. To solve this problem, domain adaptation and transfer learning try to employ training data from other related source domains. We propose a multiple sources regularization framework for this problem. The framework extends classification model with regularization by adding a special regularization term, which penalizes the target classifier far from the convex combination of source classifiers. Then this framework guarantees the target classifier minimizes the empirical risk in target domain and the distance from the convex combination of source classifier simultaneously. By the way, the weights of the convex combination of source classifiers are embedded into the learning model as parameters, and will be learned through optimization algorithm automatically, which means our framework can identify similar or related domains adaptively. We apply our framework to SVM classification model and develop an optimization algorithm to solve this problem in iterative manner. Empirical study demonstrates the proposed algorithm outperforms some state-of-art related algorithms on real-world datasets, such as text categorization and optical recognition.