Transfer Learning with an Ensemble of Background Tasks

We demonstrate the transfer of learning from an ensemble of background tasks, which becomes helpful in cases where a single background task does not transfer well. This approach is accomplished through a simple maximum a posteriori elaboration on the logistic regression approach and tested on real world data. 1 Transfer Learning Via Learned Prior Distributions Transferring knowledge from a famil iar domain or task (call it task A) to an unfamiliar or newly-encountered one (task B) is a fundamental and fascinating aspect of human learning. Although the motivating notion is intuitive, the simple approach of treating the two tasks as identical and pooling their training data does not usually work well. This is presumably because the decision boundaries for A and B are not in exactly the same places in the feature space, even when the feature spaces and input distributions are themselves identical. Hence, more sophisticated methods are required. One interesting approach is to treat task A as defining a form of Bayesian prior distribution for task B. In this paper, we study this approach in a setting where we have many task As and only one task B, and we study whether we can learn a useful prior from those multiple task As that gives effective guidance when learning task B. Consider the well-known logistic regression model,