Learning under differing training and test distributions

Abstract One of the main problems in machine learning is to train a predictive model from train-ing data and to make predictions on test data. Most predictive models are constructedunder the assumption that the training data is governed by the exact same distributionwhich the model will later be exposed to. In practice, control over the data collectionprocess is often imperfect. A typical scenario is when labels are collected by question-naires and one does not have access to the test population. For example, parts of thetest population are underrepresented in the survey, out of reach, or do not return thequestionnaire. In many applications training data from the test distribution are scarcebecause they are difficult to obtain or very expensive. Data from auxiliary sourcesdrawn from similar distributions are often cheaply available.This thesis centers around learning under differing training and test distributionsand covers several problem settings with different assumptions on the relationship be-tween training and test distributions—including multi-task learning and learning undercovariate shift and sample selection bias. Several new models are derived that directlycharacterize the divergence between training and test distributions, without the inter-mediate step of estimating training and test distributions separately. The integral partof these models are rescaling weights that match the rescaled or resampled trainingdistribution to the test distribution. Integrated models are studied where only one op-timization problem needs to be solved for learning under differing distributions. Witha two-step approximation to the integrated models almost any supervised learning al-gorithm can be adopted to biased training data.In case studies on spam filtering, HIV therapy screening, targeted advertising, andother applications the performance of the new models is compared to state-of-the-artreference methods.