Active Online Domain Adaptation

Online machine learning systems need to adapt to domain shifts. Meanwhile, acquiring label at every timestep is expensive. We propose a surprisingly simple algorithm that adaptively balances its regret and its number of label queries in settings where the data streams are from a mixture of hidden domains. For online linear regression with oblivious adversaries, we provide a tight tradeoff that depends on the durations and dimensionalities of the hidden domains. Our algorithm can adaptively deal with interleaving spans of inputs from different domains. We also generalize our results to non-linear regression for hypothesis classes with bounded eluder dimension and adaptive adversaries. Experiments on synthetic and realistic datasets demonstrate that our algorithm achieves lower regret than uniform queries and greedy queries with equal labeling budget.

[1]  Morteza Zadimoghaddam,et al.  Trading off Mistakes and Don't-Know Predictions , 2010, NIPS.

[2]  Amit Daniely,et al.  Strongly Adaptive Online Learning , 2015, ICML.

[3]  Daumé,et al.  Domain Adaptation meets Active Learning , 2010, HLT-NAACL 2010.

[4]  Benjamin Recht,et al.  Do ImageNet Classifiers Generalize to ImageNet? , 2019, ICML.

[5]  Steve Hanneke.  Rates of convergence in active learning , 2011, 1103.1790.

[6]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[7]  Claudio Gentile,et al.  On the generalization ability of on-line learning algorithms , 2001, IEEE Transactions on Information Theory.

[8]  Claudio Gentile,et al.  Worst-Case Analysis of Selective Sampling for Linear Classification , 2006, J. Mach. Learn. Res..

[9]  John Langford,et al.  Agnostic Active Learning Without Constraints , 2010, NIPS.

[10]  Manfred K. Warmuth,et al.  Reverse iterative volume sampling for linear regression , 2018, J. Mach. Learn. Res..

[11]  Benjamin Recht,et al.  Do CIFAR-10 Classifiers Generalize to CIFAR-10? , 2018, ArXiv.

[12]  Shai Ben-David,et al.  Hierarchical Label Queries with Data-Dependent Partitions , 2015, COLT.

[13]  Steve Hanneke,et al.  Theory of Disagreement-Based Active Learning , 2014, Found. Trends Mach. Learn..

[14]  Elad Hazan,et al.  Logarithmic regret algorithms for online convex optimization , 2006, Machine Learning.

[15]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.

[16]  Csaba Szepesvári,et al.  Agnostic KWIK learning and efficient approximate reinforcement learning , 2011, COLT.

[17]  Sanjoy Dasgupta,et al.  Hierarchical sampling for active learning , 2008, ICML '08.

[18]  Manfred K. Warmuth,et al.  Leveraged volume sampling for linear regression , 2018, NeurIPS.

[19]  Haipeng Luo,et al.  Equipping Experts/Bandits with Long-term Memory , 2019, NeurIPS.

[20]  Kamalika Chaudhuri,et al.  The Extended Littlestone's Dimension for Learning with Mistakes and Abstentions , 2016, COLT.

[21]  Alexandra Carpentier,et al.  Adaptivity to Noise Parameters in Nonparametric Active Learning , 2017, COLT.

[22]  Taesung Park,et al.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation , 2017, ICML.

[23]  Nagarajan Natarajan,et al.  Active Heteroscedastic Regression , 2017, ICML.

[24]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[25]  Gábor Lugosi,et al.  Minimizing Regret with Label Efficient Prediction , 2004, COLT.

[26]  Rémi Munos,et al.  Active Regression by Stratification , 2014, NIPS.

[27]  Elad Hazan,et al.  Introduction to Online Convex Optimization , 2016, Found. Trends Optim..

[28]  Avishek Saha,et al.  Active Supervised Domain Adaptation , 2011, ECML/PKDD.

[29]  Sham M. Kakade,et al.  Convergence Rates of Active Learning for Maximum Likelihood Estimation , 2015, NIPS.

[30]  Manfred K. Warmuth,et al.  Relative Loss Bounds for On-Line Density Estimation with the Exponential Family of Distributions , 1999, Machine Learning.

[31]  Claudio Gentile,et al.  Learning noisy linear classifiers via adaptive and selective sampling , 2011, Machine Learning.

[32]  Csaba Szepesvári,et al.  Improved Algorithms for Linear Stochastic Bandits , 2011, NIPS.

[33]  Manfred K. Warmuth,et al.  Tracking a Small Set of Experts by Mixing Past Posteriors , 2003, J. Mach. Learn. Res..

[34]  Stanislav Minsker,et al.  Plug-in Approach to Active Learning , 2011, J. Mach. Learn. Res..

[35]  S. Muthukrishnan,et al.  Sampling algorithms for l2 regression and applications , 2006, SODA '06.

[36]  Wouter M. Kouw.  An introduction to domain adaptation and transfer learning , 2018, ArXiv.

[37]  Thomas J. Walsh,et al.  Knows what it knows: a framework for self-aware learning , 2008, ICML '08.

[38]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[39]  Vianney Perchet,et al.  Active Linear Regression , 2019, ArXiv.

[40]  Alekh Agarwal,et al.  Selective sampling algorithms for cost-sensitive multiclass prediction , 2013, ICML.

[41]  Sanjoy Dasgupta,et al.  A General Agnostic Active Learning Algorithm , 2007, ISAIM.

[42]  Qiang Yang,et al.  A Survey of Transfer and Multitask Learning in Bioinformatics , 2011, J. Comput. Sci. Eng..

[43]  Claudio Gentile,et al.  Robust bounds for classification via selective sampling , 2009, ICML '09.

[44]  V. Vovk.  Competitive On‐line Statistics , 2001 .

[45]  Eliza Strickland.  AI-human partnerships tackle "fake news": Machine learning can get you only so far-then human judgment is required - [News] , 2018 .

[46]  Thomas P. Hayes,et al.  High-Probability Regret Bounds for Bandit Online Linear Optimization , 2008, COLT.

[47]  Min Xiao,et al.  Online Active Learning for Cost Sensitive Domain Adaptation , 2013, CoNLL.

[48]  John Langford,et al.  Agnostic active learning , 2006, J. Comput. Syst. Sci..

[49]  Liu Yang,et al.  Active Learning with a Drifting Distribution , 2011, NIPS.

[50]  Aleksander Madry,et al.  Identifying Statistical Bias in Dataset Replication , 2020, ICML.

[51]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[52]  Seshadhri Comandur,et al.  Electronic Colloquium on Computational Complexity, Report No. 88 (2007) Adaptive Algorithms for Online Decision Problems , 2022 .

[53]  Benjamin Van Roy,et al.  Eluder Dimension and the Sample Complexity of Optimistic Exploration , 2013, NIPS.