Coordinated Online Learning With Applications to Learning User Preferences

We study an online multi-task learning setting, in which instances of related tasks arrive sequentially, and are handled by task-specific online learners. We consider an algorithmic framework to model the relationship of these tasks via a set of convex constraints. To exploit this relationship, we design a novel algorithm -- COOL -- for coordinating the individual online learners: Our key idea is to coordinate their parameters via weighted projections onto a convex set. By adjusting the rate and accuracy of the projection, the COOL algorithm allows for a trade-off between the benefit of coordination and the required computation/communication. We derive regret bounds for our approach and analyze how they are influenced by these trade-off factors. We apply our results on the application of learning users' preferences on the Airbnb marketplace with the goal of incentivizing users to explore under-reviewed apartments.

[1]  Inderjit S. Dhillon,et al.  The Metric Nearness Problem , 2008, SIAM J. Matrix Anal. Appl..

[2]  Ohad Shamir,et al.  Distributed stochastic optimization and learning , 2014, 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[3]  Mladen Kolar,et al.  Distributed Multi-Task Learning , 2016, AISTATS.

[4]  Andreas Krause,et al.  Truthful incentives in crowdsourcing tasks using regret minimization mechanisms , 2013, WWW.

[5]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[6]  Feng Yan,et al.  Distributed Autonomous Online Learning: Regrets and Intrinsic Privacy-Preserving Properties , 2010, IEEE Transactions on Knowledge and Data Engineering.

[7]  Lorenzo Rosasco,et al.  Convex Learning of Multiple Tasks and their Structure , 2015, ICML.

[8]  Ya Zhang,et al.  Multi-task learning for boosting with application to web search ranking , 2010, KDD.

[9]  Ohad Shamir,et al.  Optimal Distributed Online Prediction Using Mini-Batches , 2010, J. Mach. Learn. Res..

[10]  Peter L. Bartlett,et al.  Multitask Learning with Expert Advice , 2007, COLT.

[11]  Kristen Grauman,et al.  Learning with Whom to Share in Multi-task Feature Learning , 2011, ICML.

[12]  Gábor Lugosi,et al.  Online Multi-task Learning with Hard Constraints , 2009, COLT.

[13]  Pravesh Kothari,et al.  25th Annual Conference on Learning Theory Differentially Private Online Learning , 2022 .

[14]  Avishek Saha,et al.  Online Learning of Multiple Tasks and Their Relationships , 2011, AISTATS.

[15]  Fuzhen Zhuang,et al.  Collaborating between Local and Global Learning for Distributed Online Multiple Tasks , 2015, CIKM.

[16]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[17]  Andreas Krause,et al.  Actively Learning Hemimetrics with Applications to Eliciting User Preferences , 2016, ICML.

[18]  Andreas Krause,et al.  Incentivizing Users for Balancing Bike Sharing Systems , 2015, AAAI.

[19]  Emir Kamenica,et al.  Bayesian Persuasion , 2009 .

[20]  Philip M. Long,et al.  Online Learning of Multiple Tasks with a Shared Loss , 2007, J. Mach. Learn. Res..

[21]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[22]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[23]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[24]  Maria-Florina Balcan,et al.  Distributed Learning, Communication Complexity and Privacy , 2012, COLT.

[25]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.