Active Learning Ranking from Pairwise Preferences with Almost Optimal Query Complexity

Given a set V of n elements we wish to linearly order them using pairwise preference labels which may be non-transitive (due to irrationality or arbitrary noise). The goal is to linearly order the elements while disagreeing with as few pairwise preference labels as possible. Our performance is measured by two parameters: The number of disagreements (loss) and the query complexity (number of pairwise preference labels). Our algorithm adaptively queries at most O(n poly(log n, e-1)) preference labels for a regret of e times the optimal loss. This is strictly better, and often significantly better than what non-adaptive sampling could achieve. Our main result helps settle an open problem posed by learning-to-rank (from pairwise information) theoreticians and practitioners: What is a provably correct way to sample preference labels?

[1]  David Maxwell Chickering,et al.  Here or there: preference judgments for relevance , 2008 .

[2]  Sanjoy Dasgupta,et al.  Coarse sample complexity bounds for active learning , 2005, NIPS.

[3]  Andrew McCallum,et al.  Reducing Labeling Effort for Structured Prediction Tasks , 2005, AAAI.

[4]  Steve Hanneke,et al.  A bound on the label complexity of agnostic active learning , 2007, ICML '07.

[5]  John Langford,et al.  Error-Correcting Tournaments , 2009, ALT.

[6]  John Langford,et al.  Agnostic active learning , 2006, J. Comput. Syst. Sci..

[7]  HannekeSteve,et al.  The true sample complexity of active learning , 2010 .

[8]  Nir Ailon,et al.  Aggregating inconsistent information: Ranking and clustering , 2008 .

[9]  Yoram Singer,et al.  Learning to Order Things , 1997, NIPS.

[10]  Claire Mathieu,et al.  Electronic Colloquium on Computational Complexity, Report No. 144 (2006) How to rank with few errors A PTAS for Weighted Feedback Arc Set on Tournaments , 2006 .

[11]  Adam Tauman Kalai,et al.  Analysis of Perceptron-Based Active Learning , 2009, COLT.

[12]  Maria-Florina Balcan,et al.  The true sample complexity of active learning , 2010, Machine Learning.

[13]  H. Sebastian Seung,et al.  Selective Sampling Using the Query by Committee Algorithm , 1997, Machine Learning.

[14]  Sanjoy Dasgupta,et al.  A General Agnostic Active Learning Algorithm , 2007, ISAIM.

[15]  Eli Upfal,et al.  Computing with unreliable information , 1990, STOC '90.

[16]  Dan Roth,et al.  Margin-Based Active Learning for Structured Output Spaces , 2006, ECML.

[17]  Mark Braverman,et al.  Noisy sorting without resampling , 2007, SODA '08.

[18]  Anonymous Author.  Robust Reductions from Ranking to Classification , 2006 .

[19]  Eyke Hüllermeier,et al.  Label ranking by learning pairwise preferences , 2008, Artif. Intell..

[20]  Nir Ailon,et al.  A New Active Learning Scheme with Applications to Learning to Rank from Pairwise Preferences , 2011, ArXiv.

[21]  Mehryar Mohri,et al.  Preference-based learning to rank , 2010, Machine Learning.

[22]  R. Graham,et al.  Spearman's Footrule as a Measure of Disarray , 1977 .

[23]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[24]  Nir Ailon,et al.  Ranking from pairs and triplets: information quality, evaluation methods and query complexity , 2011, WSDM '11.

[25]  Noga Alon,et al.  Ranking Tournaments , 2006, SIAM J. Discret. Math..

[26]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[27]  Tie-Yan Liu,et al.  Listwise approach to learning to rank: theory and algorithm , 2008, ICML '08.