Learning and Optimizing with Preferences

Preferences and choices are a central source of information generated by humans. They have been studied for centuries in the context of social choice theory, econometric theory, statistics and psychology. At least two Nobel prizes in economics have been awarded for work reasoning about human preferences and choices. In the last two decades computer scientists have studied preference data, which became available in unprecedented quantities: Each time we click or tap on a search result, a sponsored ad or a product recommendation, we express preference of one alternative from a small set of alternatives. Additionally, many crowsdsourcing systems explicitly ask paid? experts to solicit preferences or even full rankings of alternative sets. What are the advantages of preferences compared to other forms of information, and what combinatorial and learning theoretical challenges do they give rise to? I will present important problems and survey results.

[1]  David Maxwell Chickering,et al.  Here or there: preference judgments for relevance , 2008 .

[2]  Fabrizio Grandoni,et al.  Resilient dictionaries , 2009, TALG.

[3]  Pasquale Lops,et al.  Learning Preference Models in Recommender Systems , 2010, Preference Learning.

[4]  Ronald Fagin,et al.  Comparing and aggregating rankings with ties , 2004, PODS '04.

[5]  Steve Hanneke,et al.  A bound on the label complexity of agnostic active learning , 2007, ICML '07.

[6]  P.-C.-F. Daunou,et al.  Mémoire sur les élections au scrutin , 1803 .

[7]  Nir Ailon,et al.  Aggregating inconsistent information: Ranking and clustering , 2008 .

[8]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[9]  Yoram Singer,et al.  Learning to Order Things , 1997, NIPS.

[10]  Cynthia Rudin,et al.  The P-Norm Push: A Simple Convex Ranking Algorithm that Concentrates at the Top of the List , 2009, J. Mach. Learn. Res..

[11]  Eyke Hüllermeier,et al.  Label ranking by learning pairwise preferences , 2008, Artif. Intell..

[12]  Nir Ailon,et al.  Active Learning Using Smooth Relative Regret Approximations with Applications , 2011, COLT.

[13]  Kenneth E. Train,et al.  Discrete Choice Methods with Simulation , 2016 .

[14]  David Maxwell Chickering,et al.  Here or There , 2008, ECIR.

[15]  Atri Rudra,et al.  Ordering by weighted number of wins gives a good ranking for weighted tournaments , 2006, SODA '06.

[16]  Eli Upfal,et al.  Computing with unreliable information , 1990, STOC '90.

[17]  Maria-Florina Balcan,et al.  Agnostic active learning , 2006, J. Comput. Syst. Sci..

[18]  Martín Abadi,et al.  Security analysis of cryptographically controlled access to XML documents , 2005, PODS '05.

[19]  Luo Si,et al.  A latent pairwise preference learning approach for recommendation from implicit feedback , 2012, CIKM '12.

[20]  Moni Naor,et al.  Rank aggregation methods for the Web , 2001, WWW '01.

[21]  Mark Braverman,et al.  Noisy sorting without resampling , 2007, SODA '08.

[22]  Yuan Yao,et al.  Statistical ranking and combinatorial Hodge theory , 2008, Math. Program..

[23]  Nicolas de Condorcet.  Essai Sur L'Application de L'Analyse a la Probabilite Des Decisions Rendues a la Pluralite Des Voix , 2009 .

[24]  J. Marden.  Analyzing and Modeling Rank Data , 1996 .

[25]  Devavrat Shah,et al.  Sparse choice models , 2010, 2012 46th Annual Conference on Information Sciences and Systems (CISS).

[26]  S. Shapiro,et al.  Mathematics without Numbers , 1993 .

[27]  R. Graham,et al.  Spearman's Footrule as a Measure of Disarray , 1977 .

[28]  Nir Ailon,et al.  Ranking from pairs and triplets: information quality, evaluation methods and query complexity , 2011, WSDM '11.

[29]  L. A. Goodman,et al.  Social Choice and Individual Values , 1951 .

[30]  M. Trick,et al.  Voting schemes for which it can be difficult to tell who won the election , 1989 .

[31]  Peter Ingwersen,et al.  Developing a Test Collection for the Evaluation of Integrated Search , 2010, ECIR.

[32]  Claire Mathieu,et al.  Electronic Colloquium on Computational Complexity, Report No. 144 (2006) How to rank with few errors A PTAS for Weighted Feedback Arc Set on Tournaments , 2006 .

[33]  Filip Radlinski,et al.  Learning diverse rankings with multi-armed bandits , 2008, ICML '08.