On the Complexity of Query Result Diversification

Query result diversification is a bi-criteria optimization problem for ranking query results. Given a database D, a query Q, and a positive integer k, it is to find a set of k tuples from Q(D) such that the tuples are as relevant as possible to the query, and at the same time, as diverse as possible to each other. Subsets of Q(D) are ranked by an objective function defined in terms of relevance and diversity. Query result diversification has found a variety of applications in databases, information retrieval, and operations research. This article investigates the complexity of result diversification for relational queries. (1) We identify three problems in connection with query result diversification, to determine whether there exists a set of k tuples that is ranked above a bound with respect to relevance and diversity, to assess the rank of a given k-element set, and to count how many k-element sets are ranked above a given bound based on an objective function. (2) We study these problems for a variety of query languages and for the three objective functions proposed in Gollapudi and Sharma [2009]. We establish the upper and lower bounds of these problems, all matching, for both combined complexity and data complexity. (3) We also investigate several special settings of these problems, identifying tractable cases. Moreover, (4) we reinvestigate these problems in the presence of compatibility constraints commonly found in practice, and provide their complexity in all these settings.

[1]  Wenfei Fan,et al.  On the Complexity of Query Result Diversification , 2013, Proc. VLDB Endow..

[2]  Gena Hahn,et al.  Counting feasible solutions of the traveling salesman problem with pickups and deliveries is #P-complete , 2009, Discret. Appl. Math..

[3]  Aditya G. Parameswaran,et al.  Recommendation systems with complex constraints: A course recommendation perspective , 2011, TOIS.

[4]  Evaggelia Pitoura,et al.  Search result diversification , 2010, SGMD.

[5]  Yehoshua Sagiv,et al.  An incremental algorithm for computing ranked full disjunctions , 2007, J. Comput. Syst. Sci..

[6]  Heribert Vollmer,et al.  The satanic notations , 1995, SIGACT News.

[7]  Jorge Lobo,et al.  Qualifying Answers According to User Needs and Preferences , 1997, Fundam. Informaticae.

[8]  Evaggelia Pitoura,et al.  Diversity over Continuous Data , 2009, IEEE Data Eng. Bull..

[9]  Davide Martinenghi,et al.  Top-k bounded diversification , 2012, SIGMOD Conference.

[10]  Leslie G. Valiant,et al.  The Complexity of Computing the Permanent , 1979, Theor. Comput. Sci..

[11]  Cong Yu,et al.  It takes variety to make a world: diversification in recommender systems , 2009, EDBT '09.

[12]  Tao Li,et al.  Addressing diverse user preferences in SQL-query-result navigation , 2007, SIGMOD '07.

[13]  Oleg A. Prokopyev,et al.  The equitable dispersion problem , 2009, Eur. J. Oper. Res..

[14]  Surajit Chaudhuri.  Generalization and a framework for query modification , 1990, [1990] Proceedings. Sixth International Conference on Data Engineering.

[15]  Yi Chen,et al.  Structured Search Result Differentiation , 2009, Proc. VLDB Endow..

[16]  Aditya G. Parameswaran,et al.  Evaluating, combining and generalizing recommendations with prerequisites , 2010, CIKM.

[17]  Moshe Y. Vardi.  The complexity of relational query languages (Extended Abstract) , 1982, STOC '82.

[18]  Richard E. Ladner.  Polynomial Space Counting Problems , 1989, SIAM J. Comput..

[19]  Wenfei Fan,et al.  On the Complexity of Package Recommendation Problems , 2013 .

[20]  Sihem Amer-Yahia,et al.  Efficient Computation of Diverse Query Results , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[21]  Larry J. Stockmeyer,et al.  The Polynomial-Time Hierarchy , 1976, Theor. Comput. Sci..

[22]  Recommendation Diversification Using Explanations , 2009, 2009 IEEE 25th International Conference on Data Engineering.

[23]  Divesh Srivastava,et al.  On query result diversification , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[24]  Laks V. S. Lakshmanan,et al.  Composite recommendations: from items to packages , 2012, Frontiers of Computer Science.

[25]  Laks V. S. Lakshmanan,et al.  Breaking out of the box of recommendations: from items to packages , 2010, RecSys '10.

[26]  Georgia Koutrika,et al.  A survey on representation, composition and application of preferences in database systems , 2011, TODS.

[27]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[28]  Sihem Amer-Yahia,et al.  Complexity and algorithms for composite retrieval , 2013, WWW '13 Companion.

[29]  Mi Zhang,et al.  Avoiding monotony: improving the diversity of recommendation lists , 2008, RecSys '08.

[30]  Yuli Ye,et al.  Max-Sum diversification, monotone submodular functions and dynamic updates , 2012, PODS '12.

[31]  Phokion G. Kolaitis,et al.  Subtractive reductions and complete problems for counting complexity classes , 2000 .

[32]  Wolfgang Nejdl,et al.  Current Approaches to Search Result Diversification , 2009, LivingWeb@ISWC.

[33]  Ihab F. Ilyas,et al.  A survey of top-k query processing techniques in relational database systems , 2008, CSUR.

[34]  Sihem Amer-Yahia,et al.  Recommendation Projects at Yahoo! , 2011, IEEE Data Eng. Bull..

[35]  Sreenivas Gollapudi,et al.  Diversifying search results , 2009, WSDM '09.

[36]  Anthony K. H. Tung,et al.  Relaxing join and selection queries , 2006, VLDB.

[37]  Peter Fankhauser,et al.  DivQ: diversification for keyword search over structured databases , 2010, SIGIR.

[38]  Surajit Chaudhuri,et al.  Ranking objects based on relationships and fixed associations , 2009, EDBT '09.

[39]  Kevin Chen-Chuan Chang,et al.  RankSQL: Supporting Ranking Queries in Relational Database Management Systems , 2005, VLDB.

[40]  Neoklis Polyzotis,et al.  Evaluating rank joins with optimal cost , 2008, PODS.

[41]  Andrei Voronkov,et al.  Complexity of nonrecursive logic programs with complex values , 1998, PODS.

[42]  Yehoshua Sagiv,et al.  Optimizing and parallelizing ranked enumeration , 2011, Proc. VLDB Endow..

[43]  Surajit Chaudhuri,et al.  On the Equivalence of Recursive and Nonrecursive Datalog Programs , 1997, J. Comput. Syst. Sci..

[44]  John R. Smith,et al.  Supporting Incremental Join Queries on Ranked Inputs , 2001, VLDB.

[45]  Mark W. Krentel.  Generalizations of Opt P to the Polynomial Hierarchy , 1992, Theor. Comput. Sci..

[46]  Fabrizio Silvestri,et al.  Efficient Diversification of Web Search Results , 2011, Proc. VLDB Endow..

[47]  Jignesh M. Patel,et al.  Efficient and generic evaluation of ranked queries , 2011, SIGMOD '11.

[48]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[49]  Juliana Freire,et al.  Supporting Exploratory Queries in Databases , 2004, DASFAA.

[50]  Evaggelia Pitoura,et al.  PerK: personalized keyword search in relational databases through preferences , 2010, EDBT '10.

[51]  Theodoros Lappas,et al.  Finding a team of experts in social networks , 2009, KDD.

[52]  Georgia Koutrika,et al.  FlexRecs: expressing and combining flexible recommendations , 2009, SIGMOD Conference.

[53]  Sreenivas Gollapudi,et al.  An axiomatic approach for result diversification , 2009, WWW '09.

[54]  Ricardo A. Baeza-Yates,et al.  New Stochastic Algorithms for Scheduling Ads in Sponsored Search , 2007, 2007 Latin American Web Conference (LA-WEB 2007).

[55]  Moni Naor,et al.  Optimal aggregation algorithms for middleware , 2001, PODS.

[56]  Jon Whittle,et al.  CARD: a decision-guidance framework and application for recommending composite alternatives , 2008, RecSys '08.

[57]  Serge Abiteboul,et al.  Foundations of Databases , 1994 .