A Cooperative Coevolution Framework for Parallel Learning to Rank

We propose CCRank, the first parallel framework for learning to rank based on evolutionary algorithms (EA), aiming to significantly improve learning efficiency while maintaining accuracy. CCRank is based on cooperative coevolution (CC), a divide-and-conquer framework that has demonstrated high promise in function optimization for problems with large search space and complex structures. Moreover, CC naturally allows parallelization of sub-solutions to the decomposed sub-problems, which can substantially boost learning efficiency. With CCRank, we investigate parallel CC in the context of learning to rank. We implement CCRank with three EA-based learning to rank algorithms for demonstration. Extensive experiments on benchmark datasets in comparison with the state-of-the-art algorithms show the performance gains of CCRank in efficiency and accuracy.

[1]  John D. Lafferty,et al.  A study of smoothing methods for language models applied to Ad Hoc information retrieval , 2001, SIGIR '01.

[2]  Stephen E. Robertson,et al.  Overview of the Okapi projects , 1997, J. Documentation.

[3]  Samy Bengio,et al.  A Parallel Mixture of SVMs for Very Large Scale Problems , 2001, Neural Computation.

[4]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[5]  Alberto Moraglio,et al.  Geometric Differential Evolution on the Space of Genetic Programs , 2010, EuroGP.

[6]  Qiang Wu,et al.  Adapting boosting for information retrieval measures , 2010, Information Retrieval.

[7]  Weiguo Fan,et al.  Discovery of context-specific ranking functions for effective information retrieval using genetic programming , 2004, IEEE Transactions on Knowledge and Data Engineering.

[8]  Chiranjib Bhattacharyya,et al.  Structured learning for non-smooth ranking losses , 2008, KDD.

[9]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[10]  Fernando José Von Zuben,et al.  Learning and optimization using the clonal selection principle , 2002, IEEE Trans. Evol. Comput..

[11]  Julian Togelius,et al.  Geometric differential evolution , 2009, GECCO '09.

[12]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[13]  M. Cooper,et al.  The Evolution of Adaptive Immune Systems , 2006, Cell.

[14]  A Basten,et al.  An overview of the immune system. , 1991 .

[15]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[16]  Petr Musílek,et al.  Immune programming , 2006, Inf. Sci..

[17]  Kunle Olukotun,et al.  Map-Reduce for Machine Learning on Multicore , 2006, NIPS.

[18]  Rudolf Paul Wiegand,et al.  An analysis of cooperative coevolutionary algorithms , 2004 .

[19]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[20]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[21]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[22]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[23]  Jiming Liu,et al.  Learning to rank using evolutionary computation: immune programming or genetic programming? , 2009, CIKM.

[24]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[25]  Igor Durdanovic,et al.  Parallel Support Vector Machines: The Cascade SVM , 2004, NIPS.

[26]  Sanjay Ghemawat,et al.  MapReduce: simplified data processing on large clusters , 2008, CACM.

[27]  Jordan B. Pollack,et al.  Symbiotic Combination as an Alternative to Sexual Recombination in Genetic Algorithms , 2000, PPSN.

[28]  R. Paul Wiegand,et al.  An empirical analysis of collaboration methods in cooperative coevolutionary algorithms , 2001 .

[29]  Thierson Couto,et al.  Improving On-Demand Learning to Rank through Parallelism , 2012, WISE.

[30]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[31]  Mitchell A. Potter,et al.  EVOLVING NEURAL NETWORKS WITH COLLABORATIVE SPECIES , 2006 .

[32]  Hongfei Teng,et al.  Cooperative Co-evolutionary Differential Evolution for Function Optimization , 2005, ICNC.

[33]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[34]  Phil Husbands,et al.  Simulated Co-Evolution as the Mechanism for Emergent Planning and Scheduling , 1991, ICGA.

[35]  J. Parkin,et al.  An overview of the immune system , 2001, The Lancet.

[36]  Ian H. Witten,et al.  Managing Gigabytes: Compressing and Indexing Documents and Images , 1999 .

[37]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[38]  Anikó Ekárt,et al.  A Metric for Genetic Programs and Fitness Sharing , 2000, EuroGP.

[39]  Pável Calado,et al.  A combined component approach for finding collection-adapted ranking functions based on genetic programming , 2007, SIGIR.

[40]  Hang Li,et al.  AdaRank: a boosting algorithm for information retrieval , 2007, SIGIR.

[41]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[42]  Matthew Lease,et al.  Parallelizing ListNet training using spark , 2012, SIGIR '12.

[43]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[44]  Kilian Q. Weinberger,et al.  Web-Search Ranking with Initialized Gradient Boosted Regression Trees , 2010, Yahoo! Learning to Rank Challenge.

[45]  Seth Bullock,et al.  Combating Coevolutionary Disengagement by Reducing Parasite Virulence , 2004, Evolutionary Computation.

[46]  Xiaodong Li,et al.  Cooperatively Coevolving Particle Swarms for Large Scale Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[47]  Stephen E. Robertson,et al.  GatfordCentre for Interactive Systems ResearchDepartment of Information , 1996 .

[48]  Harris Wu,et al.  The effects of fitness functions on genetic programming-based ranking discovery forWeb search , 2004, J. Assoc. Inf. Sci. Technol..

[49]  Quoc V. Le,et al.  Learning to Rank with Nonsmooth Cost Functions , 2006, NIPS.

[50]  Thomas Hofmann,et al.  Large Margin Methods for Structured and Interdependent Output Variables , 2005, J. Mach. Learn. Res..

[51]  Xiaodong Li,et al.  Cooperative Co-evolution with delta grouping for large scale non-separable function optimization , 2010, IEEE Congress on Evolutionary Computation.

[52]  Xueqi Cheng,et al.  Top-k learning to rank: labeling, ranking and evaluation , 2012, SIGIR '12.

[53]  Xin Yao,et al.  An adaptive coevolutionary Differential Evolution algorithm for large-scale optimization , 2009, 2009 IEEE Congress on Evolutionary Computation.

[54]  Hang Li,et al.  Learning to Rank , 2009, ACL.

[55]  Tie-Yan Liu,et al.  Future directions in learning to rank , 2010, Yahoo! Learning to Rank Challenge.

[56]  Stephen E. Robertson,et al.  Okapi at TREC-3 , 1994, TREC.