A Two-Stage Ensemble of Diverse Models for Advertisement Ranking in KDD Cup 2012

This paper describes the solution of National Taiwan University for track 2 of KDD Cup 2012. Track 2 of KDD Cup 2012 aims to predict the click-through rate of ads on Tencent proprietary search engine. We exploit classication, regression, ranking, and factorization models to utilize a variety of dierent signatures captured from the dataset. We then blend our individual models to boost the performance through two stages, one on an internal validation set and one on the external test set. Our solution achieves 0.8069 AUC on the public test set and 0.8089 AUC on the private test set.

[1]  Chih-Jen Lin,et al.  Training and Testing Low-degree Polynomial Data Mappings via Linear SVM , 2010, J. Mach. Learn. Res..

[2]  Mehryar Mohri,et al.  An Efficient Reduction of Ranking to Classification , 2007, COLT.

[3]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[4]  Peter Willett,et al.  Readings in information retrieval , 1997 .

[5]  Christopher J. C. Burges,et al.  From RankNet to LambdaRank to LambdaMART: An Overview , 2010 .

[6]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[7]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[8]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[9]  Kilian Q. Weinberger,et al.  Web-Search Ranking with Initialized Gradient Boosted Regression Trees , 2010, Yahoo! Learning to Rank Challenge.

[10]  Hongyuan Zha,et al.  A regression framework for learning ranking functions using relative relevance judgments , 2007, SIGIR.

[11]  Michael Jahrer,et al.  Collaborative Filtering Ensemble for Ranking , 2012, KDD Cup.

[12]  John D. Lafferty,et al.  A study of smoothing methods for language models applied to Ad Hoc information retrieval , 2001, SIGIR '01.

[13]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[14]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[15]  D. Sculley,et al.  Combined regression and ranking , 2010, KDD.

[16]  J Allan,et al.  Readings in information retrieval. , 1998 .

[17]  Shou-De Lin,et al.  An Ensemble of Three Classifiers for KDD Cup 2009: Expanded Linear Model, Heterogeneous Boosting, and Selective Naive Bayes , 2009, KDD Cup.

[18]  Yoram Singer,et al.  Learning to Order Things , 1997, NIPS.

[19]  Yong Yu,et al.  Feature-Based Matrix Factorization , 2011, ArXiv.

[20]  Shou-De Lin,et al.  A Linear Ensemble of Individual and Blended Models for Music Rating Prediction , 2012, KDD Cup.

[21]  D. Sculley,et al.  Large Scale Learning to Rank , 2009 .

[22]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[23]  Cristina V. Lopes,et al.  Bagging gradient-boosted trees for high precision, low variance ranking models , 2011, SIGIR.

[24]  Shou-De Lin,et al.  Novel Models and Ensemble Techniques to Discriminate Favorite Items from Unrated Ones for Personalized Music Recommendation , 2012, KDD Cup.