Score Normalization Using Logistic Regression with Expected Parameters

State-of-the-art score normalization methods use generative models that rely on sometimes unrealistic assumptions. We propose a novel parameter estimation method for score normalization based on logistic regression, using the expected parameters from past queries. Experiments on the Gov2 and CluewebA collection indicate that our method is consistently more precise in predicting the number of relevant documents in the top-n ranks compared to a state-of-the-art generative approach and another parameter estimate for logistic regression.

[1]  Evangelos Kanoulas,et al.  Score distribution models: assumptions, intuition, and robustness to score manipulation , 2010, SIGIR.

[2]  Fredric C. Gey,et al.  Probabilistic Retrieval in the TIPSTER Collections: An Application of Staged Logistic Regression , 1992, TREC.

[3]  Stephen E. Robertson,et al.  Where to stop reading a ranked list?: threshold optimization using truncated score distributions , 2009, SIGIR.

[4]  Jamie Callan,et al.  DISTRIBUTED INFORMATION RETRIEVAL , 2002 .

[5]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[6]  Stephen E. Robertson,et al.  Probabilistic models in IR and their relationships , 2014, Information Retrieval.

[7]  Fredric C. Gey,et al.  Probabilistic retrieval based on staged logistic regression , 1992, SIGIR '92.

[8]  Avi Arampatzis,et al.  A signal-to-noise approach to score normalization , 2009, CIKM.

[9]  R. Manmatha,et al.  Modeling score distributions for combining the outputs of search engines , 2001, SIGIR '01.

[10]  Stephen E. Robertson,et al.  Modeling score distributions in information retrieval , 2011, Information Retrieval.

[11]  Milad Shokouhi,et al.  Introduction to special issue on the second international conference on the theory of information retrieval , 2010, Information Retrieval.

[12]  Norbert Fuhr,et al.  From Uncertain Inference to Probability of Relevance for Advanced IR Applications , 2003, ECIR.

[13]  Douglas W. Oard,et al.  Overview of the TREC 2011 Legal Track , 2011, TREC.

[14]  Peter Ingwersen,et al.  Developing a Test Collection for the Evaluation of Integrated Search , 2010, ECIR.

[15]  Charles L. A. Clarke,et al.  Efficient and effective spam filtering and re-ranking for large web datasets , 2010, Information Retrieval.

[16]  Pablo Castells,et al.  Using historical data to enhance rank aggregation , 2006, SIGIR '06.