What makes a query temporally sensitive?

This work examines factors that affect manual classifications of “temporally sensitive” information needs. We introduce the concepts of temporal relevance and temporal topicality to differentiate between different aspects of temporal retrieval research. We use qualitative and quantitative techniques to analyze 660 topics from the Text Retrieval Conference (TREC) previously used in the experimental evaluation of temporal retrieval models. We use regression analysis to model previous manual classifications. We identify factors and potential problems with previous classifications, proposing principles and guidelines for future work on the evaluation of temporal retrieval models.

[1]  Stefano Mizzaro,et al.  Relevance: The Whole History , 1997, J. Am. Soc. Inf. Sci..

[2]  Elad Yom-Tov,et al.  Updating Users about Time Critical Events , 2013, ECIR.

[3]  Feng Niu,et al.  Building an Entity-Centric Stream Filtering Test Collection for TREC 2012 , 2012, TREC.

[4]  José Luis Vicedo González,et al.  TREC: Experiment and evaluation in information retrieval , 2007, J. Assoc. Inf. Sci. Technol..

[5]  Susan T. Dumais,et al.  Understanding temporal query dynamics , 2011, WSDM '11.

[6]  Fernando Diaz,et al.  Temporal profiles of queries , 2007, TOIS.

[7]  James Parker,et al.  on Knowledge and Data Engineering, , 1990 .

[8]  Ellen M. Voorhees,et al.  Evaluating Stream Filtering for Entity Profile Updates in TREC 2012, 2013, and 2014 , 2014, TREC.

[9]  Roi Blanco,et al.  NTCIR temporalia: a test collection for temporal information access research , 2014, WWW.

[10]  Gerhard Weikum,et al.  A Language Modeling Approach for Temporal Information Needs , 2010, ECIR.

[11]  Kjetil Nørvåg,et al.  A comparison of time-aware ranking methods , 2011, SIGIR '11.

[12]  Miles Efron,et al.  Estimation methods for ranking recent information , 2011, SIGIR.

[13]  Milad Shokouhi,et al.  Detecting seasonal queries by time-series analysis , 2011, SIGIR.

[14]  Ee-Peng Lim,et al.  Analyzing feature trajectories for event detection , 2007, SIGIR.

[15]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[16]  Brian D. Davison,et al.  Freshness matters: in flowers, food, and web authority , 2010, SIGIR.

[17]  Ellen M. Voorhees,et al.  TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing) , 2005 .

[18]  Carol L. Barry,et al.  Users' Criteria for Relevance Evaluation: A Cross-situational Comparison , 1998, Inf. Process. Manag..

[19]  Luis Gravano,et al.  Answering General Time-Sensitive Queries , 2008, IEEE Transactions on Knowledge and Data Engineering.

[20]  M. de Rijke,et al.  Using temporal bursts for query modeling , 2014, Information Retrieval.

[21]  Gilad Mishne,et al.  Towards recency ranking in web search , 2010, WSDM '10.

[22]  Fuchun Peng,et al.  Improving search relevance for implicitly temporal queries , 2009, SIGIR.

[23]  Craig Willis,et al.  What Makes a Query Temporally Sensitive? , 2016, SIGIR.

[24]  Klaus Berberich,et al.  Learning to select a time-aware retrieval model , 2012, SIGIR '12.

[25]  W. Bruce Croft,et al.  Time-based language models , 2003, CIKM '03.