Overview of the NTCIR-4 WEB Navigational Retrieval Task 1

This paper describes an overview of the Navigational Retrieval Task 1 that was conducted from 2002 to 2004 as a subtask of the WEB Task at the Fourth NTCIR Workshop. In the Task, we attempted to assess the retrieval effectiveness of Web search systems from a viewpoint of “Known Item Search” using a common data set, and built a re-usable test collection. 100-gigabyte Web document data constructed at NTCIR-3 and 300 topics prepared in this task were distributed to the participants and, in turn, 16 run results were submitted by 5 groups and some by organizers. Relevance judgments were performed on the documents pooled from the results, mainly from the viewpoint of representativeness of searched items given by the topics. Several kinds of evaluation measures were applied to the run results submitted by each participant.