Training Students to Evaluate Search Engines

In this chapter, two exercises that were part of an information retrieval course are described. Both exercises were set up to assess the qualities of a search engine and to consider how the engine could be improved. Students were engaged in problem-based learning. The exercises were run for several years and found to be successful both in engaging students in the course, but also in establishing links with organisations interested in evaluation of search.

[1]  Daniel L. Schwartz,et al.  Doing with Understanding: Lessons from Research on Problem- and Project-Based Learning , 1998 .

[2]  Martin White,et al.  Making Search Work , 2007 .

[3]  William Heard Kilpatrick,et al.  The Project Method , 1918, Teachers College Record: The Voice of Scholarship in Education.

[4]  Ellen M. Voorhees.  Variations in relevance judgments and the measurement of retrieval effectiveness , 2000, Inf. Process. Manag..

[5]  Dan Morris,et al.  Investigating the querying and browsing behavior of advanced search engine users , 2007, SIGIR.

[6]  Bernard J. Jansen,et al.  Brand and its effect on user perception of search engine performance , 2009, J. Assoc. Inf. Sci. Technol..

[7]  Robin Yeates.  Making Search Work: Implementing Web, Intranet and Enterprise Search , 2007, Program.

[8]  Andrei Broder,et al.  A taxonomy of web search , 2002, SIGF.

[9]  Olivier Chapelle,et al.  Expected reciprocal rank for graded relevance , 2009, CIKM.

[10]  C. D. Gull.  Seven years of work on the organization of materials in the special library , 1956 .

[11]  Jaana Kekäläinen,et al.  IR evaluation methods for retrieving highly relevant documents , 2000, SIGIR '00.

[12]  Peter Bailey,et al.  Relevance assessment: are judges exchangeable and does it matter , 2008, SIGIR '08.

[13]  R. G. Thorne.  THE EFFICIENCY OF SUBJECT CATALOGUES AND THE COST OF INFORMATION SEARCHES , 1955 .

[14]  R. Hogarth,et al.  Order effects in belief updating: The belief-adjustment model , 1992, Cognitive Psychology.

[15]  Martin Braschler,et al.  Evaluation der Suchfunktion Deutscher Unternehmenswebsites , 2009 .