An applied ethnographic method for evaluating retrieval features

This paper reports on a study conducted to develop a framework for evaluating the retrieval features using an applied ethnographic method. Direct observation, interviews, analysis of notes, and informal social interaction were done with ten users on their application of the retrieval features and their difficulties in searching. Retrieval features evaluated were those offered by 12 database providers. Findings revealed that the proposed framework successfully gathered the data needed. Application of the features was related to users' retrieval tasks, preference and style of searching, and understanding of the features. Difficulties were related to identification of the appropriate search terms. Expected retrieval features were related to search terms, i.e. relevance feedback, list of similar terms, and assigning values to search terms. Applied ethnographic method used in this study revealed that users have a substantial amount of knowledge about the retrieval features, and that their comments were related to their subject background.

[1]  Martyn Hammersley,et al.  Ethnography : Principles in Practice , 1983 .

[2]  Paul Beynon-Davies,et al.  Ethnography and information systems development: Ethnography of, for and within is development , 1997, Inf. Softw. Technol..

[3]  Jenna Burrell,et al.  Extending ubiquitous computing to vineyards , 2003, CHI Extended Abstracts.

[4]  Sally Jo Cunningham,et al.  An ethnographic study of technical support workers: why we didn't build a tech support digital library , 2001, JCDL '01.

[5]  Tom Rodden,et al.  Designing with ethnography: a presentation framework for design , 1997, DIS '97.

[6]  David Squires.  Exploring the use of interactive information systems in academic research: borrowing from the ethnographic tradition , 1997 .

[7]  Clay Spinuzzi.  Investigating the technology-work relationship: a critical comparison of three qualitative field methods , 2000, 18th Annual Conference on Computer Documentation. ipcc sigdoc 2000. Technology and Teamwork. Proceedings. IEEE Professional Communication Society International Professional Communication Conference an.

[8]  Jesper Simonsen,et al.  Using ethnography in contextural design , 1997, CACM.

[9]  Richard Bentley,et al.  Situated evaluation for cooperative systems , 1994, CSCW '94.

[10]  Diana E. Forsythe,et al.  Using ethnography in the design of an explanation system , 1995 .

[11]  Patricia Somers,et al.  Unleashed: Web tablet integration into the home , 2000, CHI.

[12]  Michael D. Myers,et al.  Investigating Information Systems with Ethnographic Research , 1999, Commun. Assoc. Inf. Syst..

[13]  Scott Lewis,et al.  Ethnographic data for product development: a collaborative process , 1996, INTR.

[14]  Tom Rodden,et al.  Moving out from the control room: ethnography in system design , 1994, CSCW '94.

[15]  Vidar Hepsø,et al.  The development and coaching of VISOK: a web-based project in Statoil , 1999, SIGG.

[16]  Ben Shneiderman,et al.  An applied ethnographic method for redesigning user interfaces , 1995, Symposium on Designing Interactive Systems.

[17]  David M. Nichols,et al.  The contribution of ethnomethodologically-informed ethnography to the process of designing digital libraries , 1998 .