Assessing Level of Difficulty in Web Search Questions

This article focuses on assessing the level of difficulty of search questions for Web searching, looking at variations in judgments by type of questions and the reasons underlying these judgments. It analyzes the decisions and reasoning of fifty-four experienced Web searchers on sixteen search questions of four types: closed/predictable source, closed/unpredictable source, open/predictable source, and open/unpredictable source. The research uses a mixed mode of analysis--quantitative on the first research question and qualitative on the second. Participants consider closed/predictable source questions easy, open/unpredictable source questions difficult. Raters of individual questions agree strongly on ratings of most questions. Reasoning indicates that participants consider, besides the question variables addressed specifically, six (searchability, clarity, familiarity/currency, public knowledge, simplicity, and specificity) and eleven (searchability problems, diffusion of information, noise, breadth, lack of clarity, postsearch processing, overload, unspecified "feeling," likelihood of being on the Web, unfamiliarity, and complexity) additional reasons for ease and difficulty, respectively.

[1]  Pamela Jordan.  Basics of qualitative research: Grounded theory procedures and techniques , 1994 .

[2]  Ingrid Hsieh-Yee,et al.  Search Tactics of Web Users in Searching for Texts, Graphics, Known Items and Subjects , 1998 .

[3]  Helen Howard,et al.  Measures that discriminate among online searchers with different training and experience , 1982 .

[4]  Carol H. Fenichel,et al.  Online searching: Measures that discriminate among users with different types of experiences , 1981, J. Am. Soc. Inf. Sci..

[5]  Paul B. Kantor,et al.  A study of information seeking and retrieving. II. Users, questions, and effectiveness , 1988, J. Am. Soc. Inf. Sci..

[6]  Franklin J. Meine.  Webster's Columbia concise dictionary of the English language , 1940 .

[7]  Kyung-Sun Kim,et al.  Cognitive style and on-line database search experience as predictors of Web search performance , 2000, J. Am. Soc. Inf. Sci..

[8]  Thomas Childers.  Measuring the Difficulty of Reference Questions. , 1991 .

[9]  Mirja Iivonen,et al.  The choice of initial web search strategies: a comparison between Finnish and American searchers , 2001, J. Documentation.

[10]  Paul B. Kantor,et al.  A study of information seeking and retrieving. III. Searchers, searches, and overlap , 1988, J. Am. Soc. Inf. Sci..

[11]  Paul B. Kantor,et al.  A study of information seeking and retrieving. I. Background and methodology , 1997, J. Am. Soc. Inf. Sci..

[12]  Carol Tenopir,et al.  Users' interaction with World Wide Web resources: an exploratory study using a holistic approach , 2000, Inf. Process. Manag..

[13]  Iwan G. J. H. Wopereis,et al.  Differences between novice and experienced users in searching information on the World Wide Web , 2000, J. Am. Soc. Inf. Sci..