User assessment of search task difficulty: Relationships between reasons and ratings

Abstract There is an increasing research interest in search task difficulty. A recent study developed a user-perceived task difficulty reason scheme including 21 reasons (two of which arise prior to searching). Interesting questions arose as to whether the many difficulty reasons in the scheme were correlated or whether the difficulty reasons were related to task difficulty levels. Using a dataset collected from 48 participants searching on four tasks, this study examined these questions while also validating the task difficulty reason scheme. Results suggest that the 19 difficulty reasons elicited after a search task are rarely correlated, and are, instead, independent. Several difficulty reasons, such as “task requirements too specific” and “low topic knowledge,” contribute significantly to difficulty levels. The large discrepancy in the frequency of some reasons' appearance in high, mid, and low difficulty rating groups may be explained by task type features. These findings help develop an in-depth explanation of search task difficulty and contribute to task-related information retrieval research and system design.

[1]  Kalervo Järvelin,et al.  Task complexity affects information seeking and use , 1995 .

[2]  Nicholas J. Belkin,et al.  Understanding searchers' perception of task difficulty: Relationships with task type , 2011, ASIST.

[3]  Jacek Gwizdka,et al.  Search behaviors in different task types , 2010, JCDL '10.

[4]  Jingjing Liu,et al.  Why Do Users Perceive Search Tasks As Difficult? Exploring Difficulty in Different Task Types , 2013, HCIR '13.

[5]  Nicholas J. Belkin,et al.  A faceted approach to conceptualizing tasks in information seeking , 2008, Inf. Process. Manag..

[6]  Anne Aula,et al.  How does search behavior change as search becomes more difficult? , 2010, CHI.

[7]  Luanne Freund,et al.  Assigning search tasks designed to elicit exploratory search behaviors , 2012, HCIR '12.

[8]  Jacek Gwizdka,et al.  Predicting task difficulty for different task types , 2010, ASIST.

[9]  Nicholas J. Belkin,et al.  Exploring and predicting search task difficulty , 2012, CIKM '12.

[10]  Pertti Vakkari,et al.  Task complexity, problem structure and information actions - Integrating studies on information seeking and retrieval , 1999, Inf. Process. Manag..

[11]  Jeonghyun Kim,et al.  Task difficulty as a predictor and indicator of web searching interaction , 2006, CHI Extended Abstracts.

[12]  Jacek Gwizdka,et al.  What Can Searching Behavior Tell Us About the Difficulty of Information Tasks? A Study of Web Navigation , 2007, ASIST.

[13]  Peter Ingwersen,et al.  The Turn - Integration of Information Seeking and Retrieval in Context , 2005, The Kluwer International Series on Information Retrieval.

[14]  Jaime Arguello.  Predicting Search Task Difficulty , 2014, ECIR.

[15]  Jacek Gwizdka,et al.  Revisiting search task difficulty: Behavioral and individual difference measures , 2008, ASIST.

[16]  Elizabeth D. Liddy,et al.  The effects of expertise and feedback on search term selection and subsequent learning , 2005, J. Assoc. Inf. Sci. Technol..