Exploring search task difficulty reasons in different task types and user knowledge groups

Development of a search task difficulty reason scheme.Relationship between task difficulty and task type: common and different reasons across task types.Relationship between task difficulty and user knowledge: common and different reasons among different knowledge groups.Implications for general and personalized information retrieval system design. Search task difficulty has been attracting much research attention in recent years, mostly regarding its relationship with searchers' behaviors and the prediction of task difficulty from search behaviors. However, it remains unknown what makes searchers feel the difficulty. A study consisting of 48 undergraduate students was conducted to explore this question. Each participant was given 4 search tasks that were carefully designed following a task classification scheme. Questionnaires were used to elicit participants' ratings on task difficulty and why they gave those ratings. Based on the collected difficulty reasons, a coding scheme was developed, which covered various aspects of task, user, and user-task interaction. Difficulty reasons were then categorized following this scheme. Results showed that searchers reported some common reasons leading to task difficulty in different tasks, but most of the difficulty reasons varied across tasks. In addition, task difficulty had some common reasons between searchers with low and high levels of topic knowledge, although there were also differences in top task difficulty reasons between high and low knowledge users. These findings further our understanding of search task difficulty, the relationship between task difficulty and task type, and that between task difficulty and knowledge level. The findings can also be helpful with designing tasks for information search experiments, and have implications on search system design both in general and for personalization based on task type and searchers' knowledge.