Assigned tasks are not the same as self-chosen Web search tasks

Short assigned question-answering style tasks are often used as a probe to understand how users do search. While such assigned tasks are simple to test and are effective at eliciting the particulars of a given search capability, they are not the same as naturalistic searches. We studied the quantitative differences between assigned tasks and self-chosen "own" tasks finding that users behave differently when doing their own tasks, staying longer on the task, but making fewer queries and different kinds of queries overall. This finding implies that user's own tasks should be used when testing user behavior in addition to assigned tasks, which remain useful for feature testing in lab settings