Expert-Novice Differences in Searching the Web

Three web-user experts and three novices were observed in a problem-solving context. Their task was to search and evaluate information on the World Wide Web in order to write a research paper about an unfamiliar topic. A cognitive task analysis and verbal protocols were used. Results indicate differences in the ways experts approach and solve problem. These differences were in terms of (a) knowledge base; (b) problem space; (c) strategies; and (d) affect. Novice-expert differences have instructional implications for teachers, Information Technologists and Web site designers. The Problem The purpose of this exploratory study was to compare three experts and three novices in a web-based problem-solving context. This study differs from other studies on problem solving because the task is open-ended and it also includes problemfinding (Getzels & Csikszentmihalyi, 1975). Our interest in the Web stems from the fact that it has the potential to be a powerful educational tool in that it has unlimited resources and ideas of multiple representations (Owston, 1997; Windschitl, 1998). However, searching for information on the Web can be problematic due to the vastness of its database. Immediate access to a large quantity of information that has not been systematically evaluated by publishers or other authorities might present more complex problems than conducting a search in a traditional library (Fung, 1997). Therefore, the users must apply their critical thinking skills and metacognitive and navigational strategies in order to both successfully evaluate the reliability and validity of the sources (Jones, 1997), and to avoid getting lost in cyberspace. How do experts and novices deal with these complexities in searching the Web? Lessons learned from Web experts have significant instructional implications for educators and designers. Expertise: Expert-novice differences have been studied to determine the initial state of the learner (novice), and what is required to become an expert. From this research we know that the course of knowledge acquisition proceeds from a declarative to a procedural, condition-action form (Glaser & Bassock, 1989). This difference is between knowing "what " and knowing "how”. Novices may acquire the same amount of information as experts without knowing the "condition” of its application. Experts, on the other hand, seem to acquire knowledge in relation to its function and applicability (Chi and Glaser, 1985). Novices tend to categorize problems by surface features whereas experts look for patterns and analytic strategies (Chi, Feltovich and Glaser, 1981). Experts have more in-depth prior knowledge and, through chunking, can switch faster from one strategy to another when faced with difficulty on a problem (Presley and McCormick, 1995). Ericsson, Krampe, & Tech-Romer (1993) believe that it takes 10,000 hours or 10 years in the field to become an expert; but due to the relative novelty of the Web, in this study, we define expertise in terms of hours of Web access and not necessarily years of familiarity with the Web. Methodology Our sample consisted of three novices and three experts, all graduate students at the Faculty of Education, McGill University except novice 3 who was a high school student. Experts were Web-based course designers and teachers. Novices, except for 3, shared a common health education and research assistanship background. The criteria that differentiated them was the amount of time spent on the Web. Our novices’ time spent on the Web was on average less than 3 hours a week whereas for experts it averaged 15 hours a week. The task was to search information on the Web about an unfamiliar topic (inquiry-based instruction or IBI) for the purpose of writing a research paper. Performance measures were the differences between how experts and novices search the web to solve the problem at hand (see Table 1). Due to the ill-defined nature of the problem, cognitive task analysis and verbal protocols were used to gather information (Shute, 1997; Ericsson & Simon, 1993). The task lasted approximately 1.5 hours. One researcher asked the informants about their rationale for decision making while searching the Web, the other researcher videotaped the sessions. This division of task was kept constant with all participants. To save time, they were given a socio-demographic and training questionnaire to be filled out before the task. After the task was completed, they were asked about different topics related to Web searching. Their responses were triangulated with the results of the questionnaire and the cognitive task analysis in order to verify whether participants’ actions during the search coincided with the strategies they told us during the post-task interview. Two coding schemes were created. One was used to analyze data gathered through the PARI methodology (Lesgold, Lajoie, Logan, & Eggan, 1990) that resulted in categories of Table 1. The other was used to graphically represent the participants’ problem spaces.