Retrieval from a Case-Based Reasoning Database

Abstract We examined users' information retrieval performance and their perceptions of their performance for a case-based reasoning knowledge repository to determine how users should be supported for effective use. The users were able to retrieve information from the system and noticed a difference between traditional keyword searching and case-based reasoning searching, but did not understand what caused the difference. This lack of understanding may limit the full potential of alternative search engines that provide results in a different manner than familiar search engines. Introduction Teachers and those who educate them are continually searching for innovative and effective tools to assist with learning to integrate technology into teaching. When teachers are seeking resources for solving a technology integration problem, they turn to databases, work groups, and communities of practice. They look for similar situations to see how problems were solved and then adapt the information that they find to fit their own needs. The Knowledge Innovation for Technology in Education (KITE) project is a web-based knowledge repository with nearly 1000 stories or cases describing the real-life experiences of in-service teachers as they integrate technology into their teaching (Wang, Moore, Wedman, & Shyu, 2003). The technological design of this repository relies on information retrieval with case-based reasoning (CBR), which means that the users can search for cases that are either identical or similar to the desired criteria. Although the repository provides access to a wealth of knowledge, many users are not familiar with the search principles of CBR retrieval systems. With the proliferation of the new search engines, it may be necessary to create support tools, such as job-aids and tutorials that will assist users in effectively using the search interface. In order to determine what type of support users need, we decided to examine their search experience while using the KITE search engine. A survey by Jonassen and Erdelez (2005) identified that unfamiliarity with the new search environment was the most consistent problem for using the CBR database. In another study, Wu (2006) identified the importance of providing users a conceptual description of the search engine in order to facilitate higher search correctness and user satisfaction. Our study builds on this prior research by applying task-based evaluation, conducted in a usability laboratory[l]. In particular, the study collected data about users' search sessions in response to assigned search tasks, and users' perceptions of searching with CBR and keyword search interfaces. The Principles of Case-Based Reasoning According to Jonassen and Hernandez-Serrano (2002), we can prepare professionals to deal with ill-defined and ill-structured problems in the workplace by exposing them to stories generated in the same environment. People will naturally use what they have learned in a previous problem and apply it to a new problem. This principle creates the basis for utilizing cases to generate solutions, and it is used extensively in everyday, common sense reasoning. The process of problem solving that includes analysis, evaluation, synthesis, and conceptualization, produces better judgment and decision-making. From a CBR perspective, the problem-solving approach involves the process of retrieving/adapting of cases with associated solutions (Bradley, 1994). An assumption is that individuals have numerous experiences that have been indexed in their memory to make available to themselves and other people to be used in new situations. An additional assumption is that community knowledge is stored and can be captured in the form of stories (Schank, 1990). The KITE knowledge repository embodies a library of technology integration cases, with each case containing a description of the problem, the solution and/or the outcome (Wang, Moore, Wedman, & Shyu, 2003). …