Improving Adaptive Learning Technology through the Use of Response Times

Improving Adaptive Learning Technology through the Use of Response Times Everett Mettler (mettler@ucla.edu) University of California, Los Angeles Department of Psychology, 405 Hilgard Avenue, Los Angeles, CA 90095 USA Christine M. Massey (massey@seas.upenn.edu) University of Pennsylvania 3401 Walnut St., Suite 400A, Philadelphia, PA 19104 USA Philip J. Kellman (kellman@cognet.ucla.edu) University of California, Los Angeles Department of Psychology, 405 Hilgard Avenue, Los Angeles, CA 90095 USA Abstract Adaptive learning techniques have typically scheduled practice using learners' accuracy and item presentation history. We describe an adaptive learning system (Adaptive Response Time Based Sequencing—ARTS) that uses both accuracy and response time (RT) as direct inputs into sequencing. Response times are used to assess learning strength and to determine mastery, making both fluency and accuracy goals of learning. ARTS optimizes spacing by expanding item recurrence intervals as an inverse function of RT. In Experiment 1, we compared ARTS to Atkinson’s (1972) Markov model system using geography learning and found substantially greater learning efficiency for ARTS. In Experiment 2, we deployed the system in a real learning setting. Third graders attending an online school mastered basic multiplication facts in about two hours using ARTS, outperforming a control group using standard instruction. These results suggest that response time-based adaptive learning has remarkable potential to enhance learning in many domains. Keywords: learning; adaptive learning; learning technology; education; instruction and teaching; memory. Introduction Principles of learning and memory applied to instruction might be powerfully amplified in their effects if, through adaptive learning, they can be customized to the needs of individual learners and tasks. Since pioneering work by Atkinson and colleagues (e.g., Atkinson, 1972), various adaptive learning schemes have been proposed (e.g., Pavlik & Anderson, 2008; Wozniak & Gorzalanczyk, 1994). Most systems require prior research to estimate model parameters for particular domains and learners. Sequencing is usually calculated by combining parameters, response accuracy and presentation history in a learning session. We have developed a novel adaptive learning system (Adaptive Response Time Based Sequencing -- ARTS) that uses response times along with accuracy as primary inputs to govern adaptive sequencing in interactive learning. There are two primary reasons to incorporate response times in adaptive learning. First, considerable research indicates the importance of spacing in learning (for a recent review, see Pashler, Rohrer, Cepeda & Carpenter, 2007). When multiple items, categories, or procedures are to be learned, intervening intervals and/or items between presentations of a given item in a learning session can greatly improve the efficiency and durability of learning. Some important benefits of spacing relate to changing spacing as learning progresses. Using response times on interactive trials offers a more direct indicator of learning, making them a useful input into adaptive scheduling. Second, fluency itself is often a goal of learning. Using response times to set and meet learning criteria may offer important benefits for long term retention and fluent use of knowledge in complex problem solving situations. Spacing and Adaptive Learning One powerful spacing effect is that expanding intervals of retrieval practice produce better learning, relative to fixed intervals (Landauer & Bjork, 1978; Cull et al., 1996). Very recent research provides evidence for a substantial advantage of expanding the retrieval interval when material is highly susceptible to forgetting or when intervening material is processed between testing events (Storm, Bjork & Storm, 2010), conditions that apply to many formal learning situations. Most explanations of the value of expanded retrieval intervals, and other spacing principles, involve an underlying notion of learning strength. Learning strength can be thought of as a hypothetical construct related to probability of successful recall on a future test. When a new item is presented, learning strength may be low, but it typically increases with additional learning trials. The value of any new test trial varies with an item's learning strength. Specifically, evidence suggests that difficulty of successful retrieval is a crucial factor (Landauer & Bjork, 1978; Karpicke & Roediger, 2007; Pyc & Rawson, 2009). Pyc & Rawson (2009) labeled this idea the retrieval effort hypothesis : More difficult, but successful, retrievals are more beneficial. They studied the relation of number of successful retrievals to later memory performance, while manipulating the difficulty of those retrievals via number of intervening trials. Greater numbers of intervening trials led to better retention. These investigators also found evidence that, as had been suggested in other work, larger gaps produced longer average response latencies (Pyc & Rawson, 2009), a finding consistent both with the idea that a larger