Developing a Predictive Model of Postcompletion Errors

Developing a Predictive Model of Postcompletion Errors Raj M. Ratwani (rratwani@gmu.edu) 1,2 J. Gregory Trafton (trafton@itd.nrl.navy.mil) 1 Naval Research Laboratory 1 Washington, D.C. George Mason University 2 Fairfax, VA Abstract A postcompletion error is a type of procedural error that occurs after the main goal of a task has been accomplished. There is a strong theoretical foundation accounting for postcompletion errors (Altmann & Trafton, 2002; Byrne & Bovair, 1997). This theoretical foundation has been leveraged to develop a logistic regression model of postcompletion errors based on reaction time and eye movement measures (Ratwani, McCurry, & Trafton, 2008). The work presented here further develops and extends this predictive model by (1) validating the model and the general set of predictors on a new task to test the robustness of the model, and (2) determining which specific theoretical components are most important to postcompletion error prediction. Keywords: Procedural error; postcompletion error Introduction Even while performing a routine procedural task that has been performed hundreds of times in the past, occasional errors still occur (i.e. a slip or lapse) (Reason, 1990). These procedural errors have been termed skill-based errors (Rasmussen & Jensen, 1974) and occur despite having the correct knowledge of how to perform a particular task. A common type of procedural error is the postcompletion error; this error is associated with forgetting a final step which occurs after the main goal of a task has been completed (Byrne & Bovair, 1997). There are several examples of postcompletion errors, such as leaving an original document on the glass of a copy machine after making a copy or failing to attach a document to an email message. The holy grail of error research is to be able to predict when an error is going to occur before the error actually occurs (Reason, 1990). In order to be able to make advances toward error prediction, strong theoretical accounts of the cognitive mechanisms underlying procedural errors are required. In the case of postcompletion errors, these theoretical accounts do exist; Byrne and Bovair (1997) have put forward a theory specific to postcompletion errors, and Altmann and Trafton (2002) explain postcompletion errors using a general theory of goal memory, called memory for goals. Both theories are activation-based memory accounts and there is substantial overlap between the theories. Byrne and Bovair (1997) suggest that postcompletion errors are due to goal forgetting and inattention to the postcompletion step. Specifically, postcompletion errors occur because the postcompletion step of a task is not maintained in working memory and, thus, is not executed as part of the task. The main goal of a task and the subsequent subgoals are stored in working memory and must remain active to be executed. The main goal provides activation to the subgoals. When the main goal of a task is satisfied, the goal no longer provides activation to the subgoals; consequently, the remaining subgoals may fall below threshold and may not be executed. The memory for goals theory (Altmann & Trafton, 2002) accounts for goal-directed behavior with the constructs of activation and associative priming. The theory suggests that behavior is directed by the current most active goal and that the activation level of goals decay over time. In order for a goal to direct behavior, the goal must have enough activation to overcome interference from previous goals; thus, the goal must reach a certain threshold to actually direct behavior. Goal activation is determined by two main constraints. The strengthening constraint suggests that the history of a goal (i.e. how frequently and recently the goal was retrieved) will impact goal activation. The priming constraint suggests that a pending goal will be retrieved and will direct behavior if the goal is primed from an associated cue. These cues can either be in the mental or environmental context. Leveraging these theoretical accounts, Ratwani, McCurry and Trafton (2008) developed a logistic regression model predicting when a postcompletion error will occur on a computer-based procedural task. A logistic regression analysis was used because the outcome variable (occurrence of an error) was a dichotomous variable, which violates many of the assumptions of standard linear regression (Tabachnick & Fidell, 2001). A simple description of logistic regression is that it is a multiple linear regression model with a dichotomous variable as an outcome variable; a more detailed description can be found in Peng, Lee, & Ingersoll (2002). To build their logistic regression model, Ratwani et. al. (2008) recorded and developed eye movement and reaction time measures as the behavioral indicators of the cognitive constructs outlined by the Byrne and Bovair (1997) and Altmann and Trafton (2002) theories. Specifically, three predictors were used in the logistic regression model: time between actions, total number of fixations between actions, and fixation on the postcompletion action button. The logistic regression model was as follows: Predicted logit of Error = .12 + (time x -.001) + (total fixations x .63) + (postcompletion fixation x -5.7)