Raving about Ravens : modelling speed-accuracy in intelligence tests

The effect of time pressure on performance on intelligence tests is a long standing problem. In this study a computerised version of the Ravens Advanced Progressive Matrices was administered using 3 different forms of instructions: control, speed pressure, and accuracy pressure. Analyses used Rasch measures of participant ability and item difficulty, and the time each participant took to solve each problem. Raw scores were, surprisingly, more useful than Rasch measures. The time pressure group were faster but scored less well than the other two groups. Raw score had a small but significant correlation with total test time. Brighter participants took less time for easy items, but more time for hard items, which were both slower and more variable than easier items. Mean and SD were more consistent for total time than for either correct or error time. Effective models will need to incorporate these diverse results. The Raven’s Progressive Matrices is one of the most successful culture fair tests of cognitive functioning (Raven, 1956; Raven, Court, & Raven, 1988; Salthouse, 2005). Like all tests and examinations, performance is potentially subject to time pressures. The first, and most practical aim, of this study is to make use of computerised administration to investigate those time pressures at the item level. A more theoretical aim is to produce a model of performance for this complex task that parallels models of simple perceptual decision making. The aim is to generate for each individual one parameters that corresponds to rate of accumulation of useful information, and one parameter that corresponds to speed bias (like the separation of barriers in a random walk). There have been many studies comparing performance on elementary tasks such as simple and choice reaction time with performance on some version of the Raven’s. Typically, such studies use mean reaction time and d’ or Luce’s choice, ln(η) for the elementary processes, but only raw score for the Raven’s (Beh, Roberts, & Prichard Levy, 1994; Fink & Neubauer, 2001; Salthouse, 2005). Even total time, easily obtained with a stopwatch, is rarely used as a peformance measure for the Raven’s. Studies looking at time for individual items, requiring computerised administration are even rarer. There are however, several studies that analyse Ravens results using the Rasch model (Alderton & Larson, 1990; Forbes, 1964; Gallini, 1983; Green & Kluever, 1992; Pitariu, 1986). The two or three parameter Rasch models give participant ability and item difficulty measures that depend on logits of probabilities and are hence very similar to the bias and sensitivity measure of choice model. The present study measures time per item for each person for each item. It explores four measures of individual person performance: raw score, and one, two and three parameter ability. The relation between time per item for the 36 items and individual performance is explored using exploratory principal components analysis. The relation between item difficulty (proportion of people giving correct response or one, two or three parameter item difficulty) to time per item was also explored.