TEST-TAKING EFFORT AS A PREDICTOR OF PERFORMANCE IN LOW-STAKES TESTS

We studied how time measures can be used as predictors of test-taking performance in low-stakes tests. Our sample consisted of undergraduate students (N = 327) who took a computer-based cognitive abilities test. Our aim was to find how test-takers' motivation manifests itself in test-taking effort. We found that a high test-taking speed is related to low test scores (the correlation between test score and Response Time Effort was r = .71). Also, the mean time for wrong answers per item was smaller than the time for right answers (mean effect size d = .22). We found that performance in low-stakes tests is influenced by two test-taking effort characteristics: the number of items the test-taker attempts to solve and the mean time that is devoted to solve an item (β = .4-.5).We suggest that test-taking motivation should be studied further as it may provide useful information for interpreting results of tests and examinations.

[1]  Donna L. Sundre,et al.  The Student Opinion Scale: A measure of examinee motivation , 2002 .

[2]  Carol L. Barry,et al.  Do Examinees Have Similar Test-Taking Effort? A High-Stakes Question for Low-Stakes Testing , 2010 .

[3]  C. Spearman.  The Abilities of Man their Nature and Measurement , 2020, Nature.

[4]  Jeffrey K. Smith,et al.  Consequence of Performance, Test, Motivation, and Mentally Taxing Items , 1995 .

[5]  L. Cronbach.  Essentials of psychological testing , 1960 .

[6]  Steven L. Wise,et al.  Response Time Effort: A New Measure of Examinee Motivation in Computer-Based Tests , 2005 .

[7]  O. Wilhelm,et al.  The relation of speeded and unspeeded reasoning with mental speed , 2002 .

[8]  W. Furneaux,et al.  Intellectual Abilities and Problem-solving Behaviour , 1973 .

[9]  C. SPEARMAN,et al.  The Measurement of Intelligence , 1927, Nature.

[10]  A. Jensen,et al.  The g factor , 1996, Nature.

[11]  Dena A. Pastor,et al.  Correlates of Rapid-Guessing Behavior in Low-Stakes Testing: Implications for Test Development and Measurement Practice , 2009 .

[12]  Jürgen Baumert,et al.  Test motivation in the assessment of student skills: The effects of incentives on motivation and performance , 2001 .

[13]  Hanna Eklöf,et al.  Development and Validation of Scores From an Instrument Measuring Student Test-Taking Motivation , 2006 .

[14]  Steven L. Wise,et al.  Examinee Noneffort and the Validity of Program Assessment Results , 2010 .

[15]  Rolf Loeber,et al.  Role of test motivation in intelligence testing , 2011, Proceedings of the National Academy of Sciences.

[16]  J. Eccles,et al.  Motivational beliefs, values, and goals. , 2002, Annual review of psychology.

[17]  Philip A. Vernon,et al.  Intelligence and speed of information-processing: A review of 50 years of research , 2008 .

[18]  Christine E. DeMars,et al.  Low Examinee Effort in Low-Stakes Assessment: Problems and Potential Solutions , 2005 .

[19]  Richard E. Mayer.  Thorndike, Edward L. , 2005 .

[20]  R. Herrnstein,et al.  The Bell Curve , 1994 .

[21]  Paul De Boeck,et al.  How Much Power and Speed Is Measured in This Test? , 2013, Assessment.

[22]  Denny Borsboom,et al.  Cognitive psychology meets psychometric theory: on the relation between process models for decision making and latent variable models for individual differences. , 2011, Psychological review.

[23]  Investigating Differences Between Low- and High-Stakes Test Performance on a General Education Exam , 2008 .

[24]  Jean-Paul Fox,et al.  Evaluating cognitive theory: a joint modeling approach using responses and response times. , 2009, Psychological methods.

[25]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[26]  Robert J. Mislevy,et al.  WHAT CAN WE LEARN FROM INTERNATIONAL ASSESSMENTS , 1995 .

[27]  Donna L. Sundre,et al.  An exploration of the psychology of the examinee: Can examinee self-regulation and test-taking motivation predict consequential and non-consequential test performance? , 2004 .

[28]  Deborah L. Schnipke,et al.  Modeling Item Response Times With a Two-State Mixture Model: A New Method of Measuring , 1997 .

[29]  Hanna Eklöf,et al.  Skill and will: test‐taking motivation and assessment quality , 2010 .

[30]  J. Wicherts,et al.  Test Anxiety and the Validity of Cognitive Tests: A Confirmatory Factor Analysis Perspective and Some Empirical Findings. , 2010 .

[31]  Willem J. van der Linden,et al.  Test Design and Speededness , 2011 .

[32]  Carol L. Barry,et al.  1 Exploring Change in Test-Taking Motivation , 2009 .

[33]  J. Eccles,et al.  Expectancy-Value Theory of Achievement Motivation. , 2000, Contemporary educational psychology.

[34]  C. Brand,et al.  The g Factor: General Intelligence and Its Implications , 1996 .