Enhancing a Computer-Based Testing Environment with Optimum Item Response Time

Background:As technology has become more advanced and accessible in teaching and learning settings, there has been an upward trend in computer-based testing in the last decades.Materials and methods:The present experimental study examines students’ behaviors during computer-based testing in two different conditions and explores to what extent the test results were affected by those conditions.Results:Study results indicate that some of the psychometric features of a test (reliability and validity) could be enriched on computer-based testing when students are provided optimum item response time. In addition, it was found that providing optimum response time for each item influenced the students in the experimental group to not engage in rapid guess behaviors. Thereby, students spent a reasonable amount of time answering the questions, which resulted in more reliable and valid scores, aforementioned. Lastly, there was no statistically significant difference in two groups in terms of student performance.Conclusions:Psychometric features of a test could be enriched when students are provided additional features on a computer-based testing environment.

[1]  Daniel J. Bernstein,et al.  An Examination of the Equivalence between Non-Adaptive Computer-Based and Traditional Testing , 2001 .

[2]  Masood Jawaid,et al.  Computer Based Assessment (CBA): Perception of residents at Dow University of Health Sciences , 2014, Pakistan journal of medical sciences.

[3]  Steven L. Wise,et al.  Taking the Time to Improve the Validity of Low-Stakes Tests: The Effort-Monitoring CBT. , 2006 .

[4]  Muhterem Dindar,et al.  Multimedia in Test Items: Animated Questions vs. Static Graphics Questions , 2013 .

[5]  L. S. Vygotksy.  Mind in society: the development of higher psychological processes , 1978 .

[6]  Mary Peat,et al.  Supporting student learning: the use of computer-based formative assessment modules , 2002, Br. J. Educ. Technol..

[7]  P. Pintrich.  Understanding Interference and Inhibition Processes from a Motivational and Self-Regulated Learning Perspective: Comments on Dempster and Corkill , 1999 .

[8]  E. Zillmer,et al.  Computer-Based Assessment of Sports-Related Concussion , 2003, Applied neuropsychology.

[9]  Concurrent and Predictive Validity of Computer-adaptive Freshman English Test for College Freshman English in Taiwan , 2014 .

[10]  Michael Russell,et al.  Computer-based Testing and Validity: a look back into the future , 2003 .

[11]  Meredith Lawley,et al.  Benefits and drawbacks of computer-based assessment and feedback systems: Student and educator perspectives , 2016, Br. J. Educ. Technol..

[12]  Vincent Koenig,et al.  Concept maps: A useful and usable tool for computer-based knowledge assessment? A literature review with a focus on usability , 2014, Comput. Educ..

[13]  P. Schatz,et al.  Cross-Validation of Measures Used for Computer-Based Assessment of Concussion , 2006, Applied neuropsychology.

[14]  Yan Piaw Chua,et al.  Effects of computer-based educational achievement test on test performance and test takers' motivation , 2013, Comput. Hum. Behav..

[15]  Steven L. Wise,et al.  Setting the Response Time Threshold Parameter to Differentiate Solution Behavior From Rapid-Guessing Behavior , 2007 .

[16]  Rolph E. Anderson,et al.  Multivariate Data Analysis (7th ed. , 2009 .

[17]  J. Abedi.  The Use of Computer Technology in Designing Appropriate Test Accommodations for English Language Learners , 2014 .

[18]  Ian Jones,et al.  Touch Arithmetic: A process-based Computer-Aided Assessment approach for capture of problem solving steps in the context of elementary mathematics , 2014, Comput. Educ..

[19]  Philip Schatz,et al.  Applications of Computer-based Neuropsychological Assessment , 2002, The Journal of head trauma rehabilitation.

[20]  Hanho Jeong.  A comparative study of scores on computer-based tests and paper-based tests , 2014, Behav. Inf. Technol..

[21]  M. J. Allen.  Introduction to Measurement Theory , 1979 .

[22]  Erhan Delen,et al.  A computer-based peer nomination form to identify gifted and talented students , 2014 .

[23]  B. Zimmerman.  A social cognitive view of self-regulated academic learning. , 1989 .

[24]  S DeAngelis,et al.  Equivalency of computer-based and paper-and-pencil testing. , 2000, Journal of allied health.

[25]  M. Prensky.  Digital Natives, Digital Immigrants Part 1 , 2001 .

[26]  C. Ricketts,et al.  Improving Student Performance Through Computer-based Assessment: Insights from recent research , 2002 .

[27]  Francisco José García-Peñalvo,et al.  Advances in E-Learning: Experiences and Methodologies , 2008 .

[28]  B. Rosner.  Percentage Points for a Generalized ESD Many-Outlier Procedure , 1983 .

[29]  Steven L. Wise,et al.  An Application of Item Response Time: The Effort‐Moderated IRT Model , 2006 .

[30]  H. Douglas Brown,et al.  Language Assessment: Principles and Classroom Practices , 2003 .

[31]  Roy B. Clariana,et al.  Paper-based versus computer-based assessment: key factors associated with the test mode effect , 2002, Br. J. Educ. Technol..

[32]  Julio Olea,et al.  eCAT-Listening: design and psychometric properties of a computerized adaptive test on English Listening. , 2011, Psicothema.

[33]  Steven L. Wise,et al.  Response Time Effort: A New Measure of Examinee Motivation in Computer-Based Tests , 2005 .

[34]  A. E. Ogletree,et al.  Transition to Online Assessments: A Personal Perspective of Meeting Common Core State Standards in an Elementary School in Georgia , 2014 .

[35]  Jodi L. Davenport,et al.  Comparing three online testing modalities: Using static, active, and interactive online testing modalities to assess middle school students' understanding of fundamental ideas and use of inquiry skills related to ecosystems , 2014 .

[36]  Yi-Hsuan Lee,et al.  Using response time to investigate students' test-taking behaviors in a NAEP computer-based study , 2014, Large-scale Assessments in Education.

[37]  Okan Bulut,et al.  Application of Computerized Adaptive Testing to Entrance Examination for Graduate Studies in Turkey , 2012 .

[38]  Yan Piaw Chua,et al.  Effects of computer-based testing on test performance and testing motivation , 2012, Comput. Hum. Behav..

[39]  Mohamad Jafre Zainol Abidin,et al.  ScienceDirect International Conference on Current Trends in ELT Comparability of Test Results of Computer Based Tests ( CBT ) and Paper and Pencil Tests ( PPT ) among English Language Learners in Iran , 2014 .

[40]  Erhan Delen,et al.  Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments , 2014, Comput. Educ..