Computer-Based and Paper-Based Testing: Does the Test Administration Mode Influence the Reliability and Validity of Achievement Tests?.

This article reports the findings of a study that sought to investigate whether computer-based vs. paper-based test-delivery mode has an impact on the reliability and validity of an achievement test for a pedagogical content knowledge course in an English teacher education program. A total of 97 university students enrolled in the English as a foreign language (EFL) teacher education program were randomly assigned to the experimental group that took the computer-based achievement test online and the control group that took the same test in paper-and-pencil based format. Results of Spearman Rank order and Mann-Whitney U tests indicated that test-delivery mode did not have any impact on the reliability and validity of the tests administered in either way. Findings also demonstrated that there was not any significant difference in test scores between participants who took the computer-based test and those who took the paper-based test. Findings were discussed in terms of the idea that computer technology could be integrated into the curriculum not only for instructional practices but also for assessment purposes.

[1]  William C. Treurniet,et al.  Extended Reading of Continuous Text on Television Screens , 1982 .

[2]  Susan Gross,et al.  Use of a Computerized Adaptive Testing Model for ESOL/Bilingual Entry/Exit Decision Making: A Preliminary Study. , 1986 .

[3]  Stephen E. Newstead,et al.  Proof-reading on VDUs , 1987 .

[4]  A. Hughes.  Testing for language teachers , 1989 .

[5]  Robert F. Conry,et al.  Effects of Computer-Based Tests on the Achievement, Anxiety, and Attitudes of Grade 10 Science Students , 1991 .

[6]  Cynthia G. Parshall,et al.  Computer Testing versus Paper-and-Pencil Testing: An Analysis of Examinee Characteristics Associated with Mode Effect. , 1993 .

[7]  Vance Stevens,et al.  Computer assisted language learning and testing: Research issues and practice , 1993 .

[8]  Andrew Dillon,et al.  Designing Usable Electronic Text: Ergonomic Aspects Of Human Information Usage , 1994 .

[9]  Alan C. Bugbee,et al.  The Equivalence of Paper-and-Pencil and Computer-Based Testing. , 1996 .

[10]  E. Shohamy.  Second Language Assessment , 1997 .

[11]  C. Chapelle.  Construct definition and validity inquiry in SLA research , 1999 .

[12]  Peter Brusilovsky,et al.  Web-based Testing for Distance Education , 1999, WebNet.

[13]  Carol A. Chapelle,et al.  VALIDITY IN LANGUAGE ASSESSMENT , 1999, Annual Review of Applied Linguistics.

[14]  Neil J. Dorans,et al.  Caveats, Pitfalls, and Unexpected Consequences of Implementing Large-Scale Computerized Testing , 2000 .

[15]  J. Charles Alderson,et al.  Technology in testing: the present and the future , 2000 .

[16]  J. Mortensen,et al.  A web-based testing system with dynamic question generation , 2001, 31st Annual Frontiers in Education Conference. Impact on Engineering and Science Education. Conference Proceedings (Cat. No.01CH37193).

[17]  Tianyou Wang,et al.  Evaluating Comparability in Computerized Adaptive Testing: Issues, Criteria and an Example , 2001 .

[18]  Carsten Roever,et al.  WEB-BASED LANGUAGE TESTING , 2001 .

[19]  Cynthia G. Parshall,et al.  Practical Considerations in Computer-Based Testing , 2002 .

[20]  R. Bennett.  Online Assessment and the Comparability of Score Meaning , 2003 .

[21]  Jaap Scheerens,et al.  Educational eveluation, assessment, and monitoring : a systemic approach , 2003 .

[22]  H. Douglas Brown,et al.  Language Assessment: Principles and Classroom Practices , 2003 .

[23]  Jaeyool Boo,et al.  Comparability of a paper-based language test and a computer-based language test , 2003 .

[24]  Michael Russell,et al.  Computer-based Testing and Validity: a look back into the future , 2003 .

[25]  Mariana Lilley,et al.  The development and evaluation of a software prototype for computer-adaptive testing , 2004, Comput. Educ..

[26]  Thomas Hoffmann,et al.  Examining the Effect of Computer-Based Passage Presentation on Reading Test Performance , 2005 .

[27]  Eduardo Guzmán,et al.  Self-assessment in a feasible, adaptive web-based testing system , 2005, IEEE Transactions on Education.

[28]  G. Stobart.  The validity of formative assessment , 2011 .

[29]  Shudong Wang,et al.  A Meta-Analysis of Testing Mode Effects in Grade K-12 Mathematics Tests , 2007 .

[30]  Do Hong Kim,et al.  Comparability of Computer and Paper-and-Pencil Versions of Algebra and Biology Assessments , 2007 .

[31]  Fred Davidson,et al.  Language Testing and Assessment: An Advanced Resource Book , 2007 .

[32]  Malcolm L. Van Blerkom.  Measurement and Statistics for Teachers , 2008 .

[33]  Ayse Oguz,et al.  Computer-based testing: An alternative for the assessment of Turkish undergraduate students , 2008, Comput. Educ..

[34]  J. Noyes,et al.  Computer- vs. paper-based tasks: Are they equivalent? , 2008, Ergonomics.

[35]  Jenifer Larson-Hall,et al.  A Guide to Doing Statistics in Second Language Research Using SPSS , 2009 .

[36]  Michael Levy,et al.  Computer applications in second language acquisition : Foundations for teaching , testing and research , 2009 .

[37]  John Dermo,et al.  e-Assessment and the student learning experience: A survey of student perceptions of e-assessment , 2009, Br. J. Educ. Technol..

[38]  George Palaigeorgiou,et al.  Computer based testing using "digital ink": Participatory design of a Tablet PC based assessment application for secondary education , 2009, Comput. Educ..

[39]  Olson,et al.  Computer-Based & Paper-Pencil Test Comparability Studies , 2010 .

[40]  Hong Wang,et al.  Comparability of Computerized Adaptive and Paper-Pencil Tests , 2010 .

[41]  Contextual clues in Semi-direct interviews for computer assisted language testing , 2010 .

[42]  Wangjie,et al.  Advantages and disadvantages of Computer-based Testing: A case study of service learning , 2010, The 2nd International Conference on Information Science and Engineering.

[43]  Şeniz Özalp-Yaman,et al.  Paper-based versus computer-based testing in engineering education , 2010, IEEE EDUCON 2010 Conference.

[44]  Reima Abobaker,et al.  Assessing Language through Computer Technology , 2012 .

[45]  Yan Piaw Chua,et al.  Effects of computer-based testing on test performance and testing motivation , 2012, Comput. Hum. Behav..

[46]  Tsung-Hsun Tsai,et al.  A Score Comparability Study for the NBDHE , 2013, Evaluation & the health professions.

[47]  Hanho Jeong.  A comparative study of scores on computer-based tests and paper-based tests , 2014, Behav. Inf. Technol..

[48]  Mohamad Jafre Zainol Abidin,et al.  ScienceDirect International Conference on Current Trends in ELT Comparability of Test Results of Computer Based Tests ( CBT ) and Paper and Pencil Tests ( PPT ) among English Language Learners in Iran , 2014 .

[49]  Erhan Delen,et al.  Enhancing a Computer-Based Testing Environment with Optimum Item Response Time , 2015 .

[50]  Kiersten Kenning Hensley,et al.  Examining the effects of paper-based and computer-based modes of assessment on mathematics curriculum-based measurement , 2015 .

[51]  Tracy Logan.  The influence of test mode and visuospatial ability on mathematics assessment performance , 2015 .

[52]  B. Hakim.  Comparative Study on Validity of Paper-Based Test and Computer-Based Test in the Context of Educational and Psychological Assessment among Arab Students , 2017 .

[53]  Susan M. Gass,et al.  Second Language Research: Methodology and Design , 2021 .