Can online course-based assessment methods be fair and equitable? Relationships between students' preferences and performance within online and offline assessments

To address concerns raised regarding the use of online course-based summative assessment methods, a quasi-experimental design was implemented in which students who completed a summative assessment either online or offline were compared on performance scores when using their self-reported preferred or non-preferred modes. Performance scores were found not to differ depending on whether the assessment was completed in the preferred or non-preferred mode. These findings provide preliminary support for the validity of online assessment methods. Future studies could help determine the extent to which this finding generalizes beyond the assessment procedures and type of sample used here. Suggestions for follow-up studies are offered, including exploring the validity of more complex computer-related online assessment tasks and investigating the impact of using preferred and non-preferred modes upon the quality of the student experience. © 2012 Wiley Periodicals, Inc.

[1]  Vivien Sieber.  Diagnostic online assessment of basic IT skills in 1st-year undergraduates in the Medical Sciences Division, University of Oxford , 2009, Br. J. Educ. Technol..

[2]  Brian C. Cronk,et al.  Personality research on the Internet: A comparison of Web-based and traditional instruments in take-home and in-class settings , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[3]  D. Henly,et al.  Use of Web-based formative assessment to support student learning in a metabolism/nutrition unit. , 2003, European journal of dental education : official journal of the Association for Dental Education in Europe.

[4]  Jocelyn Armarego,et al.  The Development of On-line Tests Based on Multiple Choice Questions , 2003, Web-Powered Databases.

[5]  Andrew Boyle,et al.  Sophisticated tasks in e‐assessment: what are they and what are their benefits? , 2009 .

[6]  Mark J. Brosnan,et al.  The impact of computer anxiety and self-efficacy upon performance , 1998, J. Comput. Assist. Learn..

[7]  Keith J. Topping,et al.  Student reflections on formative e‐assessment: expectations and perceptions , 2008 .

[8]  Reg Dennick,et al.  Online eAssessment: AMEE Guide No. 39 , 2009, Medical teacher.

[9]  R. N. Davis,et al.  Web-based administration of a personality questionnaire: Comparison with traditional methods , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[10]  J. Epstein,et al.  Insuring sample equivalence across internet and paper-and-pencil assessments , 2001, Comput. Hum. Behav..

[11]  John Dermo,et al.  e-Assessment and the student learning experience: A survey of student perceptions of e-assessment , 2009, Br. J. Educ. Technol..

[12]  Claire Hewson.  Web-MCQ: A set of methods and freely available open source code for administering online multiple choice question assessments , 2007, Behavior research methods.

[13]  John L. Smith,et al.  Using the Internet for psychological research: personality testing on the World Wide Web. , 1999, British journal of psychology.

[14]  Bill Warburton,et al.  Quick win or slow burn: modelling UK HE CAA uptake , 2009 .

[15]  Ruth Grady,et al.  Staff and student perceptions of computer-assisted assessment for physiology practical classes. , 2006, Advances in physiology education.

[16]  Steve New,et al.  INQUIRE: a case study in evaluating the potential of online MCQ tests in a discursive subject , 2004 .

[17]  Juan Herrero,et al.  Short Web-based versions of the perceived stress (PSS) and Center for Epidemiological Studies-Depression (CESD) Scales: a comparison to pencil and paper responses among Internet users , 2006, Comput. Hum. Behav..

[18]  Judith Watson,et al.  Does regular online testing enhance student learning in the numerical sciences? Robust evidence from a large data set , 2009, Br. J. Educ. Technol..

[19]  Paul Meyerson,et al.  Validating Internet research: A test of the psychometric equivalence of Internet and in-person samples , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[20]  Min Liu,et al.  Exploring the use of multimedia examination formats in undergraduate teaching: results from the fielding testing , 2001, Comput. Hum. Behav..

[21]  Jerrell C. Cassady,et al.  The Effects of Online Formative and Summative Assessment on Test Anxiety and Performance , 2005 .

[22]  Denise Whitelock.  Editorial: e-assessment: developing new dialogues for the digital age , 2009, Br. J. Educ. Technol..

[23]  Doug Mahar,et al.  The effects of computer anxiety, state anxiety, and computer experience on users' performance of computer based tasks , 1997 .

[24]  John P. Charlton,et al.  Comparing Online and Offline Administration of Multiple Choice Question Assessments to Psychology Undergraduates: Do Assessment Modality or Computer Attitudes Influence Performance? , 2007 .

[25]  Thomas Buchanan,et al.  The efficacy of a World-Wide Web mediated formative assessment , 2001, J. Comput. Assist. Learn..

[26]  Mark J. Brosnan,et al.  Does computer anxiety reach levels which conform to DSM IV criteria for specific phobia? , 2007, Comput. Hum. Behav..

[27]  Sally E. Jordan,et al.  e-Assessment for learning? The potential of short-answer free-text questions with tailored feedback , 2009, Br. J. Educ. Technol..

[28]  Steve Cushion,et al.  Design and Evaluation of an Online Test: Assessment Conceived as a Complementary CALL Tool , 2003 .

[29]  C. Ricketts,et al.  Improving Student Performance Through Computer-based Assessment: Insights from recent research , 2002 .

[30]  Claire Hewson,et al.  Measuring health beliefs on the Internet: A comparison of paper and Internet administrations of the Multidimensional Health Locus of Control Scale , 2005, Behavior research methods.

[31]  Joseph J. Pedulla,et al.  Performance Differences According to Test Mode and Computer Familiarity on a Practice Graduate Record Exam , 2002 .

[32]  Shoichi Matsumura,et al.  Computer Anxiety and Students' Preferred Feedback Methods in EFL Writing. , 2004 .

[33]  Pru Marriott,et al.  Students' evaluation of the use of online summative assessment on an undergraduate financial accounting module , 2009, Br. J. Educ. Technol..

[34]  Alan Sangster,et al.  Using internet-based on-line assessment: A case study , 2005 .