Optimising Moodle quizzes for online assessments

BackgroundComputer-aided learning management systems (LMSs) are widely used in higher education and are viewed as beneficial when transitioning from conventional face-to-face teaching to fully online courses. While LMSs have unique tools for transferring and assessing knowledge, their ability to engage and assess learners needs further investigation. This paper focuses on a study examining the LMS “Moodle” to ascertain the effectiveness of “Moodle quizzes” to improve, assess and distinguish knowledge in a civil engineering course at an Australian university. The course has a database comprising 62 formative and 61 summative quiz questions with embedded text, images, audio and video. This study investigates the use of these quiz questions with four course cohorts and 169 students. The quizzes assessed competencies of students during various stages of a study period through automated marking. The suitability of questions to assess and distinguish student knowledge levels was determined using a psychometric analysis based on facility index (FI) and the discrimination index (DI) statistics embedded within the Moodle quizzes.ResultsThis study highlights strategies used to set and review quiz questions for formative and summative assessments. Results indicated that students were engaged and satisfied in the formative assessment because they viewed the interactive videos between 2 and 6 times and 65% of students attempted all the formative questions. The FI indicated student pass rate for the summative questions and DI indicated the difficulty of these questions, while the combination of FI and DI results separated students with different knowledge levels. Using these Moodle statistics provided information to make effective decisions on how to improve the summative quizzes.ConclusionThe multimodal quizzes were effective in teaching and assessing a theoretical engineering course and provided efficient methods to replace conventional assessments. The FI and DI indexes are useful statistical tools in redesigning appropriate sets of questions. Time-poor academics will benefit from using these easily attainable Moodle statistics to inform decisions while revising the quizzes and making assessments more autonomous.

[1]  Ramón Corral-Higuera,et al.  Calibrating Questionnaires by Psychometric Analysis to Evaluate Knowledge , 2013 .

[2]  B. Cook,et al.  Active learning through online quizzes: better learning and less (busy) work , 2017 .

[3]  J. Tondeur,et al.  Improving teacher professional development for online and blended learning: a systematic meta-aggregative review , 2019, Educational Technology Research and Development.

[4]  Rebecca Francis,et al.  Engaging with blended learning to improve students’ learning outcomes , 2013 .

[5]  Desmond Adair,et al.  Time pressure in scenario-based online construction safety quizzes and its effect on students’ performance , 2017 .

[6]  Santiago Vicente,et al.  Influencia del conocimiento matemático y situacional en la resolución de problemas aritméticos verbales: ayudas textuales y gráficas , 2008 .

[7]  Marc Fuertes-Alpiste,et al.  Exploring the effectiveness of continuous activity with automatic feedback in online calculus , 2018, Comput. Appl. Eng. Educ..

[8]  Donita Cohen,et al.  Online quizzes in a virtual learning environment as a tool for formative assessment , 2016 .

[9]  Suraya Miskon,et al.  Initial recommendations of MOOCs characteristics for academic discipline clusters , 2016 .

[10]  S. R. Ting,et al.  Predicting academic success of first-year engineering students from standardized test scores and psychosocial variables , 2001 .

[11]  Ken Wojcikowski,et al.  Immediate detailed feedback to test-enhanced learning: An effective online educational tool , 2013, Medical teacher.

[12]  Jared A Danielson,et al.  A quiz becomes a multidirectional dialogue with Web-based instructional tools for an anatomical pathology rotation. , 2005, Journal of veterinary medical education.

[13]  Misook Heo,et al.  The Impact of Computer Augmented Online Learning and Assessment Tool , 2005, J. Educ. Technol. Soc..

[14]  S. G. Martins.  Weekly online quizzes to a mathematics course for engineering students , 2016 .

[15]  Shaobo Huang,et al.  Predicting student academic performance in an engineering dynamics course: A comparison of four types of predictive mathematical models , 2013, Comput. Educ..

[16]  Chrysostomos D. Stylios,et al.  A Trustworthy and Privacy Preserving Model for Online Competence Evaluation System , 2018, DepCoS-RELCOMEX.

[17]  M. V. Kustritz,et al.  Canine Theriogenology for Dog Enthusiasts: Teaching Methodology and Outcomes in a Massive Open Online Course (MOOC) , 2014 .

[18]  Wojciech Zamojski.  Contemporary Complex Systems and Their Dependability - Proceedings of the 13th International Conference on Dependability and Complex Systems DepCoS-RELCOMEX, July 2-6, 2018, Brunów, Poland , 2019, DepCoS-RELCOMEX.

[19]  Daniel Sullivan,et al.  An Integrated Approach to Preempt Cheating on Asynchronous, Objective, Online Assessments in Graduate Business Classes , 2016 .

[20]  Catherine Thevenot.  Arithmetic word problem solving: evidence for the construction of a mental model. , 2010, Acta psychologica.

[21]  Jean-Michel I. Maarek.  Benefits of Active Learning Embedded in Online Content Material Supporting a Flipped Classroom , 2018 .

[22]  Shahron Williams van Rooij,et al.  Open-source learning management systems: a predictive model for higher education , 2012 .

[23]  Robert H. Ennis.  Critical thinking assessment , 1993 .

[24]  Cansu Cigdem Aydin,et al.  OPEN SOURCE LEARNING MANAGEMENT SYSTEMS IN DISTANCE LEARNING , 2010 .

[25]  Christopher F. Bauer,et al.  Student perceptions of immediate feedback testing in student centered chemistry classes , 2018 .

[26]  Sarah Ellen Ransdell,et al.  Predicting college success: the importance of ability and non-cognitive variables , 2001 .

[27]  John D Mahan,et al.  Utility of intermittent online quizzes as an early warning for residents at risk of failing the pediatric board certification examination , 2018, BMC Medical Education.

[28]  M. R. Root Kustritz.  Canine theriogenology for dog enthusiasts: teaching methodology and outcomes in a massive open online course (MOOC). , 2014, Journal of veterinary medical education.

[29]  Josep M. Duart,et al.  New era and challenges for a consolidated journal in educational technology in higher education , 2016, International Journal of Educational Technology in Higher Education.

[30]  V. Dinh,et al.  Effectiveness of a 1-Hour Extended Focused Assessment With Sonography in Trauma Session in the Medical Student Surgery Clerkship. , 2017, Journal of surgical education.

[31]  J. Broadbent,et al.  Profiles in self-regulated learning and their correlates for online and blended learning students , 2018 .

[32]  S. O’Flynn,et al.  A systematic review of stakeholder views of selection methods for medical schools admission , 2018, BMC medical education.

[33]  Vera D Rinaldi,et al.  Evaluating a technology supported interactive response system during the laboratory section of a histology course , 2017, Anatomical sciences education.

[34]  Marta Ginovart Gisbert,et al.  Moodle quizzes for assessing statistical topics in engineering studies , 2010 .

[35]  J. West,et al.  Enhancing the assessment experience: improving student perceptions, engagement and understanding using online video feedback , 2016 .

[36]  Rosalind James,et al.  Tertiary student attitudes to invigilated, online summative examinations , 2016, International Journal of Educational Technology in Higher Education.

[37]  Pierre Barrouillet,et al.  Why does placing the question before an arithmetic word problem improve performance? A situation model account , 2007, Quarterly journal of experimental psychology.