The role of attitudinal factors in mathematical on-line assessments: a study of undergraduate STEM students

Abstract This study explores student attitudes to the use of substantive on-line assessments that require mathematical answers. Since there is limited guidance available for their use in a university setting, our goal is to learn what are the important aspects in student acceptance of e-assessments that support learning of mathematical subjects in higher education. To that end we analyse the effects of a variety of attitudinal factors towards such assessments amongst a cross-section of first year students in an English university, using a detailed questionnaire. These students were all previously exposed to on-line assessments containing substantial mathematical work, including testing of and feedback on the algebraic structure of their answers, based on identifiable misconceptions underlying these answers. Since students received highly tailored feedback, the expectation was that the usefulness of this feedback would be the key driver in their usage of educational technology. The results indeed suggest that students find on-line feedback more enjoyable and useful than conventional feedback, but enjoyment and attitude are the two most important factors.

[1]  G. Martínez-Sierra,et al.  Social Representations of High School Students About Mathematics Assessment , 2016 .

[2]  Douglas R. Vogel,et al.  Predicting user acceptance of collaborative technologies: An extension of the technology acceptance model for e-learning , 2013, Comput. Educ..

[3]  R. Bagozzi,et al.  On the evaluation of structural equation models , 1988 .

[4]  Rune Johan Krumsvik,et al.  Creating formative feedback spaces in large lectures , 2015, Comput. Educ..

[5]  G. Gibbs,et al.  Conditions Under Which Assessment Supports Students’ Learning , 2005 .

[6]  Brian Doig,et al.  Mathematical power of special-needs pupils: An ICT-based dynamic assessment format to reveal weak pupils' learning potential , 2009, Br. J. Educ. Technol..

[7]  Bernard P. Veldkamp,et al.  Effects of feedback in a computer-based assessment for learning , 2012, Comput. Educ..

[8]  Validación y confiabilidad de una escala de Actitudes hacia las Matemáticas y hacia las Matemáticas Enseñadas con Computadora , 2004 .

[9]  Jerrell C. Cassady,et al.  The Effects of Online Formative and Summative Assessment on Test Anxiety and Performance , 2005 .

[10]  Viswanath Venkatesh,et al.  Determinants of Perceived Ease of Use: Integrating Control, Intrinsic Motivation, and Emotion into the Technology Acceptance Model , 2000, Inf. Syst. Res..

[11]  Fred D. Davis,et al.  Extrinsic and Intrinsic Motivation to Use Computers in the Workplace1 , 1992 .

[12]  Shi-Jinn Horng,et al.  Applying an online game-based formative assessment in a flowchart-based intelligent tutoring system for improving problem-solving skills , 2016, Comput. Educ..

[13]  Timothy Teo,et al.  Modelling technology acceptance in education: A study of pre-service teachers , 2009, Comput. Educ..

[14]  M. Miller,et al.  Sample Size Requirements for Structural Equation Models , 2013, Educational and psychological measurement.

[15]  R. Sutherland,et al.  Perspectives on School Algebra , 2002 .

[16]  A. Bandura.  Self-Efficacy: The Exercise of Control , 1997, Journal of Cognitive Psychotherapy.

[17]  Susan Powers,et al.  Effects of delayed and immediate feedback in the computer-based testing environment , 2009 .

[18]  Rolph E. Anderson,et al.  Multivariate Data Analysis (7th ed. , 2009 .

[19]  Denise Whitelock,et al.  Developing a roadmap for e-assessment: which way now? , 2006 .

[20]  Denise Whitelock.  Editorial: e-assessment: developing new dialogues for the digital age , 2009, Br. J. Educ. Technol..

[21]  D. Prozesky,et al.  Assessment of Learning , 2020, The SAGE Encyclopedia of Higher Education.

[22]  David W. Brooks,et al.  An Analysis of Student Learning at a Testing Web Site Emphasizing Descriptive Chemistry , 2002 .

[23]  Torsten Reiners,et al.  Investigating Factors Affecting the Uptake of Automated Assessment Technology , 2011, J. Inf. Technol. Educ. Res..

[24]  Anastasios A. Economides,et al.  The effect of emotional feedback on behavioral intention to use computer based assessment , 2012, Comput. Educ..

[25]  Larry D. Rosen,et al.  Computer anxiety: A cross-cultural comparison of university students in ten countries , 1995 .

[26]  I. Ajzen,et al.  Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research , 1977 .

[27]  Wynne W. Chin,et al.  Handbook of Partial Least Squares , 2010 .

[28]  D. Nicol,et al.  Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice , 2006 .

[29]  Judith Watson,et al.  Does regular online testing enhance student learning in the numerical sciences? Robust evidence from a large data set , 2009, Br. J. Educ. Technol..

[30]  S. Bloxham,et al.  Developing Effective Assessment in Higher Education: A Practical Guide , 2007 .

[31]  A. Simpson,et al.  Students' perceptions of assessment in undergraduate mathematics , 2013 .

[32]  Hsiu-Li Liao,et al.  Impact of media richness and flow on e-learning technology acceptance , 2009, Comput. Educ..

[33]  Sally Jordan,et al.  E-assessment for learning? Exploring the potential of computer-marked assessment and computer-generated feedback, from short-answer questions to assessment analytics. , 2014 .

[34]  P. Galbraith,et al.  Disentangling the Nexus: Attitudes to Mathematics and Technology in a Computer Learning Environment , 1998 .

[35]  C. Kramer,et al.  A learning environment. , 1977, Journal - American Health Care Association.

[36]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[37]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[38]  Richard,et al.  Extrinsic and Intrinsic Motivation to Use Computers in the Workplace , 2022 .

[39]  Giuliana Dettori,et al.  From Arithmetic to Algebraic Thinking by Using a Spreadsheet , 2001 .

[40]  Semiral Öncü,et al.  E-Assessment Adaptation at a Military Vocational College: Student Perceptions. , 2015 .

[41]  Tess Miller.  Formative computer‐based assessment in higher education: the effectiveness of feedback in supporting student learning , 2009 .

[42]  Karl M. Kapp,et al.  A Gamified Approach on Learning Logic Gates to Improve Student’s Engagement , 2012, IOP Conference Series: Materials Science and Engineering.

[43]  Sandra T. Miller.  Formative Computer-based Assessments: The potentials and pitfalls of two formative computer-based assessments used in professional learning programs , 2009 .

[44]  Ian Jones,et al.  A Semi-Automatic Computer-Aided Assessment Approach for Marking and Providing Feedback Comments , 2015, CSEDU.

[45]  Peter Williams,et al.  The Learning Web , 2002 .

[46]  Fred D. Davis.  User Acceptance of Information Technology: System Characteristics, User Perceptions and Behavioral Impacts , 1993, Int. J. Man Mach. Stud..

[47]  Young-Gul Kim,et al.  Extending the TAM for a World-Wide-Web context , 2000, Inf. Manag..

[48]  Chorng-Shyong Ong,et al.  Factors affecting engineers' acceptance of asynchronous e-learning systems in high-tech companies , 2004, Inf. Manag..

[49]  A. Tenbrunsel,et al.  Organizational Behavior and Human Decision Processes , 2013 .

[50]  Marion Birch,et al.  Using online assessment to provide instant feedback , 2012 .

[51]  A KirschnerPaul,et al.  Effects of attitudes and behaviours on learning mathematics with computer tools , 2010 .

[52]  I. Ajzen.  The theory of planned behavior , 1991 .

[53]  Timothy Teo,et al.  Examining the intention to use technology among pre-service teachers: an integration of the Technology Acceptance Model and Theory of Planned Behavior , 2012, Interact. Learn. Environ..

[54]  Jane M. Howell,et al.  Personal Computing: Toward a Conceptual Model of Utilization , 1991, MIS Q..

[55]  Liwen Chen,et al.  Improving student performance in a first-year geography course: Examining the importance of computer-assisted formative assessment , 2011, Comput. Educ..

[56]  Christopher J. Sangwin,et al.  Automation of mathematics examinations , 2016, Comput. Educ..

[57]  Eva Heinrich,et al.  An Investigation into E-Tool Use for Formative Assignment Assessment - Status and Recommendations , 2009, J. Educ. Technol. Soc..

[58]  Chris Sangwin.  Computer Aided Assessment of Mathematics , 2013 .

[59]  A. Bandura.  Social Foundations of Thought and Action: A Social Cognitive Theory , 1985 .

[60]  Yigal Attali,et al.  Gamification in assessment: Do points affect test performance? , 2015, Comput. Educ..

[61]  Paula Hodgson,et al.  Effective formative e-assessment of student learning: a study on a statistics course , 2012 .

[62]  Kerstin Liehr-Gobbers,et al.  Evaluation of Structural Equation Models Using the Partial Least Squares (PLS) Approach , 2010 .

[63]  Joanna Bull,et al.  Assessing student learning in higher education , 1997 .

[64]  Caroline V. Gipps,et al.  What is the role for ICT-based assessment in universities? , 2005 .

[65]  Anastasios A. Economides,et al.  The acceptance and use of computer based assessment , 2011, Comput. Educ..

[66]  D. Kaplan.  Structural Equation Modeling: Foundations and Extensions , 2000 .

[67]  Anastasios A. Economides,et al.  The impact of paper-based, computer-based and mobile-based self-assessment on students' science motivation and achievement , 2016, Comput. Hum. Behav..

[68]  John W. Rice,et al.  The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education , 2012, Int. J. Gaming Comput. Mediat. Simulations.

[69]  Denise Whitelock,et al.  Taking an electronic mathematics examination from home: what the students think , 2003 .

[70]  Paul A. Kirschner,et al.  Effects of attitudes and behaviours on learning mathematics with computer tools , 2010, Comput. Educ..