Building a Formative Assessment System That is Easy to Adopt Yet Supports Long-term Improvement: A Review of the Literature and Design Recommendations

Formative assessment has been shown to be an effective teaching tool, yet is infrequently used in practice. With the intent of building a formative e-assessment platform, we examine research on formative practices and supporting computer-based systems with a focus on: institutional barriers to adoption of previous systems; senses in which students and teachers can improve their practices across varying timescales; and collectible data (self-reported or otherwise) necessary or advantageous in supporting these processes. From this research we identify the minimal set of data which adequately supports these processes of improvement, arrive at a set of requirements and recommendations for an innovative system which collects, processes, and presents this data appropriately, and from these requirements design the architecture of an extensible electronic formative assessment system which balances the need for complex long-term analytics with that of accessibility.

[1]  Mark Wilson,et al.  Measuring Progressions: Assessment Structures Underlying a Learning Progression , 2009 .

[2]  R. Bennett.  Formative assessment: a critical review , 2011 .

[3]  J. W. Gikandi,et al.  Online formative assessment in higher education: A review of the literature , 2011, Comput. Educ..

[4]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[5]  Xiaoling Zou,et al.  Effect of different score reports of Web-based formative test on students' self-regulated learning , 2013, Comput. Educ..

[6]  Annelies Raes,et al.  What if pupils can assess their peers anonymously? A quasi-experimental study , 2015, Comput. Educ..

[7]  Dylan Wiliam,et al.  Integrating Assessment with Learning: What Will It Take to Make It Work? , 2007 .

[8]  D. Nicol,et al.  Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice , 2006 .

[9]  Richard Hall,et al.  Scoping the pedagogic relationship between self‐efficacy and Web 2.0 technologies , 2010 .

[10]  H. F. Spitzer.  Studies in retention. , 1939 .

[11]  Shane Dawson,et al.  Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan , 2012, J. Educ. Technol. Soc..

[12]  Chih-Ping Chu,et al.  Effects of concept map extraction and a test-based diagnostic environment on learning achievement and learners' perceptions , 2016, Br. J. Educ. Technol..

[13]  J. Kotter,et al.  The Heart of Change: Real-Life Stories of How People Change Their Organizations , 2002 .

[14]  P. Black,et al.  Working inside the Black Box: Assessment for Learning in the Classroom , 2004 .

[15]  Giovannina Albano,et al.  Formative Self-assessment to Support Self-driven Mathematics Education at University Level , 2014, ICWL Workshops.

[16]  Denise Whitelock,et al.  OpenEssayist: extractive summarisation and formative assessment of free−text essays. , 2013 .

[17]  Janet W. Looney,et al.  Integrating Formative and Summative Assessment: Progress Toward a Seamless System? , 2011 .

[18]  Russell Boyatt,et al.  Exploring factors that influence adoption of e-learning within higher education , 2015, Br. J. Educ. Technol..

[19]  P. Black,et al.  Inside the Black Box: Raising Standards through Classroom Assessment , 2010 .

[20]  D. Nicol.  E‐assessment by design: using multiple‐choice tests to good effect , 2007 .

[21]  Carol Russell,et al.  A systemic framework for managing e-learning adoption in campus universities: individual strategies in context , 2009 .

[22]  E. Rogers.  Diffusion of Innovations: Modifications of a Model for Telecommunications , 1995 .

[23]  P. Black,et al.  Developing the theory of formative assessment , 2009 .

[24]  Edward F. Gehringer.  A Survey of Methods for Improving Review Quality , 2014, ICWL Workshops.

[25]  Benita Cox,et al.  Feedback in technology-based instruction: Learner preferences , 2016, Br. J. Educ. Technol..

[26]  Amparo Alonso-Betanzos,et al.  Including Content-Based Methods in Peer-Assessment of Open-Response Questions , 2015, 2015 IEEE International Conference on Data Mining Workshop (ICDMW).

[27]  Paulo Oliveira,et al.  A system for formative assessment and monitoring of students' progress , 2014, Comput. Educ..

[28]  María del Puerto Paule Ruíz,et al.  Students' LMS interaction patterns and their relationship with achievement: A case study in higher education , 2016, Comput. Educ..

[29]  Robert A. Bjork,et al.  Optimum rehearsal patterns and name learning , 1978 .

[30]  D. Hestenes,et al.  Force concept inventory , 1992 .

[31]  Samuel Greiff,et al.  Computer-generated log-file analyses as a window into students' minds? A showcase study based on the PISA 2012 assessment of problem solving , 2015, Comput. Educ..

[32]  Françoise Blin,et al.  Why hasn't technology disrupted academics' teaching practices Understanding resistance to change through the lens of activity theory , 2008, Comput. Educ..

[33]  Dylan Wiliam,et al.  Formative Assessment: Getting the Focus Right , 2006 .

[34]  Lauri Malmi,et al.  Observations on student misconceptions—A case study of the Build – Heap Algorithm , 2006, Comput. Sci. Educ..

[35]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[36]  Bernard Scott,et al.  Competencies for learning design: A review of the literature and a proposed framework , 2011, Br. J. Educ. Technol..