Investigation of The Web-based Self-Assessment System Based on Assessment Analytics in Terms of Perceived Self-intervention

The self-assessment system was developed based on the assessment as learning (AaL) approach, in which learners take responsibility for their own learning and use feedback for self-intervention. The feedback as the system’s principal component was produced using assessment data in the system. It is essential for the assessment as learning that students construct their interventions to their learning processes by using assessment feedback. From this point, this study aimed to evaluate the web-based self-assessment system through the lens of student perceived self-intervention. The one-shot case study design (one-group post-test) model, one of weak experimental design, was used as a research design. The web-based self-assessment system was employed over a 10-week period to support face-to-face lessons. At the beginning of the implementation process, students taking the “Measurement and Evaluation in Education” lesson in the education faculty were informed about the system. After the implementation process, the perceived self-intervention scale was used to determine the students' perception. A total of 214 students (71%) took at least one test in the system and gained access to the feedback module. The high level of participation in a non-mandatory system may be considered a success. Learners appeared to have utilized feedback gained from the system in planning self-intervention and monitoring and evaluating the planned intervention's effectiveness.

[1]  Eleanor T. Loiacono-Mello,et al.  Gender Feedback and Decision Making: How Men and Women Differ on the Use of Computerized Feedback? , 2006, AMCIS.

[2]  Anastasios A. Economides,et al.  An Assessment Analytics Framework (AAF) for Enhancing Students’ Progress , 2016 .

[3]  Stanley L. Deno Curriculum-based measurement and special education services: A fundamental and direct relationship. , 1989 .

[4]  Erik Duval,et al.  Learning Analytics Dashboard Applications , 2013 .

[5]  Lynne D. Roberts,et al.  Give Me a Customizable Dashboard: Personalized Learning Analytics Dashboards in Higher Education , 2017, Technology, Knowledge and Learning.

[6]  Doug Clow,et al.  The learning analytics cycle: closing the loop effectively , 2012, LAK.

[7]  Tracii Ryan,et al.  Conditions that enable effective feedback , 2019, Higher Education Research & Development.

[8]  N. Winstone,et al.  ‘Feedback interpreters’: the role of learning development professionals in facilitating university students’ engagement with feedback , 2018, Teaching in Higher Education.

[9]  Gavin T. L. Brown Student self-assessment , 2013 .

[10]  I. Clark Formative Assessment: Assessment Is for Self-regulated Learning , 2012, Educational Psychology Review.

[11]  P. Black Formative and Summative Assessment by Teachers , 1993 .

[12]  Prerna Lal,et al.  Designing Online Learning Strategies through Analytics , 2014 .

[13]  Muhittin Şahin,et al.  Learners’ Needs in Online Learning Environments and Third Generation Learning Management Systems (LMS 3.0) , 2020, Technology, Knowledge and Learning.

[14]  J. Hattie,et al.  The Power of Feedback , 2007 .

[15]  John Hattie,et al.  INSTRUCTION BASED ON FEEDBACK , 2010 .

[16]  Doug Clow An overview of learning analytics , 2013 .

[17]  David Nicol,et al.  The power of internal feedback: exploiting natural comparison processes , 2020 .

[18]  D. Nicol,et al.  Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice , 2006 .

[19]  S. Deno,et al.  Curriculum-Based Measurement: The Emerging Alternative , 1985, Exceptional children.

[20]  Anastasios A. Economides,et al.  Exhibiting achievement behavior during computer-based testing: What temporal trace data and personality traits tell us? , 2017, Comput. Hum. Behav..

[21]  Cath Ellis,et al.  The Importance of E-Portfolios for Effective Student-Facing Learning Analytics , 2017 .

[22]  Dirk Ifenthaler,et al.  Development and Validation of a Learning Analytics Framework: Two Case Studies Using Support Vector Machines , 2014, Technology, Knowledge and Learning.

[23]  Linda Corrin,et al.  How do students interpret feedback delivered via dashboards? , 2015, LAK.

[24]  Robert A. Nash,et al.  Responsibility-Sharing in the Giving and Receiving of Assessment Feedback , 2017, Front. Psychol..

[25]  Ross Miller,et al.  Facilitating Student Success in Introductory Chemistry with Feedback in an Online Platform , 2018, Technol. Knowl. Learn..

[26]  Wenjie Zeng,et al.  Towards a learning-oriented assessment to improve students’ learning—a critical review of literature , 2018, Educational Assessment, Evaluation and Accountability.

[27]  Anastasios A. Economides,et al.  The acceptance and use of computer based assessment , 2011, Comput. Educ..

[28]  Jonathan D Kibble Voluntary participation in online formative quizzes is a sensitive predictor of student success. , 2011, Advances in physiology education.

[29]  Sabine Seufert,et al.  A Pedagogical Perspective on Big Data and Learning Analytics: A Conceptual Model for Digital Learning Support , 2019, Technology, Knowledge and Learning.

[30]  Nian-Shing Chen,et al.  Use of Dashboards and Visualization Techniques to Support Teacher Decision Making , 2015 .

[31]  J. Kibble,et al.  Use of unsupervised online quizzes as formative assessment in a medical physiology course: effects of incentives on student participation and performance. , 2007, Advances in physiology education.

[32]  David Boud,et al.  Enhancing learning through self assessment , 1995 .

[33]  Gregory Schraw Promoting general metacognitive awareness , 1998 .

[34]  Larry Hatcher,et al.  A step-by-step approach to using SAS for factor analysis and structural equation modeling , 2014 .

[35]  Stephanie D. Teasley Student Facing Dashboards: One Size Fits All? , 2017, Technol. Knowl. Learn..

[36]  Albert H. Segars,et al.  Assessing the unidimensionality of measurement : a paradigm and illustration within the context of information systems research , 1997 .

[37]  D. Boud,et al.  Student Self-Assessment in Higher Education: A Meta-Analysis , 1989 .

[38]  R. W. Kulhavy,et al.  Feedback in written instruction: The place of response certitude , 1989 .

[39]  Hale Ilgaz,et al.  Providing online exams for online learners: Does it really matter for them? , 2019, Education and Information Technologies.

[40]  L. Earl,et al.  Getting to the Core of Learning: Using Assessment for Self-Monitoring and Self-Regulation , 2012 .

[41]  Stephanie Boehm,et al.  Information Dashboard Design The Effective Visual Communication Of Data , 2016 .

[42]  Bob Rudis,et al.  Data-Driven Security: Analysis, Visualization and Dashboards , 2014 .

[43]  Naomi Winstone,et al.  ‘It'd be useful, but I wouldn't use it’: barriers to university students’ feedback seeking and recipience , 2017 .

[44]  Oylum Korkut Altuna,et al.  Impact of the Number of Scale Points on Data Characteristics and Respondents’ Evaluations: An Experimental Design Approach Using 5-Point and 7-Point Likert-type Scales , 2016 .

[45]  Dirk T. Tempelaar,et al.  In search for the most informative data for feedback generation: Learning analytics in a data-rich context , 2015, Comput. Hum. Behav..

[46]  Ulrik Schroeder,et al.  A reference model for learning analytics , 2012 .

[47]  G. Crisp Integrative assessment: reframing assessment practice for current and future learning , 2012 .

[48]  D. Sadler,et al.  Formative Assessment: revisiting the territory , 1998 .

[49]  Cath Ellis,et al.  Broadening the scope and increasing the usefulness of learning analytics: The case for assessment analytics , 2013, Br. J. Educ. Technol..

[50]  Francisco Javier Palacios Hidalgo,et al.  MOOCs: Origins, Concept and Didactic Applications: A Systematic Review of the Literature (2012–2019) , 2020, Technol. Knowl. Learn..

[51]  M. Taras,et al.  ASSESSMENT – SUMMATIVE AND FORMATIVE – SOME THEORETICAL REFLECTIONS , 2005 .

[52]  Oana Velcu,et al.  A review of dashboards in performance management: Implications for design and research , 2012, Int. J. Account. Inf. Syst..

[53]  D. Sluijsmans,et al.  The use of self-, peer and co-assessment in higher education: A review , 1999 .

[54]  Ruth Dann,et al.  Assessment as learning: blurring the boundaries of assessment and learning for theory, policy and practice , 2014 .

[55]  Xiangdong Li Self-assessment as ‘assessment as learning’ in translator and interpreter education: validity and washback , 2018, New Perspectives on Assessment in Translator Education.

[56]  R. Glaser,et al.  Knowing What Students Know: The Science and Design of Educational Assessment , 2001 .

[57]  D. Boud Sustainable Assessment: Rethinking assessment for the learning society , 2000 .

[58]  Lilia Cheniti-Belcadhi,et al.  An Enhanced xAPI Data Model Supporting Assessment Analytics , 2018, KES.

[59]  J. W. Gikandi,et al.  Online formative assessment in higher education: A review of the literature , 2011, Comput. Educ..

[60]  Halil Yurdugül,et al.  Web-tabanlı Öz-Değerlendirme Sisteminde Öğrenenlerin Öz-Müdahale Algısı ve Test Alma Davranışlarının Başarı Üzerine Etkisi , 2016 .

[61]  E. Mory Feedback research revisited. , 2004 .

[62]  J. Flavell Metacognition and Cognitive Monitoring: A New Area of Cognitive-Developmental Inquiry. , 1979 .

[63]  C. Evans Making Sense of Assessment Feedback in Higher Education , 2013 .

[64]  V. Shute Focus on Formative Feedback , 2008 .

[65]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[66]  Tami Seifert,et al.  Online self-assessment and peer-assessment as a tool to enhance student-teachers’ assessment skills , 2018, Assessment & Evaluation in Higher Education.

[67]  Susanne Narciss,et al.  Conditions and Effects of Feedback Viewed Through the Lens of the Interactive Tutoring Feedback Model , 2017 .

[68]  A. Colman,et al.  Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. , 2000, Acta psychologica.

[69]  D. Nicol,et al.  Rethinking technology-supported assessment practices in relation to the seven principles of good feedback practice. , 2006 .

[70]  Patrick Jermann,et al.  Designing for impact: a conceptual framework for learning analytics as self-assessment tools , 2020 .

[71]  Mary E. Webb,et al.  Assessment as, for, and of Twenty-First Century Learning Using Information Technology: An Overview , 2018 .

[72]  L. Earl Assessment as Learning , 2007 .

[73]  Robert A. Nash,et al.  Individual differences in self-reported use of assessment feedback: the mediating role of feedback beliefs , 2019, Educational Psychology.

[74]  Katrien Verbert,et al.  Review of Research on Student-Facing Learning Analytics Dashboards and Educational Recommender Systems , 2017, IEEE Transactions on Learning Technologies.

[75]  Lilia Cheniti-Belcadhi,et al.  An Ontological Model for Assessment Analytics , 2017, WEBIST.

[76]  María Soledad Ibarra-Sáiz,et al.  Assessment as Learning and Empowerment: Towards Sustainable Learning in Higher Education , 2015 .

[77]  Gregory Schraw Promoting general metacognitive awareness , 1998 .

[78]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[79]  Heidi Andrade,et al.  Promoting Learning and Achievement Through Self-Assessment , 2009 .

[80]  Mary Webb,et al.  Challenges for IT-Enabled Formative Assessment of Complex 21st Century Skills , 2018, Technol. Knowl. Learn..

[81]  D. Boud,et al.  Rethinking models of feedback for learning: the challenge of design , 2013 .

[82]  Yeonjeong Park,et al.  Development of the Learning Analytics Dashboard to Support Students' Learning Performance , 2015, J. Univers. Comput. Sci..

[83]  Uwe Maier,et al.  Effects of a computer-assisted formative assessment intervention based on multiple-tier diagnostic items and different feedback types , 2016, Comput. Educ..

[84]  George Siemens,et al.  Content analytics: the definition, scope, and an overview of published research , 2017 .

[85]  David Boud,et al.  The development of student feedback literacy: enabling uptake of feedback , 2018 .

[86]  Abelardo Pardo,et al.  A feedback model for data-rich learning experiences , 2018 .

[87]  L. Ahmed,et al.  Identifying predictors of students’ perception of and engagement with assessment feedback , 2018 .

[88]  Philip J. Dutton,et al.  Investigation of Formative Assessment of Learning (INFORMAL): The Performance Indicator Tool (PIT) , 2017, Technol. Knowl. Learn..

[89]  Susan M. Brookhart,et al.  How to give effective feedback to your students , 2008 .

[90]  Danakorn Nincarean Eh Phon,et al.  Blended Learning Adoption and Implementation in Higher Education: A Theoretical and Systematic Review , 2020, Technology, Knowledge and Learning.

[91]  O. Ventista Self-assessment in Massive Open Online Courses , 2018, E-Learning and Digital Media.