CASA: An Architecture to Support Complex Assessment Scenarios

The evaluation or assessment of student performance and knowledge is a central element of most instructional design models, as it provides the information required to take remediation actions and improve the learning process. However, the assessment contexts present such a diverse range of cases due to course, teacher and student idiosyncrasies that it is difficult to support all the possibilities via software. To deal with this problem, this paper presents an architecture that satisfies the requirements to support complex assessment scenarios, those that potentially involve different evaluators using diverse assessment methods, tools and techniques at the same time. An analysis of the literature and the assessment scenarios of courses from Computer Engineering grades was carried out to identify the requirements for systems that support such complex assessment scenarios and to infer the information needed to support them. This information was modeled and formalized in the Complex Assessment Scenarios Architecture (CASA) that comprises functional modules, knowledge bases and the relationships among them. Following this architecture, an assessment system (AdESMuS) was developed using agile methodologies. AdESMuS has been satisfactorily tested under different complex assessment scenarios proving to be able to support those scenarios’ requirements. This paper presents CASA, the architecture for developing a system to support complex assessment scenarios along with the satisfactory case study of AdESMuS.

[1]  Dannelle D. Stevens,et al.  Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning , 2004 .

[2]  Natasha A. Jankowski,et al.  Equity and Assessment: Moving towards Culturally Responsive Assessment. Occasional Paper #29. , 2017 .

[3]  Ainhoa Alvarez,et al.  Supporting complex assessment processes , 2017, TEEM.

[4]  M. Taras,et al.  ASSESSMENT – SUMMATIVE AND FORMATIVE – SOME THEORETICAL REFLECTIONS , 2005 .

[5]  Anders Jonsson,et al.  The Use of Scoring Rubrics for Formative Assessment Purposes Revisited: A Review. , 2013 .

[6]  Viktor Kuncak,et al.  Software verification and graph similarity for automated evaluation of students' assignments , 2012, Inf. Softw. Technol..

[7]  Susan M. Brookhart,et al.  Appropriate Criteria: Key to Effective Rubrics , 2018, Front. Educ..

[8]  Pau Fonseca i Casas,et al.  Evaluation and assessment of professional skills in the Final Year Project , 2014, 2014 IEEE Frontiers in Education Conference (FIE) Proceedings.

[9]  E. Valderrama,et al.  Guidelines for the final year project assessment in engineering , 2009, 2009 39th IEEE Frontiers in Education Conference.

[10]  Mikel Larrañaga Olagaray,et al.  Desarrollo y Validación de un Conjunto de Rúbricas para la Evaluación de Trabajos Fin de Grado , 2017 .

[11]  Maha A. I. Dandis,et al.  Benefits and criticism. Should we support or neglect using rubrics? Evidences from a literature review , 2014 .

[12]  Darryl M. Hunter.  The Use of Holistic versus Analytic Scoring for Large-Scale Assessment of Writing , 1996, Canadian Journal of Program Evaluation.

[13]  Leston D. Drake,et al.  Reclaiming Instructional Design. , 1996 .

[14]  Norman Vaughan,et al.  Student Engagement and Blended Learning: Making the Assessment Connection , 2014 .

[15]  Elena Montañés,et al.  Obtaining Rubric Weights for Assessments by More than One Lecturer Using a Pairwise Learning Model. , 2009, EDM 2009.

[16]  Thomas Lagkas,et al.  How to improve the peer review method: Free-selection vs assigned-pair protocol evaluated in a computer networking course , 2012, Comput. Educ..

[17]  Ana Arruarte Lasa,et al.  Dealing with common problems in engineering degrees' Final Year Projects , 2014, 2014 IEEE Frontiers in Education Conference (FIE) Proceedings.

[18]  Barbara M. Moskal,et al.  Scoring Rubrics: What, When and How? , 2000 .

[19]  Barbara Wasson,et al.  Integrating multiple data sources for learning analytics—review of literature , 2019, Research and Practice in Technology Enhanced Learning.

[20]  Roy Fielding,et al.  Architectural Styles and the Design of Network-based Software Architectures"; Doctoral dissertation , 2000 .

[21]  Candice A. Rideout.  Students’ choices and achievement in large undergraduate classes using a novel flexible assessment approach , 2018 .