Peer-assessment and grading of open answers in a web-based e-learning setting

Grading the answers given to open ended questions (and questionnaires) is a rather heavy task for teachers and lecturers. Several techniques have been used to automatically analyze the student's essays, but natural language processing is not yet ready to completely solve the task. We propose a solution, for the automated support to answers grading, where the students' peer assessment is checked against the teacher's marking, while such teacher's marking has to be performed on only a subset of the answers (so relieving the teacher by at least a part of the marking task). Our aim is twofold: 1) to make grading more effective by propagating the teacher's assessments through the network of peer assessments (so to maintain a progressively improved evaluations of the peer assessments precision and make them more likely to be used for final grading); 2) to suggest iteratively the teacher with the next most informative essays to grade (respect to the network, that is one of the answers whose teacher's mark will inject most information into the network and make the system better able to deduce the rest of the grades. To this aim we have used a very simple Bayesian model of peers and peer assessment, and we have built a web-based system to support the teacher. We show the present state of the model and its implementation. Moreover, we show, through a simulation protocol, how the system can support the teacher, obtaining a reasonably good correction with just a subset of the whole grades.

[1]  Satoshi Morinaga,et al.  Mining product reputations on the Web , 2002, KDD.

[2]  Carla Limongelli,et al.  An Ontology-Driven OLAP System to Help Teachers in the Analysis of Web Learning Object Repositories , 2010, Inf. Syst. Manag..

[3]  Marco Temperini,et al.  Learning from Peers: Motivating Students through Reputation Systems , 2008, 2008 International Symposium on Applications and the Internet.

[4]  Marco Temperini,et al.  Supporting Assessment of Open Answers in a Didactic Setting , 2012, 2012 IEEE 12th International Conference on Advanced Learning Technologies.

[5]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[6]  Marco Temperini,et al.  OpenAnswer, a framework to support teacher's management of open answers through peer assessment , 2013, 2013 IEEE Frontiers in Education Conference (FIE).

[7]  Marco Temperini,et al.  Correcting open-answer questionnaires through a Bayesian-network model of peer-based assessment , 2012, 2012 International Conference on Information Technology Based Higher Education and Training (ITHET).

[8]  Dennis Zielke,et al.  Design and Implementation of a Learning Analytics Toolkit for Teachers , 2012, J. Educ. Technol. Soc..

[9]  Carla Limongelli,et al.  Lecomps5: A Web-Based Learning System For Course Personalization And Adaptation , 2008, e-Learning.

[10]  Andrea Formisano,et al.  Instructing Equational Set-Reasoning with Otter , 2001, IJCAR.

[11]  Paul A. Kirschner,et al.  Using integrated electronic environments for collaborative teaching/learning , 2001 .

[12]  Paul A. Kirschner,et al.  Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: a review of the research , 2003, Comput. Hum. Behav..

[13]  Kevin Palmer,et al.  On-line assessment and free-response input – a pedagogic and technical model for squaring the circle , 2003 .

[14]  Marek J. Druzdzel,et al.  SMILE: Structural Modeling, Inference, and Learning Engine and GeNIE: A Development Environment for Graphical Decision-Theoretic Models , 1999, AAAI/IAAI.

[15]  B. Bloom.  Taxonomy of educational objectives , 1956 .

[16]  Rafael Valencia-García,et al.  Semantic Web Technologies for supporting learning assessment , 2011, Inf. Sci..

[17]  Heng-Yu Ku,et al.  An investigation of the effects of reciprocal peer tutoring , 2009, Comput. Hum. Behav..

[18]  Maria De Marsico,et al.  A Strategy to Join Adaptive and Reputation-Based Social-Collaborative E-Learning, Through the Zone of Proximal Development , 2013, Int. J. Distance Educ. Technol..

[19]  Elvira Popescu,et al.  Providing collaborative learning support with social media in an integrated environment , 2012, World Wide Web.

[20]  William M. K. Trochim,et al.  Concept Mapping as an Alternative Approach for the Analysis of Open-Ended Survey Responses , 2002 .

[21]  Temperini Marco,et al.  eLearning for Knowledge Management in Collaborative Architectural Design , 2011 .

[22]  Marco Temperini,et al.  Dealing with Open-Answer Questions in a Peer-Assessment Environment , 2012, ICWL.

[23]  Hang Li,et al.  Mining Open Answers in Questionnaire Data , 2001, IEEE Intell. Syst..

[24]  Marco Temperini,et al.  Analysis of open answers via mediated peer-assessment , 2013, 2013 17th International Conference on System Theory, Control and Computing (ICSTCC).

[25]  Ralf Klamma,et al.  u-Annotate: An Application for User-Driven Freeform Digital Ink Annotation of E-Learning Content , 2006 .

[26]  Marco Temperini,et al.  Selection and sequencing constraints for personalized courses , 2010, 2010 IEEE Frontiers in Education Conference (FIE).

[27]  Carla Limongelli,et al.  The Lecomps5 framework for personalized web-based learning: A teacher's satisfaction perspective , 2011, Comput. Hum. Behav..

[28]  Mohamed Jemni,et al.  A fully personalization strategy of E-learning scenarios , 2010, Comput. Hum. Behav..

[29]  Malinka Ivanova,et al.  Formal and Informal Learning Flows Cohesion in Web 2.0 Environment , 2011, Int. J. Inf. Syst. Soc. Chang..

[30]  Elvira Popescu,et al.  Adaptation provisioning with respect to learning styles in a Web-based educational system: an experimental study , 2010, J. Comput. Assist. Learn..

[31]  Maria De Marsico,et al.  A Framework to Support Social-Collaborative Personalized e-Learning , 2013, HCI.

[32]  Maria De Marsico,et al.  The Definition of a Tunneling Strategy between Adaptive Learning and Reputation-based Group Activities , 2011, 2011 IEEE 11th International Conference on Advanced Learning Technologies.

[33]  Naïma El-Kechaï,et al.  Evaluating the Performance of a Diagnosis System in School Algebra , 2011, ICWL.