Cognitive Feedback and Behavioral Feedforward Automation Perspectives for Modeling and Validation in a Learning Context

State-of-the-art technologies have made it possible to provide a learner with immediate computer-assisted feedback by delivering a feedback targeting cognitive aspects of learning, (e.g. reflecting on a result, explaining a concept, i.e. improving understanding). Fast advancement of technology has recently generated increased interest for previously non-feasible approaches for providing feedback based on learning behavioral observations by exploiting different traces of learning processes stored in information systems. Such learner behavior data makes it possible to observe different aspects of learning processes in which feedback needs of learners (e.g. difficulties, engagement issues, inefficient learning processes, etc.) based on individual learning trajectories can be traced. By identifying problems earlier in a learning process it is possible to deliver individualized feedback helping learners to take control of their own learning, i.e. to become self-regulated learners, and teachers to understand individual feedback needs and/or adapt their teaching strategies. In this work we (i) propose cognitive computer-assisted feedback mechanisms using a combination of MDE based simulation augmented with automated feedback, and (ii) discuss perspectives for behavioral feedback, i.e. feedforward, that can be based on learning process analytics in the context of learning conceptual modeling. Aggregated results of our previous studies assessing the effectiveness of the proposed cognitive feedback method with respect to improved understanding on different dimensions of knowledge, as well as feasibility of behavioral feedforward automation based on learners behavior patterns, are presented. Despite our focus on conceptual modeling and specific diagrams, the principles of the approach presented in this work can be used to support educational feedback automation for a broader spectrum of diagram types beyond the scope of conceptual modeling.

[1]  Gayane Sedrakyan.  Process-oriented feedback perspectives based on feedback-enabled simulation and learning process data analytics. , 2016 .

[2]  Viswanath Venkatesh,et al.  User acceptance of information technology : a unified view , 1998 .

[3]  Keng Siau,et al.  Can UML Be Simplified? Practitioner Use of UML in Separate Domains , 2007, EMMSAD.

[4]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[5]  Monique Snoeck,et al.  A PIM-to-Code Requirements Engineering Framework , 2013, MODELSWARD.

[6]  Monique Snoeck,et al.  Technology-Enhanced Support for Learning Conceptual Modeling , 2012, BMMDS/EMMSAD.

[7]  V. Shute.  Focus on Formative Feedback , 2008 .

[8]  Monique Snoeck,et al.  Computer Aided Modelling Exercises , 2007, Informatics Educ..

[9]  Carlos Delgado Kloos,et al.  Evaluating usability of affective state visualizations using AffectVis, an affect-aware dashboard for students , 2017 .

[10]  Rod Ellis,et al.  A typology of written corrective feedback types , 2008 .

[11]  Jan T. van der Veen,et al.  The learning effects of computer simulations in science education , 2012, Comput. Educ..

[12]  Monique Snoeck,et al.  Do we need to teach testing skills in courses on requirements engineering and modelling , 2014 .

[13]  D. Nicol,et al.  Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice , 2006 .

[14]  Martin Bichler,et al.  Design science in information systems research , 2006, Wirtschaftsinf..

[15]  Tammy Schellens,et al.  Students' perceptions about the use of video games in the classroom , 2010, Comput. Educ..

[16]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[17]  R. Ellis.  Corrective Feedback and Teacher Development , 2009 .

[18]  Remigijus Gustas,et al.  Conceptual Modeling and Integration of Static and Dynamic Aspects of Service Architectures , 2010, ONTOSE.

[19]  Barbara H Wixom,et al.  A Theoretical Integration of User Satisfaction and Technology Acceptance , 2005, Inf. Syst. Res..

[20]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[21]  Monique Snoeck,et al.  Tutorial: Boosting requirements analysis and validation skills through feedback-enabled semantic prototyping , 2015 .

[22]  Monique Snoeck,et al.  Lightweight Semantic Prototyper for Conceptual Modeling , 2014, ER Workshops.

[23]  Sean Brophy,et al.  Teaching an aerospace engineering design course via virtual worlds: A comparative assessment of learning outcomes , 2013, Comput. Educ..

[24]  Monique Snoeck,et al.  Enterprise Information Systems Engineering : The MERODE Approach , 2014 .

[25]  John Krogstie,et al.  Validating Conceptual Models by Transformational Prototyping , 1993, CAiSE.

[26]  Monique Snoeck,et al.  Process-mining enabled feedback: "Tell me what I did wrong" vs. "tell me how to do it right" , 2016, Comput. Hum. Behav..

[27]  Nicola Careddu,et al.  Mine simulation for educational purposes: A case study , 2015, Comput. Appl. Eng. Educ..

[28]  Paul A. Kirschner,et al.  Ten Steps to Complex Learning: A Systematic Approach to Four-Component Instructional Design , 2007 .

[29]  Björn Lundell,et al.  UML model interchange in heterogeneous tool environments: an analysis of adoptions of XMI 2 , 2006, MoDELS'06.

[30]  A. Irons,et al.  Enhancing Learning through Formative Assessment and Feedback , 2007 .

[31]  Sanna Järvelä,et al.  Conceptual framework for feedback automation and personalization for designing learning analytics dashboards , 2016 .

[32]  S. Narciss.  Feedback Strategies for Interactive Learning Tasks , 2007 .

[33]  Ivan Porres,et al.  Model interchange using OMG standards , 2005, 31st EUROMICRO Conference on Software Engineering and Advanced Applications.

[34]  D C Prather,et al.  Trial-and-error versus errorless learning: training, transfer, and stress. , 1971, The American journal of psychology.

[35]  Wang Wang,et al.  Empirical investigations of conceptual modeling and the modeling process , 2007, 2007 Winter Simulation Conference.

[36]  Shihong Huang,et al.  Towards interoperability of UML tools for exchanging high-fidelity diagrams , 2007, SIGDOC '07.

[37]  Mario Piattini,et al.  A conceptual modeling quality framework , 2011, Software Quality Journal.

[38]  Monique Snoeck,et al.  Enriching Model Execution with Feedback to Support Testing of Semantic Conformance between Models and Requirements - Design and Evaluation of Feedback Automation Architecture , 2016, AMARETTO@MODELSWARD.

[39]  Chin-Lung Hsu,et al.  Consumer behavior in online game communities: A motivational factor perspective , 2007, Comput. Hum. Behav..

[40]  Stijn Hoppenbrouwers,et al.  Cognitive Mechanisms of Conceptual Modelling - How Do People Do It? , 2013, ER.

[41]  Ashim K. Datta,et al.  Simulation as an integrator in an undergraduate biological engineering curriculum , 2013, Comput. Appl. Eng. Educ..

[42]  Arne Sølvberg,et al.  Understanding quality in conceptual modeling , 1994, IEEE Software.

[43]  Monique Snoeck,et al.  Assessing the effectiveness of feedback enabled simulation in teaching conceptual modeling , 2014, Comput. Educ..

[44]  B. Zimmerman.  Investigating Self-Regulation and Motivation: Historical Background, Methodological Developments, and Future Prospects , 2008, American Educational Research Journal.

[45]  F. Lateef.  Simulation-based learning: Just like the real thing , 2010, Journal of emergencies, trauma, and shock.

[46]  Monique Snoeck,et al.  Feedback-Enabled MDA-Prototyping Effects on Modeling Knowledge , 2013, BMMDS/EMMSAD.

[47]  David M Gaba,et al.  The future vision of simulation in health care , 2004, Quality and Safety in Health Care.

[48]  Timothy A. Hess.  Investigation of Prototype Roles in Conceptual Design using Case Study and Protocol Study Methods , 2012 .

[49]  Monique Snoeck,et al.  Effects of Simulation on Novices' Understanding of the Concept of Inheritance in Conceptual Modeling , 2015, ER Workshops.

[50]  Monique Snoeck,et al.  Assessing the influence of feedback-inclusive rapid prototyping on understanding the semantics of parallel UML statecharts by novice modellers , 2017, Inf. Softw. Technol..

[51]  Monique Snoeck,et al.  Process mining analysis of conceptual modeling behavior of novices - empirical study using JMermaid modeling and experimental logging environment , 2014, Comput. Hum. Behav..

[52]  Ramesh Sharda,et al.  Innovative Teaching Using Simulation and Virtual Environments , 2012 .

[53]  Keng Siau,et al.  Identifying Difficulties in Learning Uml , 2006, Inf. Syst. Manag..

[54]  Monique Snoeck,et al.  Enterprise Information Systems Engineering , 2014, The Enterprise Engineering Series.

[55]  E. Tansley,et al.  Using ontology to validate conceptual models , 2003, CACM.

[56]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[57]  Monique Snoeck,et al.  Tutorial: Novel way of training conceptual modeling skills by means of feedback-enabled simulation , 2015 .

[58]  Carlos Delgado Kloos,et al.  Evaluating Student-Facing Learning Dashboards of Affective States , 2017, EC-TEL.

[59]  Nicholas P. Vitalari,et al.  Differences Between Novice and Expert Systems Analysts: What Do We Know and What Do We Do? , 1998, J. Manag. Inf. Syst..

[60]  Maria C. Yang,et al.  A study of prototypes, design activity, and design outcome , 2005 .

[61]  Dimitrios Stefanidis,et al.  Optimal acquisition and assessment of proficiency on simulators in surgery. , 2010, The Surgical clinics of North America.