Mediating skills on risk management for improving the resilience of Supply Networks by developing and using a serious game

conceptualization (thinking) In Beware this is supported in two waysduring game play, the student can draw his conclusion based on how his indicators (quality, cost and time) emerge. The players are encouraged to use the left hand elicitation method during the game for this process in combination with the introduced methods for strategic decisionmaking; however, this is a challenge for several students, so that they often do only apply the methods. This process is strongly supported in the common debriefing session and by the facilitation of the game. Active experimentation (doing) Based upon the outcome of the previous phase, the player changes the scenario according to the analysis and observation carried out so far. 7.2.3 Implemented changes Based on the formative evaluation of the course, several changes have been introduced. The ones affecting the game design and its functionalities are described in more detail chapter 4, but outlined for the overall evaluation again here. For the structure of the course, the following changes have been implemented and lead to improvements: • The course used to be given at two following days, as full day courses. This was exhausting for the students, and they reported that they did not have 7.3 Changes in the software based on evaluation 125 time enough to reflect and internalise the methods they should use. Since they had introduction, game play and debriefing on one and the same day. The first action was to have more time between the two days. That helped, but not enough so that it is now divided in four half-day parts, with one week in between. The first day is for introduction and hands-on. The second and third days is for playing level one, two and debriefings. The fourth day is used for presentations of the tasks of the game, group discussions and reflection. So far the results and the feedback from the students are positive. • The second major change regarding the structure was to increase the time for introducing relevant methods and processes for risk assessment and risk management and also for explaining how risks can occur and behave and how this affects the resilience. This is actually a normal lecture, and teacher centric. The effect has been that the students have a better level of knowledge on this topic before playing the game, and thus can concentrate more on the actually task in the gamei.e. to identify, asses and manage risks in enterprise network and to apply different methods, observing how it impacts the resilience. • As described previously in chapter 4, based on the evaluation the game has got a facilitator tool, the functions events and quality enhance measure have been implemented, the roles, the tasks and the processes have been redesigned. 7.3 Changes in the software based on evaluation The first usability studies showed that the students had difficulties in finding the right information and used too much time on getting along with the GUI. In addition, information on their tasks and their role was still paper-based, which distracted the players while playing. In a first step, a hands on, instead of a manual was introduced. This helped the student to concentrate on the game play instead of the interface. Still it is room for improvement, but a new GUI is under development. This is based on the analysis of using be.mog for different gaming scenarios. The second large change done, based on the evaluation of the usability of the game was that the information lacking is included so that the students find everything in the game. In addition, the introduction of the facilitator tool ensures that the students can get help as soon as they need. Furthermore, the implementation of the events helps to simulate a real working situation. It helps the facilitator to adapt the learning content to the individual level of the players (i.e. if a student has a lot of experience in working in networks or specialised in project management, he can get different tasks than a different player having less experience and knowledge). The possibility of having a sort of personal adaption helps to keep all students engaged and to reduce the risk of feeling overwhelmed or bored. 7 Results of the evaluation 126 The third main change that arise from the usability studies were the inclusion of the indicators as a feedback tool for the students to see how they performed during gameplay. It also improved the possibilities of analysing the relevance of communication to the results for the facilitator and is a useful help for the debriefing section. However, the used requires much experience. 7.4 Evaluation of the use of evaluation methods Tracking the communication as well as all the actions taken by the participants is very helpful, but requires a lot of experience of the facilitator. This information is also supporting the debriefing sessions. Using pre, mid and post questionnaires as well as collecting communication data and using in-built performance measures is time consuming. The experiments shows, that the students are motivated and reach the learning goals. However, the evaluation process is complex (especially the part based on interaction and communication), gives good results, but is time consuming and does not support immediate feedback. Validation of the learning goals-The results show that for students without any, or with a little knowledge of risk management, it is important to make their task more visible in the first game level. Furthermore, it was seen that the process of playing one game, debriefing it, and then playing another game level as expected helped to increase the performance in the second gameplay, most likely due to the transfer of knowledge. The participants identified the risks, as well as developed strategies for reducing the collaboration risks to a much higher degree. The continuous evaluation of the learning effect demonstrates that the time required to transfer information into knowledge does not only depends on the essential debriefing phase, but also relies on the experience that the participant already has. This needs to be taken into consideration at an early stage of the experimental set up, so that the students can be supplied with the necessary information on methods and approaches in advance.