Assessing the influence of feedback-inclusive rapid prototyping on understanding the semantics of parallel UML statecharts by novice modellers

Abstract Context UML diagrams are the de facto standard for analysing, communicating and designing software systems, as well as automated code generation. However there is a certain degree of difficulty in understanding a system represented by means of UML diagrams. Object Our previous research demonstrates a significant improvement in understanding the structural aspects of a system represented as a UML class diagram when using a feedback-inclusive prototype of a model. This paper extends our previous work with an empirical validation study for the effectiveness of the feedback-inclusive rapid prototyping (FIRP) method, on the comprehension of system dynamics represented as multiple interacting UML statecharts . Because models often combine structural and behavioural views that are highly intertwined, we additionally evaluate the effectiveness of the proposed method with respect to comprehension of the between-view consistency. Method The FIRP environment was built following the principles of Design Science Research in Information Systems. This study targets the empirical validation of the effectiveness of the proposed technique using an experimental study method. Two experiments were conducted with the participation of 65 final-year master students in the context of different modelling courses from different study programs at KU Leuven using two two-group factorial experimental designs. The effectiveness of the FIRP method was measured by comparing students’ performance between the cycles with and without the use of the method, using the understandability (comprehension test results) as the dependent variable and the use of FIRP as the independent variable. Effects from unknown variables were neutralized by means of randomized group compositions. The effectiveness of FIRP was additionally assessed with respect to personal characteristics (age, gender, previous knowledge, self-efficacy) and user acceptance (perceived ease of use, perceived utility, preference, satisfaction). Results The findings reveal a significant positive impact of the use of the prototyping technique on students’ comprehension of system dynamics represented as multiple interacting statecharts. Conclusions The findings provide empirical support for the advantage of the use of FIRP over manual inspection of interacting statecharts. The findings also suggest that the method is suitable for training system's analysis and modelling skills when UML statecharts are involved.