Usability Evaluation of Variability Modeling by means of Common Variability Language

Common Variability Language (CVL) is a recent proposal for OMG's upcoming Variability Modeling standard. CVL models variability in terms of Model Fragments.  Usability is a widely-recognized quality criterion essential to warranty the successful use of tools that put these ideas in practice. Facing the need of evaluating the usability of CVL modeling tools, this paper presents a Usability Evaluation of CVL applied to a Modeling Tool for firmware code of Induction Hobs. This evaluation addresses the configuration, scoping and visualization facets. The evaluation involved the end users of the tool whom are engineers of our Induction Hob industrial partner. Effectiveness and efficiency results indicate that model configuration in terms of model fragment substitutions is intuitive enough but both scoping and visualization require improved tool support. Results also enabled us to identify a list of usability problems which may contribute to alleviate scoping and visualization issues in CVL.

[1]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[2]  Olivier Barais,et al.  Towards managing variability in the safety design of an automotive hall effect sensor , 2014, SPLC.

[3]  Tanja E. J. Vos,et al.  An empirical approach for evaluating the usability of model-driven tools , 2013, Sci. Comput. Program..

[4]  J. B. Brooke,et al.  SUS: a retrospective , 2013 .

[5]  Paul Grünbacher,et al.  Flexibility and End-User Support in Model-Based Product Line Tools , 2009, 2009 35th Euromicro Conference on Software Engineering and Advanced Applications.

[6]  Goetz Botterweck,et al.  Visual Tool Support for Configuring and Understanding Software Product Lines , 2008, 2008 12th International Software Product Line Conference.

[7]  Dominique L. Scapin,et al.  Ergonomic criteria for evaluating the ergonomic quality of interactive systems , 1997, Behav. Inf. Technol..

[8]  Natalia Juristo Juzgado,et al.  How to integrate usability into the software development process , 2006, ICSE.

[9]  Robert A. Virzi,et al.  Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? , 1992 .

[10]  Jean Vanderdonckt,et al.  Usability Evaluation of User Interfaces Generated with a Model-Driven Architecture Tool , 2008, Maturing Usability.

[11]  Andreas Holzinger,et al.  Usability engineering methods for software developers , 2005, CACM.

[12]  Miguel Goulão,et al.  Usability Evaluation of Domain-Specific Languages , 2012, 2012 Eighth International Conference on the Quality of Information and Communications Technology.

[13]  David E. Kieras.  Using the Keystroke-Level Model to Estimate Execution Times , 2003 .

[14]  Birger Møller-Pedersen,et al.  A Generic Language and Tool for Variability Modeling , 2009 .

[15]  Krzysztof Czarnecki,et al.  A survey of variability modeling in industrial practice , 2013, VaMoS.

[16]  Daniel Amyot,et al.  Evaluation of Development Tools for Domain-Specific Modeling Languages , 2006, SAM.

[17]  Hoan Anh Nguyen,et al.  Complete and accurate clone detection in graph-based models , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[18]  Alexandr Murashkin,et al.  Visualization and exploration of optimal variants in product line engineering , 2013, SPLC '13.

[19]  Iris Reinhartz-Berger,et al.  Comprehending Feature Models Expressed in CVL , 2014, MoDELS.

[20]  Mathieu Acher,et al.  Moving toward product line engineering in a nuclear industry consortium , 2014, SPLC.

[21]  Birger Møller-Pedersen,et al.  Adding Standardized Variability to Domain Specific Languages , 2008, 2008 12th International Software Product Line Conference.

[22]  Paul Grünbacher,et al.  Product Line Tools are Product Lines Too: Lessons Learned from Developing a Tool Suite , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[23]  Iris Reinhartz-Berger,et al.  Comprehensibility of orthogonal variability modeling languages: the cases of CVL and OVM , 2014, SPLC.

[24]  Paul Grünbacher,et al.  A qualitative study on user guidance capabilities in product configuration tools , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[25]  Christian Wende,et al.  On Controlled Visualisations in Software Product Line Engineering , 2008, SPLC.

[26]  Thomas S. Tullis,et al.  A Comparison of Questionnaires for Assessing Website Usability , 2004 .