Evaluating usefulness, ease of use and usability of an UML-based Software Product Line Tool

Software Product Line (SPL) is a software development approach that systematically applies reuse of artifacts in a specific domain. In the last years, the industry has increasingly required the support of tools for most SPL life cycle activities, targeting feature models and related diagrams, variability management and SPL specific products configuration. However, existing literature does not present any tools with native support to UML-based SPLs. In addition, relying on manipulating XMI files for general-purpose UML tools for such SPLs takes significant effort, and it is time-consuming and error-prone. In this scenario, we developed SMartyModeling, with support to UML stereotype-based variability management. To evolve our tool, we evaluated it throughout a survey answered by 37 participants. We adopted questions from the Technology Acceptance Model (TAM) and the System Usability Scale (SUS). We organized it into three sections of Likert-scaled questions for usefulness, ease of use, and usability. A last section consisted of open questions focused on positive and negative aspects and an overview of the evalaution. SMartyModeling was well evaluated in relation to usefulness, ease of use, and usability. We analyzed and interpreted the respondents quotes using correlation techniques and open and axial coding. The analysis of open questions allowed us a direct identification of points to improve the tool, its limitations and positive aspects.