Empirical evidence about the UML: a systematic literature review

The Unified Modeling Language (UML) was created on the basis of expert opinion and has now become accepted as the ‘standard’ object‐oriented modelling notation. Our objectives were to determine how widely the notations of the UML, and their usefulness, have been studied empirically, and to identify which aspects of it have been studied in most detail. We undertook a mapping study of the literature to identify relevant empirical studies and to classify them in terms of the aspects of the UML that they studied. We then conducted a systematic literature review, covering empirical studies published up to the end of 2008, based on the main categories identified. We identified 49 relevant publications, and report the aggregated results for those categories for which we had enough papers— metrics, comprehension, model quality, methods and tools and adoption. Despite indications that a number of problems exist with UML models, researchers tend to use the UML as a ‘given’ and seem reluctant to ask questions that might help to make it more effective. Copyright © 2010 John Wiley & Sons, Ltd.

[1]  Mario Piattini,et al.  Does object coupling really affect the understanding and modifying of OCL expressions? , 2006, SAC '06.

[2]  Brian Henderson-Sellers,et al.  Open Modeling Language (OML)-Refer-ence Manual , 1998 .

[3]  Genny Tortora,et al.  Assessing the Support of ER and UML Class Diagrams during Maintenance Activities on Data Models , 2008, 2008 12th European Conference on Software Maintenance and Reengineering.

[4]  Daniel L. Moody,et al.  The “Physics” of Notations: Toward a Scientific Basis for Constructing Visual Notations in Software Engineering , 2009, IEEE Transactions on Software Engineering.

[5]  Parastoo Mohagheghi,et al.  Definitions and approaches to model quality in model-based software development - A review of literature , 2009, Inf. Softw. Technol..

[6]  Mario Piattini,et al.  Evaluating the effect of composite states on the understandability of UML statechart diagrams , 2005, MoDELS'05.

[7]  Mario Piattini,et al.  A Survey of Metrics for UML Class Diagrams , 2005, J. Object Technol..

[8]  Richard V. McCarthy,et al.  Does UML make the grade? Insights from the software development community , 2005, Inf. Softw. Technol..

[9]  Mario Piattini,et al.  A Controlled Experiment for Validating Class Diagram Structural Complexity Metrics , 2002, OOIS.

[10]  Mario Piattini,et al.  Early Measures for UML Class Diagrams , 2000, Obj. Logiciel Base données Réseaux.

[11]  David A. Carrington,et al.  UML Class Diagram Syntax: An Empirical Study of Comprehension , 2001, InVis.au.

[12]  Matthew Lang,et al.  Experiences of using rational rose/visio for UML modeling in an undergraduate software engineering course: a student perspective , 2004 .

[13]  Pearl Brereton,et al.  Presenting software engineering results using structured abstracts: a randomised experiment , 2008, Empirical Software Engineering.

[14]  José Ambrosio Toval Álvarez,et al.  A systematic review of UML model consistency management , 2009, Inf. Softw. Technol..

[15]  José Javier Dolado,et al.  An empirical comparison of the dynamic modeling in OML and UML , 2005, J. Syst. Softw..

[16]  Maria Kutar,et al.  An empirical study of factors that affect user performance when using UML interaction diagrams , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[17]  Michel R. V. Chaudron,et al.  An empirical assessment of completeness in UML designs , 2004, ICSE 2004.

[18]  David A. Carrington,et al.  Empirical Evaluation of Aesthetics-based Graph Layout , 2002, Empirical Software Engineering.

[19]  David A. Carrington,et al.  UML collaboration diagram syntax: an empirical study of comprehension , 2002, Proceedings First International Workshop on Visualizing Software for Understanding and Analysis.

[20]  Mario Piattini,et al.  Building measure-based prediction models for UML class diagram maintainability , 2007, Empirical Software Engineering.

[21]  Jeffrey Parsons,et al.  How UML is used , 2006, CACM.

[22]  Franz Lehner,et al.  Experimental comparison of coarse-grained concepts in UML, OML, and TOS , 2001, J. Syst. Softw..

[23]  Mario Piattini,et al.  Empirical validation of class diagram metrics , 2002, Proceedings International Symposium on Empirical Software Engineering.

[24]  Mario Piattini,et al.  MEASURING OCL EXPRESSIONS: AN APPROACH BASED ON COGNITIVE TECHNIQUES , 2005 .

[25]  Bente Anda,et al.  Investigating the Role of Use Cases in the Construction of Class Diagrams , 2005, Empirical Software Engineering.

[26]  Christian Bunse,et al.  Using patterns for the refinement and translationof UML models: A controlled experiment , 2006, Empirical Software Engineering.

[27]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[28]  Barry W. Boehm,et al.  An empirical study of eServices product UML sizing metrics , 2004, Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE '04..

[29]  Michel R. V. Chaudron,et al.  Empirical Analysis of the Relation between Level of Detail in UML Models and Defect Density , 2008, MoDELS.

[30]  Marco Torchiano.  Empirical assessment of UML static object diagrams , 2004, Proceedings. 12th IEEE International Workshop on Program Comprehension, 2004..

[31]  Mario Piattini,et al.  Assessing the impact of coupling on the understandability and modifiability of OCL expressions within UML/OCL combined models , 2005, 11th IEEE International Software Metrics Symposium (METRICS'05).

[32]  Michel R. V. Chaudron,et al.  A survey into the rigor of UML use and its perceived impact on quality and productivity , 2008, ESEM '08.

[33]  Shinji Kusumoto,et al.  An experimental comparison of checklist-based reading and perspective-based reading for UML design document inspection , 2002, Proceedings International Symposium on Empirical Software Engineering.

[34]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[35]  David Budgen,et al.  A mapping study on empirical evidence related to the models and forms used in the uml , 2008, ESEM '08.

[36]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A systematic literature review , 2009, Inf. Softw. Technol..

[37]  Bente Anda,et al.  Experiences from introducing UML-based development in a large safety-critical project , 2006, Empirical Software Engineering.

[38]  Miroslaw Staron,et al.  Empirical assessment of using stereotypes to improve comprehension of UML models: A set of experiments , 2006, J. Syst. Softw..

[39]  Michel R. V. Chaudron,et al.  Supporting task-oriented modeling using interactive UML views , 2007, J. Vis. Lang. Comput..

[40]  Mario Piattini,et al.  Finding "early" indicators of UML class diagrams understandability and modifiability , 2004, Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE '04..

[41]  Dov Dori,et al.  Object-process methodology - a holistic systems paradigm , 2013 .

[42]  Mrv Michel Chaudron,et al.  Effects of defects in UML models , 2006 .

[43]  Tore Dybå,et al.  Evidence-based software engineering , 2004, Proceedings. 26th International Conference on Software Engineering.

[44]  Dale L. Goodhue,et al.  Development and Measurement Validity of a Task-Technology Fit Instrument for User Evaluations of Inf , 1998 .

[45]  Danilo Caivano,et al.  Assessing the Influence of Stereotypes on the Comprehension of UML Sequence Diagrams: A Controlled Experiment , 2008, MoDELS.

[46]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences: A Practical Guide , 2005 .

[47]  Shihong Huang,et al.  A qualitative assessment of the efficacy of UML diagrams as a form of graphical documentation in aiding program understanding , 2003, SIGDOC '03.

[48]  Michel R. V. Chaudron,et al.  Effects of defects in UML models: an experimental investigation , 2006, ICSE.

[49]  Mario Piattini,et al.  Building UML class diagram maintainability prediction models based on early metrics , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[50]  José Javier Dolado,et al.  Evaluation of the comprehension of the dynamic modeling in UML , 2004, Inf. Softw. Technol..

[51]  Lionel C. Briand,et al.  Automated impact analysis of UML models , 2006, J. Syst. Softw..

[52]  Narasimha Bolloju,et al.  Assisting novice analysts in developing quality conceptual models with UML , 2006, CACM.

[53]  Marco Torchiano,et al.  The Role of Experience and Ability in Comprehension Tasks Supported by UML Stereotypes , 2007, 29th International Conference on Software Engineering (ICSE'07).

[54]  Michele Marchesi.  OOA metrics for the Unified Modeling Language , 1998, Proceedings of the Second Euromicro Conference on Software Maintenance and Reengineering.

[55]  Michel R. V. Chaudron,et al.  In practice: UML software architecture and design description , 2006, IEEE Software.

[56]  Cheng Zhang,et al.  What Do We Know about the Effectiveness of Software Design Patterns? , 2012, IEEE Transactions on Software Engineering.

[57]  Tore Dybå,et al.  A systematic review of statistical power in software engineering experiments , 2006, Inf. Softw. Technol..

[58]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A tertiary study , 2010, Inf. Softw. Technol..

[59]  Dov Dori,et al.  OPM vs. UML--Experimenting with Comprehension and Construction of Web Application Models , 2004, Empirical Software Engineering.

[60]  Peretz Shoval,et al.  Quality and comprehension of UML interaction diagrams-an experimental comparison , 2005, Inf. Softw. Technol..

[61]  Maria Kutar,et al.  An empirical study of user preference and performance with UML diagrams , 2002, Proceedings IEEE 2002 Symposia on Human Centric Computing Languages and Environments.

[62]  Pearl Brereton,et al.  Using Mapping Studies in Software Engineering , 2008, PPIG.

[63]  José Javier Dolado,et al.  An Initial Experimental Assessment of the Dynamic Modelling in UML , 2004, Empirical Software Engineering.