Studying the Effect of UML-Based Models on Source-Code Comprehensibility: Results from a Long-Term Investigation

In this paper, we present final results of our long-term investigation whose goal was to study the contribution of software models on source-code comprehensibility. In this investigation we considered unified modeling language UML models produced in the analysis and design phases, and we conducted 12 controlled experiments in different contexts with different kinds of participants e.g., PhD students and software practitioners. The total number of observations from these experiments was 333. We considered both source-code comprehensibility and the time to complete comprehension tasks. We use a meta-analysis to integrate gathered data and to obtain a global effect of analysis and design models on source-code comprehensibility. Results suggest that the use of UML models affects source-code comprehensibility, but in two opposite directions. In particular, models produced in the analysis phase reduce source-code comprehensibility and increase the time to complete comprehension tasks, while models produced in the design phase improve source-code comprehensibility and reduce the time to complete comprehension tasks.

[1]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2012, Springer Berlin Heidelberg.

[2]  Liming Zhu,et al.  Evaluating guidelines for reporting empirical software engineering studies , 2008, Empirical Software Engineering.

[3]  Forrest Shull,et al.  The empirical investigation of Perspective-Based Reading , 1995, Empirical Software Engineering.

[4]  Giuseppe Scanniello,et al.  An empirical investigation on the relation between analysis models and source code comprehension , 2010, SAC '10.

[5]  Giuseppe Scanniello,et al.  A controlled experiment for assessing the contribution of design pattern documentation on software maintenance , 2010, ESEM '10.

[6]  Karin Ried,et al.  Interpreting and understanding meta-analysis graphs--a practical guide. , 2006, Australian family physician.

[7]  Victor R. Basili,et al.  The TAME Project: Towards Improvement-Oriented Software Environments , 1988, IEEE Trans. Software Eng..

[8]  Ritu Agarwal,et al.  Object-oriented modeling with UML: a study of developers' perceptions , 2003, CACM.

[9]  A. N. Oppenheim,et al.  Questionnaire Design, Interviewing and Attitude Measurement , 1992 .

[10]  Michael Philippsen,et al.  Two Controlled Experiments Assessing the Usefulness of Design Pattern Documentation in Program Maintenance , 2002, IEEE Trans. Software Eng..

[11]  Richard V. McCarthy,et al.  Does UML make the grade? Insights from the software development community , 2005, Inf. Softw. Technol..

[12]  H. James Nelson,et al.  Research Review: A Systematic Literature Review on the Quality of UML Models , 2011, J. Database Manag..

[13]  Giuseppe Scanniello,et al.  Source-code comprehension tasks supported by UML design models: Results from a controlled experiment and a differentiated replication , 2015, J. Vis. Lang. Comput..

[14]  Nicolas Anquetil,et al.  A study of the documentation essential to software maintenance , 2005, SIGDOC '05.

[15]  Giuseppe Scanniello,et al.  Investigating the Role of UML in the Software Modeling and Maintenance - A Preliminary Industrial Survey , 2010, ICEIS.

[16]  Jeffrey C. Carver,et al.  A Framework for Software Engineering Experimental Replications , 2008, 13th IEEE International Conference on Engineering of Complex Computer Systems (iceccs 2008).

[17]  Dietmar Pfahl,et al.  Reporting Experiments in Software Engineering , 2008, Guide to Advanced Empirical Software Engineering.

[18]  Eirik Tryggeseth,et al.  Report from an Experiment: Impact of Documentation on Maintenance , 1997, Empirical Software Engineering.

[19]  Mikael Lindvall,et al.  An introduction to agile methods , 2004, Adv. Comput..

[20]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[21]  Natalia Juristo Juzgado,et al.  Basics of Software Engineering Experimentation , 2010, Springer US.

[22]  Siw Elisabeth Hove,et al.  The impact of UML documentation on software maintenance: an experimental evaluation , 2006, IEEE Transactions on Software Engineering.

[23]  Giuseppe Scanniello,et al.  Documenting Design-Pattern Instances: A Family of Experiments on Source-Code Comprehensibility , 2015, TSEM.

[24]  Tatyana Shamliyan,et al.  Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program. , 2011, Journal of clinical epidemiology.

[25]  Pearl Brereton,et al.  Empirical evidence about the UML: a systematic literature review , 2011, Softw. Pract. Exp..

[26]  Silvia Mara Abrahão,et al.  Assessing the Effectiveness of Sequence Diagrams in the Comprehension of Functional Requirements: Results from a Family of Five Experiments , 2013, IEEE Transactions on Software Engineering.

[27]  Barbara A. Kitchenham,et al.  Combining empirical results in software engineering , 1998, Inf. Softw. Technol..

[28]  Keng Siau,et al.  Theoretical and practical complexity of modeling methods , 2007, CACM.

[29]  Bente Anda,et al.  Experiences from introducing UML-based development in a large safety-critical project , 2006, Empirical Software Engineering.

[30]  Amela Karahasanovic,et al.  A survey of controlled experiments in software engineering , 2005, IEEE Transactions on Software Engineering.

[31]  Giuseppe Scanniello,et al.  On the impact of UML analysis models on source-code comprehensibility and modifiability , 2014, ACM Trans. Softw. Eng. Methodol..