Program comprehension of domain-specific and general-purpose languages: replication of a family of experiments using integrated development environments

Domain-specific languages (DSLs) allow developers to write code at a higher level of abstraction compared with general-purpose languages (GPLs). Developers often use DSLs to reduce the complexity of GPLs. Our previous study found that developers performed program comprehension tasks more accurately and efficiently with DSLs than with corresponding APIs in GPLs. This study replicates our previous study to validate and extend the results when developers use IDEs to perform program comprehension tasks. We performed a dependent replication of a family of experiments. We made two specific changes to the original study: (1) participants used IDEs to perform the program comprehension tasks, to address a threat to validity in the original experiment and (2) each participant performed program comprehension tasks on either DSLs or GPLs, not both as in the original experiment. The results of the replication are consistent with and expanded the results of the original study. Developers are significantly more effective and efficient in tool-based program comprehension when using a DSL than when using a corresponding API in a GPL. The results indicate that, where a DSL is available, developers will perform program comprehension better using the DSL than when using the corresponding API in a GPL.

[1]  Arie van Deursen,et al.  Domain-specific language design requires feature descriptions , 2002 .

[2]  Jeffrey C. Carver,et al.  The role of replications in Empirical Software Engineering , 2008, Empirical Software Engineering.

[3]  Ho-Won Jung,et al.  Measuring software product quality: a survey of ISO/IEC 9126 , 2004, IEEE Software.

[4]  Hanspeter Mössenböck,et al.  Monaco - A domain-specific language solution for reactive process control programming with hierarchical components , 2013, Comput. Lang. Syst. Struct..

[5]  Bernhard Hoisl,et al.  Comparing Three Notations for Defining Scenario-Based Model Tests: A Controlled Experiment , 2014, 2014 9th International Conference on the Quality of Information and Communications Technology.

[6]  Emden R. Gansner,et al.  Drawing graphs with dot , 2006 .

[7]  Marjan Mernik,et al.  Domain-Specific Languages: A Systematic Mapping Study , 2016, Inf. Softw. Technol..

[8]  Mickey Williams.  Microsoft Visual C# (Core Reference) , 2002 .

[9]  Andrei Chis,et al.  Practical domain-specific debuggers using the Moldable Debugger framework , 2015, Comput. Lang. Syst. Struct..

[10]  M Mernik,et al.  When and how to develop domain-specific languages , 2005, CSUR.

[11]  Jeffrey C. Carver,et al.  Assessing the Frequency of Empirical Evaluation in Software Modeling Research , 2011, EESSMod.

[12]  Miguel Goulão,et al.  Quality in model-driven engineering: a tertiary study , 2016, Software Quality Journal.

[13]  Arie van Deursen,et al.  A Controlled Experiment for Program Comprehension through Trace Visualization , 2011, IEEE Transactions on Software Engineering.

[14]  Barbara A. Kitchenham,et al.  The role of replications in empirical software engineering—a word of warning , 2008, Empirical Software Engineering.

[15]  Charles Consel,et al.  Architecture Software Using: A Methodology for Language Development , 1998, PLILP/ALP.

[16]  Paul Hudak,et al.  Modular domain specific languages and tools , 1998, Proceedings. Fifth International Conference on Software Reuse (Cat. No.98TB100203).

[17]  Ariadi Nugroho,et al.  Level of detail in UML models and its impact on model comprehension: A controlled experiment , 2009, Inf. Softw. Technol..

[18]  Marjan Mernik,et al.  An object-oriented approach to language compositions for software language engineering , 2013, J. Syst. Softw..

[19]  Jon Louis Bentley,et al.  Programming pearls: little languages , 1986, CACM.

[20]  Sebastian Erdweg,et al.  Language composition untangled , 2012, LDTA.

[21]  Jeffrey C. Carver,et al.  Replication types: towards a shared taxonomy , 2014, EASE '14.

[22]  Ruth Breu,et al.  Is business domain language support beneficial for creating test case specifications: A controlled experiment , 2016, Inf. Softw. Technol..

[23]  Nuno Oliveira,et al.  Comparing general-purpose and domain-specific languages: An empirical study , 2010, Comput. Sci. Inf. Syst..

[24]  Cristina Cachero,et al.  Comparison of a textual versus a graphical notation for the maintainability of MDE domain models: an empirical pilot study , 2015, Software Quality Journal.

[25]  Arie van Deursen,et al.  REPORT RAPPORT , 1997 .

[26]  Margaret-Anne D. Storey,et al.  Theories, Methods and Tools in Program Comprehension: Past, Present and Future , 2005, IWPC.

[27]  Diomidis Spinellis,et al.  Guest Editors' Introduction: What Kinds of Nails Need a Domain-Specific Hammer? , 2009, IEEE Software.

[28]  Miguel Goulão,et al.  Usability driven DSL development with USE-ME , 2018, Comput. Lang. Syst. Struct..

[29]  Arie van Deursen,et al.  Domain-specific languages: an annotated bibliography , 2000, SIGP.

[30]  Simone Diniz Junqueira Barbosa,et al.  Quantifying usability of domain-specific languages: An empirical study on software maintenance , 2015, J. Syst. Softw..

[31]  Jordi Cabot,et al.  An empirical study on simplification of business process modeling languages , 2015, SLE.

[32]  Maria João Varanda Pereira,et al.  Ontological approach for DSL development , 2016, Comput. Lang. Syst. Struct..

[33]  Alan R. Hevner,et al.  The Impact of Function Extraction Technology on Next-Generation Software Engineering , 2005 .

[34]  Arie van Deursen,et al.  Little languages: little maintenance , 1998 .

[35]  Arie van Deursen,et al.  Domain-Specific Languages in Practice: A User Study on the Success Factors , 2009, MoDELS.

[36]  Wilhelm Hasselbring,et al.  Effectiveness and efficiency of a domain-specific language for high-performance marine ecosystem simulation: a controlled experiment , 2016, Empirical Software Engineering.

[37]  Jeffrey C. Carver.  Towards Reporting Guidelines for Experimental Replications: A Proposal , 2010 .

[38]  Jeffrey C. Carver,et al.  Program comprehension of domain-specific and general-purpose languages: comparison using a family of experiments , 2011, Empirical Software Engineering.

[39]  Roberto da Silva Bigonha,et al.  An on-the-fly grammar modification mechanism for composing and defining extensible languages , 2015, Comput. Lang. Syst. Struct..

[40]  David S. Wile,et al.  Supporting the DSL Spectrum , 2001 .

[41]  Sjouke Mauw,et al.  Language-driven system design , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.