Demonstrating the usage of single-case designs in experimental software engineering

Abstract Experimental software engineering is the subdiscipline of empirical software engineering which uses experimentation to analyze, improve, and to validate software engineering methods (concepts, techniques, models). The efficacy and efficiency of some methods have already been demonstrated by experiments. To achieve this factorial designs were used so far. Critics of software engineering experiments argue that doing an experiment would be incredibly expensive. Furthermore, for doing software engineering experiments one would need hundreds of subjects. This paper shows that this is not necessarily right. An approach is presented that shows how to conduct software engineering experiments in a cost-effective way. The approach is useful to analyze software engineering problems specific to one subject, to conduct pilot experiments that precede in-depth experiments, and to accompany technology transfer. To demonstrate the usage of a single-subject experiment the domain of reuse was chosen.

[1]  J. Levin,et al.  Single-case research design and analysis : new directions for psychology and education , 1992 .

[2]  Erik Kamsties,et al.  An Empirical Evaluation of Three Defect-Detection Techniques , 1995, ESEC.

[3]  P. Lachenbruch,et al.  Design Sensitivity: Statistical Power for Experimental Research. , 1989 .

[4]  Abraham Bookstein,et al.  Fuzzy requests: An approach to weighted boolean searches , 1980, J. Am. Soc. Inf. Sci..

[5]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[6]  Leonard A. Marascuilo,et al.  Statistical analysis in single-case research: Issues, procedures, and recommendations, with applications to multiple behaviors. , 1992 .

[7]  Franz Lehner,et al.  Experimental comparison of coarse-grained concepts in UML, OML, and TOS , 2001, J. Syst. Softw..

[8]  Rubén Prieto-Díaz.  Implementing faceted classification for software reuse , 1991, CACM.

[9]  Alan F. Blackwell,et al.  Mental imagery in program design and visual programming , 1999, Int. J. Hum. Comput. Stud..

[10]  M. Hersen.  Single case experimental designs , 1992 .

[11]  Anneliese Amschler Andrews,et al.  Program understanding needs during corrective maintenance of large scale software , 1997, Proceedings Twenty-First Annual International Computer Software and Applications Conference (COMPSAC'97).

[12]  Rubén Prieto Díaz,et al.  A software classification scheme , 1985 .

[13]  Kevin Mills.  An Experimental Evaluation of Specification Techniques for Improving Functional Testing , 1996, J. Syst. Softw..

[14]  Ted J. Biggerstaff,et al.  Concepts and models , 1989 .

[15]  Marx W. Wartofsky,et al.  Conceptual foundations of scientific thought , 1968 .

[16]  Susan Wiedenbeck,et al.  Novice comprehension of small programs written in the procedural and object-oriented styles , 1999, Int. J. Hum. Comput. Stud..

[17]  Marian G. Williams,et al.  Comparison of visual and textual languages via task modeling , 1999, Int. J. Hum. Comput. Stud..

[18]  Victor R. Basili,et al.  Experimentation in software engineering , 1986, IEEE Transactions on Software Engineering.

[19]  J. Gentile,et al.  An analysis-of-variance model for the intrasubject replication design. , 1972, Journal of applied behavior analysis.

[20]  Chris F. Kemerer,et al.  Software complexity and software maintenance: A survey of empirical research , 1995, Ann. Softw. Eng..

[21]  Sidney Addelman,et al.  trans-Dimethanolbis(1,1,1-trifluoro-5,5-dimethylhexane-2,4-dionato)zinc(II) , 2008, Acta crystallographica. Section E, Structure reports online.

[22]  Michelle Cartwright,et al.  An empirical view of inheritance , 1998, Inf. Softw. Technol..

[23]  Sholom Cohen,et al.  Object-oriented technology and domain analysis , 1998, Proceedings. Fifth International Conference on Software Reuse (Cat. No.98TB100203).

[24]  M. Bunge,et al.  Scientific Research I: The Search for System , 1967 .

[25]  Robert J. Kauffman,et al.  Repository Evaluation of Software Reuse , 1993, IEEE Trans. Software Eng..

[26]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[27]  D. Barlow,et al.  Single Case Experimental Designs: Strategies for Studying Behavior Change , 1976 .

[28]  W. W. Daniel.  Applied Nonparametric Statistics , 1979 .

[29]  Eugene S. Edgington.  Nonparametric tests for single-case experiments. , 2015 .

[30]  David F. Foster,et al.  Experimental Psychology: A Small-N Approach , 1979 .

[31]  Jean Scholtz,et al.  Introduction to the Special Issue "Best of Empirical Studies of Programmers 7" , 1999, Int. J. Hum. Comput. Stud..

[32]  Anneliese Amschler Andrews,et al.  Program understanding behavior during corrective maintenance of large-scale software , 1999, Int. J. Hum. Comput. Stud..

[33]  William G. Cochran,et al.  Experimental Designs, 2nd Edition , 1950 .

[34]  Gail E. Kaiser,et al.  An Information Retrieval Approach For Automatically Constructing Software Libraries , 1991, IEEE Trans. Software Eng..

[35]  William B. Frakes,et al.  Representing reusable software , 1990 .

[36]  Scott Henninger,et al.  An evolutionary approach to constructing effective software reuse repositories , 1997, TSEM.

[37]  D. Campbell,et al.  EXPERIMENTAL AND QUASI-EXPERIMENT Al DESIGNS FOR RESEARCH , 2012 .

[38]  S. Sonnentag.  Expertise in professional software design: a process study. , 1998, The Journal of applied psychology.

[39]  William B. Frakes,et al.  An Empirical Study of Representation Methods for Reusable Software Components , 1994, IEEE Trans. Software Eng..

[40]  T. M. Rajkumar,et al.  Comparison of analysis techniques for information requirement determination , 1988, CACM.

[41]  Hall Chapman,et al.  Software Reuse and Reverse Engineering in Practice , 1992 .

[42]  Walter F. Tichy,et al.  A Controlled Experiment to Assess the Benefits of Procedure Argument Type Checking , 1998, IEEE Trans. Software Eng..