Research Methods for the Empirical Assessment of Software Processes

For the state-of-practice in software engineering to improve, industry needs to apply methods and tools from software engineering research. To enable industrial software engineers to select which results are useful for them, research institutions need to provide sound evidence about the effectiveness of proposed methods and tools in practice. Research methodology for empirical software engineering has not fully matured yet and no consensus exists on what is acceptable as proof in empirical software engineering. This paper describes what problems exists with current research approaches and proposes a research plan to improve insights in what standards need to be met by research methods.

[1]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[2]  W. Chua.  Radical Developments in Accounting Thought , 1986 .

[3]  Neil Young,et al.  After the gold rush , 2013, Genome Biology.

[4]  M. Büchler,et al.  Perspectives of Evidence-Based Surgery , 2003, Digestive Surgery.

[5]  Tore Dybå,et al.  Evidence-based software engineering , 2004, Proceedings. 26th International Conference on Software Engineering.

[6]  Sjaak Brinkkemper,et al.  Measuring IT Infrastructure Project Size: Infrastructure Effort Points , 2005, CAiSE.

[7]  Magne Jørgensen,et al.  Generalization and theory-building in software engineering research , 2004, ICSE 2004.

[8]  Lucas Layman,et al.  Toward a framework for evaluating extreme programming , 2004, ICSE 2004.

[9]  James Miller.  Statistical significance testing--a panacea for software technology experiments? , 2004, J. Syst. Softw..

[10]  William E. Riddle,et al.  Software technology maturation , 1985, ICSE '85.

[11]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[12]  Mark C. Paulk,et al.  Capability Maturity Model for Software, Version 1.1 , 1993 .

[13]  Michael D. Myers,et al.  A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems , 1999, MIS Q..

[14]  Sjaak Brinkkemper,et al.  Discovering the Relation Between Project Factors and Project Success in Post-mortem Evaluations , 2004, EuroSPI.

[15]  Marvin V. Zelkowitz,et al.  Experimental Models for Validating Technology , 1998, Computer.

[16]  Mary Shaw,et al.  Writing good software engineering research papers , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[17]  Robert M. Davison,et al.  Principles of canonical action research , 2004, Inf. Syst. J..

[18]  Sjaak Brinkkemper,et al.  Assessing the Effects of Facilitated Workshops in Requirements Engineering , 2004, ICSE 2004.

[19]  Ray Welland,et al.  Collecting Feedback during Software Engineering Experiments , 2005, Empirical Software Engineering.

[20]  A. Weale.  Embedded Case Study Methods: Integrating Quantitative and Qualitative Knowledge , 2003 .

[21]  Paul Lukowicz,et al.  Experimental evaluation in computer science: A quantitative study , 1995, J. Syst. Softw..

[22]  Barbara Ann Kitchenham,et al.  Evaluating software engineering methods and tool part 1: The evaluation context and evaluation methods , 1996, SOEN.