Methodology and experimental research in software engineering

There has been a very rapid increase in the use of psychological experimentation as a method for addressing problems in the area of software engineering. However, there is a total lack of understanding on how to use this technique effectively so that the results of the experiment will have both validity and wide applicability. This paper describes what we feel is a major new research area which aims at formalizing the use of controlled group experimentation in programming language design and software engineering. The paper describes the growth of the problem, the critical need for this new research area and finally categorizes the problems that must be addressed as part of the research into these problems. An annotated bibliography contains literature references (through 1978) to the use of controlled group experimentation in software engineering and related areas of Computer Science.

[1]  H. Sackman,et al.  An Exploratory Investigation of Programmer Performance Under On-Line and Off-Line Conditions , 1967 .

[2]  T. R. G. Green.  Conditional program statements and their comprehensibility to professional programmers , 1977 .

[3]  H. Sackmanh.  Experimental Analysis of Man-Computer Problem-Solving , 1970 .

[4]  Ben Shneiderman,et al.  Measuring Computer Program Quality and Comprehension , 1977, Int. J. Man Mach. Stud..

[5]  Henry C. Lucas,et al.  A Structured Programming Experiment , 1976, Comput. J..

[6]  Larry Weissman,et al.  Psychological complexity of computer programs: an experimental methodology , 1974, SIGP.

[7]  John D. Gould,et al.  An Exploratory Study of Computer Program Debugging1 , 1974 .

[8]  Edward A. Youngs.  Human Errors in Programming , 1974, Int. J. Man Mach. Stud..

[9]  Thomas R. G. Green,et al.  Scope Marking in Computer Conditionals - A Psychological Evaluation , 1977, Int. J. Man Mach. Stud..

[10]  Lance A. Miller.  Programming by Non-Programmers , 1974, Int. J. Man Mach. Stud..

[11]  David W. Embley,et al.  Empirical and formal language design applied to a unified control construct for interactive computing , 1978 .

[12]  Gerald M. Weinberg,et al.  Goals and Performance in Computer Programming , 1974 .

[13]  Phyllis Reisner,et al.  Use of Psychological Experimentation as an Aid to Development of a Query Language , 1977, IEEE Transactions on Software Engineering.

[14]  Glenford J. Myers,et al.  A controlled experiment in program testing and code walkthroughs/inspections , 1978, CACM.

[15]  John D. Gould.  Writing and Following Procedural, Descriptive, and Restricted Syntax Language Instructions. , 1976 .

[16]  Gordon B. Davis,et al.  A study of errors, error-proneness, and error diagnosis in Cobol , 1976, CACM.

[17]  Donald E. Knuth,et al.  An empirical study of FORTRAN programs , 1971, Softw. Pract. Exp..

[18]  Brian R. Gaines,et al.  The Fuzzy Decade: A Bibliography of Fuzzy Systems and Closely Related Topics , 1977, Int. J. Man Mach. Stud..

[19]  T. C. S. Kennedy,et al.  Some Behavioral Factors Affecting the Training of Naive Users of an Interaction Computer System , 1975, Int. J. Man Mach. Stud..

[20]  John D Gould,et al.  Use of an IQF-Like Query Language by Non-programmers. , 1975 .

[21]  James J. Horning,et al.  Language design for programming reliability , 1975, IEEE Transactions on Software Engineering.

[22]  Ben Shneiderman,et al.  Experimental investigations of the utility of detailed flowcharts in programming , 1977, CACM.

[23]  Edsger W. Dijkstra,et al.  The humble programmer , 1972, CACM.

[24]  Richard Kammann,et al.  The Comprehensibility of Printed Instructions and the Flowchart Alternative , 1975 .

[25]  John Gannon.  An experiment for the evaluation of language features , 1976 .

[26]  John D. Gould,et al.  Syntactic Errors in Computer Programming , 1974 .

[27]  John A. Gould,et al.  Some Psychological Evidence on How People Debug Computer Programs , 1975, Int. J. Man Mach. Stud..

[28]  David A. Bell,et al.  Programmer Selection and Programming Errors , 1976, Comput. J..

[29]  Tom Love,et al.  A Review and Evaluation of Software Science , 1978, CSUR.

[30]  John D. Gannon.  An experimental evaluation of data type conventions , 1977, CACM.