Rigor in software complexity measurement experimentation

Abstract The lack of widespread industry acceptance of much of the research into the measurement of software complexity must be due at least in part to the lack of experimental rigor associated with many of the studies. This article examines 13 areas in which previous empirical problems have arisen, citing examples where appropriate, and provides recommendations regarding more adequate procedures.

[1]  H. E. Dunsmore,et al.  Software Science Revisited: A Critical Analysis of the Theory and Its Empirical Support , 1983, IEEE Transactions on Software Engineering.

[2]  Sallie M. Henry,et al.  Software Structure Metrics Based on Information Flow , 1981, IEEE Transactions on Software Engineering.

[3]  Linda M. Ottenstein,et al.  Predicting numbers of errors using software science , 1981, Measurement and evaluation of software quality.

[4]  G. D. Frewin,et al.  M.H. Halstead's Software Science - a critical examination , 1982, ICSE '82.

[5]  Richard A. Kemmerer,et al.  Complexity measures for assembly language programs , 1985, J. Syst. Softw..

[6]  Ron Weber,et al.  Some factors affecting program repair maintenance: an empirical study , 1983, CACM.

[7]  Tom Love,et al.  An experimental investigation of the effect of program structure on program understanding , 1977, Language Design for Reliable Software.

[8]  Murat M. Tanik,et al.  Two experiments on a program complexity perception by programmers , 1980, SIGP.

[9]  WeberRon,et al.  Some factors affecting program repair maintenance , 1983 .

[10]  Hubert E. Dunsmore Software metrics: An overview of an evolving methodology , 1984, Inf. Process. Manag..

[11]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[12]  Volney Rodriguez,et al.  SOFTWARE METRICS INTERPRETATION THROUGH EXPERIMENTATION. , 1986 .

[13]  Scott N. Woodfield An Experiment on Unit Increase in Problem Complexity , 1979, IEEE Transactions on Software Engineering.

[14]  Lee L. Gremillion Determinants of program repair maintenance requirements , 1984, CACM.

[15]  Joseph K. Kearney,et al.  Software complexity measurement , 1986, CACM.

[16]  Murat M. Tanik,et al.  A comparison of two different program complexity measures , 1981, PERV.

[17]  Stuart H. Zweben Surveyor's Forum: Heads I Win, Tails You Lose , 1979, CSUR.

[18]  Victor R. Basili,et al.  Metric Analysis and Data Validation Across Fortran Projects , 1983, IEEE Transactions on Software Engineering.

[19]  Scott N. Woodfield,et al.  A study of several metrics for programming effort , 1981, J. Syst. Softw..

[20]  Michael L. Cook,et al.  Software metrics , 1982, ACM SIGSOFT Softw. Eng. Notes.

[21]  Andrew M. Lister,et al.  Software Science - The Emperor's New Clothes? , 1982, Aust. Comput. J..

[22]  Frederick G. Sayward Experimental design methodologies in software science , 1984, Inf. Process. Manag..

[23]  John E. Gaffney,et al.  Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation , 1983, IEEE Transactions on Software Engineering.

[24]  Dennis G. Kafura,et al.  The Use of Software Complexity Metrics in Software Maintenance , 1987, IEEE Transactions on Software Engineering.

[25]  John D. Gannon,et al.  Analysis of the effects of programming factors on programming effort , 1980, J. Syst. Softw..

[26]  Larry Weissman,et al.  Psychological complexity of computer programs: an experimental methodology , 1974, SIGP.

[27]  D. J. Cook,et al.  4th international conference on software engineering , 1979 .

[28]  Ben Shneiderman,et al.  Program indentation and comprehensibility , 1983, CACM.

[29]  S. Sitharama Iyengar,et al.  An Experimental Study of the Logical Complexity of Data Structures , 1985 .

[30]  Victor R. Basili,et al.  An Empirical Study of a Syntactic Complexity Family , 1983, IEEE Transactions on Software Engineering.

[31]  Sandra D. Hartman A counting tool for RPG , 1982, SCORE '82.

[32]  K. Vairavan,et al.  An Experimental Study of Software Metrics for Real-Time Software , 1985, IEEE Transactions on Software Engineering.

[33]  Victor R. Basili,et al.  Experimentation in software engineering , 1986, IEEE Transactions on Software Engineering.

[34]  Tom DeMarco,et al.  Controlling Software Projects , 1982 .

[35]  Vianney Côté,et al.  Software metrics: An overview of recent results , 1988, J. Syst. Softw..

[36]  Johnette Hassell,et al.  A proposal for a measure of program understanding , 1983, SIGCSE '83.

[37]  John Stephen Davis,et al.  A Study of the Applicability of Complexity Measures , 1988, IEEE Trans. Software Eng..

[38]  Victor R. Basili,et al.  Evaluating and comparing software metrics in the software engineering laboratory , 1981, SIGMETRICS Perform. Evaluation Rev..

[39]  Martin Shepperd,et al.  An evaluation of software product metrics , 1988 .

[40]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[41]  Ben Shneiderman,et al.  An Experiment Using Memorization/Reconstruction as a Measure of Programmer Ability , 1980, Int. J. Man Mach. Stud..

[42]  J. C. Rault An approach towards reliable software , 1979, ICSE 1979.

[43]  Larry Weissman,et al.  Psychological complexity of computer programs , 1973, SIGP.

[44]  Frederick P. Brooks,et al.  No Silver Bullet: Essence and Accidents of Software Engineering , 1987 .

[45]  G. C. Moss Developing a usable metric toolkit for a commercial environment , 1988 .

[46]  M. Shepperd,et al.  A critique of cyclomatic complexity as a software metric , 1988, Softw. Eng. J..

[47]  Bill Curtis,et al.  Measuring the Psychological Complexity of Software Maintenance Tasks with the Halstead and McCabe Metrics , 1979, IEEE Transactions on Software Engineering.

[48]  Wei-Tek Tsai,et al.  A tool for discriminant analysis and classification of software metrics , 1987 .

[49]  E. GaffneyJ.,et al.  Software Function, Source Lines of Code, and Development Effort Prediction , 1983 .

[50]  Martin R. Woodward The Application of Halstead's software science. Theory to Algol 68 programs , 1984, Softw. Pract. Exp..

[51]  Ronald D. Gordon,et al.  Measuring Improvements in Program Clarity , 1979, IEEE Transactions on Software Engineering.

[52]  Linda M. Ottenstein Predicting numbers of errors using software science , 1981, SIGMETRICS Perform. Evaluation Rev..

[53]  Tom DeMarco An algorithm for sizing software products , 1984, PERV.

[54]  Tom Love An experimental investigation of the effect of program structure on program understanding , 1977 .

[55]  D. C. Ince,et al.  System design metrics: a review and perspective , 1988 .

[56]  Stephen S. Yau,et al.  Some Stability Measures for Software Maintenance , 1979, IEEE Transactions on Software Engineering.

[57]  Scott N. Woodfield,et al.  The effect of modularization and comments on program comprehension , 1981, ICSE '81.

[58]  W. M. Evangelist Software complexity metric sensitivity to program structuring rules , 1983, J. Syst. Softw..

[59]  D. Potier,et al.  Experiments with computer software complexity and reliability , 1982, ICSE '82.

[60]  James A. Senn,et al.  System structure and software maintenance performance , 1989, Commun. ACM.

[61]  James L. Elshoff,et al.  An investigation into the effects of the counting method used on software science measurements , 1978, SIGP.

[62]  John S. Davis,et al.  Chunks: A basis for complexity measurement , 1984, Inf. Process. Manag..

[63]  Victor R. Basili,et al.  Evaluating Software Development by Analysis of Changes: Some Data from the Software Engineering Laboratory , 1985, IEEE Transactions on Software Engineering.

[64]  B. Curtis,et al.  Software Metrics: Guest Editor's Introduction , 1983, IEEE Trans. Software Eng..