The 28:1 Grant/Sackman legend is misleading, or: How large is interpersonal variation really?

How long do different programmers take to solve the same task? In 1967, Grant and Sackman published their now famous number of 28:1 interpersonal performance differences, which is both incorrect and misleading. This report presents the analysis of a much larger dataset of software engineering work time data with respect to the same question. It corrects the false 28:1 value, proposes more appropriate metrics, presents the results for the larger dataset, and presents results of several further analyses: distribution shapes, effect sizes, and the performance of various significance tests.

[1]  M. Graffar.  [Modern epidemiology]. , 1971, Bruxelles medical.

[2]  Michelle Cartwright,et al.  An empirical view of inheritance , 1998, Inf. Softw. Technol..

[3]  Lutz Prechelt,et al.  Replication of the first controlled experiment on the usefulness of design patterns: Detailed description and evaluation , 1997 .

[4]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[5]  Lutz Prechelt.  Accelerating Learning from Experience: Avoiding Defects Faster , 2001, IEEE Softw..

[6]  B. Curtis,et al.  Substantiating programmer variability , 1981, Proceedings of the IEEE.

[7]  Eugene Miya,et al.  On "Software engineering" , 1985, SOEN.

[8]  L. Prechelt,et al.  A Controlled Experiment on Inheritance Depth as a Cost Factor for Maintenance , 1999 .

[9]  T.E. Dickey,et al.  Programmer variability , 1981, Proceedings of the IEEE.

[10]  Lutz Prechelt,et al.  A controlled experiment on the effects of PSP training: Detailed description and evaluation , 1999 .

[11]  John William Daly,et al.  Replication and a multi-method approach to empirical software engineering research , 1996 .

[12]  H. Sackman,et al.  An Exploratory Investigation of Programmer Performance Under On-Line and Off-Line Conditions , 1967 .

[13]  Walter F. Tichy,et al.  A Controlled Experiment in Maintenance Comparing Design Patterns to Simpler Solutions , 2001, IEEE Trans. Software Eng..

[14]  Walter F. Tichy,et al.  A Controlled Experiment to Assess the Benefits of Procedure Argument Type Checking , 1998, IEEE Trans. Software Eng..

[15]  Walter F. Tichy,et al.  A Controlled Experiment Measuring the Effect of Procedure Argument Type Checking on Programmer Productivity , 1996 .

[16]  E. E. Grant,et al.  Exploratory experimental studies comparing online and offline programming performance , 1968, CACM.

[17]  Lutz Prechelt.  An experiment on the usefulness of design patterns: Detailed description and evaluation , 1997 .

[18]  Michael Philippsen,et al.  Two controlled experiments assessing the usefulness of design pattern information during program maintenance [online] , 1998 .

[19]  Tom DeMarco,et al.  Programmer performance and the effects of the workplace , 1985, ICSE '85.