Impact of experience on maintenance skills

This study reports results from an empirical study of 54 software maintainers in the software maintenance department of a Norwegian company. The study addresses the relationship between amount of experience and maintenance skills. The findings were, amongst others, as follows. (1) While there may have been a reduction in the frequency of major unexpected problems from tasks solved by very inexperienced to medium experienced maintainers, additional years of general software maintenance experience did not lead to further reduction. More application specific experience, however, further reduced the frequency of major unexpected problems. (2) The most experienced maintainers did not predict maintenance problems better than maintainers with little or medium experience. (3) A simple one-variable model outperformed the maintainers' predictions of maintenance problems, i.e. the average prediction performance of the maintainers seems poor. An important reason for the weak correlation between length of experience and ability to predict maintenance problems may be the lack of meaningful feedback on the predictions.

[1]  B. Efron,et al.  A Leisurely Look at the Bootstrap, the Jackknife, and , 1983 .

[2]  Mark Weiser,et al.  Programming Problem Representation in Novice and Expert Programmers , 1983, Int. J. Man Mach. Stud..

[3]  A. Sockloff,et al.  Statistical power analysis for the behavioral sciences: (revised edition), by Jacob Cohen. New York: Academic Press, 1977, xv + 474 pp., $24.50. , 1978 .

[4]  R. Hogarth,et al.  Confidence in judgment: Persistence of the illusion of validity. , 1978 .

[5]  Susan A. Sherer.  Using risk analysis to manage software maintenance , 1997 .

[6]  Jack B. Soll,et al.  Overconfidence: It Depends on How, What, and Whom You Ask. , 1999, Organizational behavior and human decision processes.

[7]  Terry Connolly,et al.  Decomposed versus holistic estimates of effort required for software writing tasks , 1997 .

[8]  K. A. Ericsson,et al.  Expert and exceptional performance: evidence of maximal adaptation to task constraints. , 1996, Annual review of psychology.

[9]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[10]  B. Fischhoff,et al.  Assessing uncertainty in physical constants , 1986 .

[11]  A. Tversky,et al.  On the psychology of prediction , 1973 .

[12]  H. D. Rombach,et al.  THE EXPERIENCE FACTORY , 1999 .

[13]  Prashant Palvia,et al.  Software maintenance management: Changes in the last decade , 1990, J. Softw. Maintenance Res. Pract..

[14]  Gerd Gigerenzer,et al.  Do Studies of Statistical Power Have an Effect on the Power of Studies? , 2004 .

[15]  Chris F. Kemerer,et al.  An empirical validation of software cost estimation models , 1987, CACM.

[16]  Stuart E. Madnick,et al.  Special Feature: Impact of Schedule Estimation on Software Project Behavior , 1986, IEEE Software.

[17]  Ingunn Myrtveit,et al.  A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models , 1999, IEEE Trans. Software Eng..

[18]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[19]  Baruch Fischhoff,et al.  Calibrating databases , 1986, J. Am. Soc. Inf. Sci..

[20]  B. Brehmer.  In one word: Not from experience. , 1980 .

[21]  Barry G. Silverman,et al.  Software Cost and Productivity Improvements: An Analogical View , 1985, Computer.

[22]  T. Capers Jones,et al.  Estimating software costs , 1998 .

[23]  W. Rosenberger.  Dealing with multiplicities in pharmacoepidemiologic studies , 1996, Pharmacoepidemiology and drug safety.

[24]  Robert J. Crutcher,et al.  The role of deliberate practice in the acquisition of expert performance. , 1993 .

[25]  Magne Jørgensen,et al.  Impact of effort estimates on software project work , 2001, Inf. Softw. Technol..

[26]  Peter Brössler,et al.  Project Experience Database: A Report Based on First Practical Experience , 2000, PROFES.

[27]  A. Tversky,et al.  The weighing of evidence and the determinants of confidence , 1992, Cognitive Psychology.

[28]  M. Doherty,et al.  Effects of cognitive feedback on performance. , 1989 .

[29]  J. Kagan,et al.  Rational choice in an uncertain world , 1988 .

[30]  Nigel Harvey,et al.  Combining forecasts: What information do judges need to outperform the simple average? , 1999 .

[31]  Charles R. Symons,et al.  Software sizing and estimating - Mk II FPA, function point analysis , 1991, Wiley series in software engineering practice.

[32]  Magne Jørgensen,et al.  Empirical studies on effort estimation in software development projects , 2000, IRMA Conference.

[33]  Rupert G. Miller.  Simultaneous Statistical Inference , 1966 .

[34]  J. Reitman,et al.  Knowledge organization and skill differences in computer programmers , 1981, Cognitive Psychology.

[35]  Robert C. Blattberg,et al.  Database Models And Managerial Intuition: 50% Model + 50% Manager , 1990 .

[36]  Girish H. Subramanian,et al.  An Examination of Some Software Development Effort and Productivity Determinants in ICASE Tool Projects , 1996, J. Manag. Inf. Syst..

[37]  Susan A. Sherer.  Using risk analysis to manage software maintenance , 1997, J. Softw. Maintenance Res. Pract..

[38]  Jacob Cohen,et al.  The statistical power of abnormal-social psychological research: a review. , 1962, Journal of abnormal and social psychology.

[39]  Andy Laws,et al.  Training for software maintenance , 1998 .

[40]  Claes Wohlin,et al.  An experimental study of individual subjective effort estimations and combinations of the estimates , 1998, Proceedings of the 20th International Conference on Software Engineering.

[41]  Paul Goodwin,et al.  Correct or combine? Mechanically integrating judgmental forecasts with statistical methods , 2000 .

[42]  Björn Regnell,et al.  Is a Design Rationale Vital when Predicting Change Impact? A Controlled Experiment on Software Architecture Evolution , 2000, PROFES.

[43]  M A Branch,et al.  Software maintenance management , 1986 .

[44]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[45]  David P. Hale,et al.  Towards a model of programmers' cognitive processes in software maintenance: A structural learning theory approach for debugging , 1991, J. Softw. Maintenance Res. Pract..

[46]  Magne Jørgensen,et al.  An empirical study of software maintenance tasks , 1995, J. Softw. Maintenance Res. Pract..

[47]  Jean-Marc Desharnais,et al.  A comparison of software effort estimation techniques: Using function points with neural networks, case-based reasoning and regression models , 1997, J. Syst. Softw..

[48]  A. Marie Vans,et al.  Program Comprehension And Enhancement Of Software , 1998 .

[49]  A. Boonstra.  Information systems and organizational change , 2002 .

[50]  S. Plous.  The psychology of judgment and decision making , 1994 .

[51]  B Lakhanpal,et al.  Understanding the factors influencing the performance of software development groups: An exploratory group-level analysis , 1993, Inf. Softw. Technol..

[52]  Roberta L. Klatzky,et al.  Expertise in a Computer Operating System: Conceptualization and Performance , 1990, Hum. Comput. Interact..

[53]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[54]  David V. Budescu,et al.  Encoding subjective probabilities: A psychological and psychometric review , 1983 .

[55]  Magne Jøgensen,et al.  The quality of questionnaire based software maintenance studies , 1995, SOEN.

[56]  Anneliese Amschler Andrews,et al.  Comprehension processes during large scale maintenance , 1994, Proceedings of 16th International Conference on Software Engineering.