Objective: To ascertain whether programmers gain more knowledge about an unfamiliar program by enhancing the code or documenting the code. The context of this work was investigating whether maintenance programmers faced with an unfamiliar system should start by actively working on the system or spend time passively exploring the system before attempting to make changes.
Method:We designed a laboratory experiment where subjects initially either enhanced or documented a program and then we measured how they performed when carrying out a further task on the given code. Our hypothesis was that programmers would gain more knowledge performing one of the two tasks. The experiment was repeated three times with different groups of students, all at the same stage of their education.
Results: There was no significant difference between the performance of the two groups who had performed different initial tasks. However, there was a strong correlation between performance in the measured task and the students' programming ability, as measured by a previous academic assessment. As not all subjects completed the measured task within the given time, we needed to use Kaplan-Meier survival curves and the Cox Proportional Hazard Model to analyse our data. Detailed inspection of the code produced during the experiment revealed some interesting qualitative results.
Conclusions: We were unable to show a significant difference between the value of enhancing or documenting code as a way of gaining knowledge about unfamiliar programs. In the context of software maintenance this means that there is no advantage in spending unproductive time documenting code to gain knowledge.
[1]
Adele E. Howe,et al.
Program understanding behaviour during enhancement of large-scale software
,
1997,
J. Softw. Maintenance Res. Pract..
[2]
Susan Elliott Sim,et al.
The ramp-up problem in software projects: a case study of how software immigrants naturalize
,
1998,
Proceedings of the 20th International Conference on Software Engineering.
[3]
Andy Laws,et al.
Training for software maintenance
,
1998
.
[4]
Janice Singer.
Practices of software maintenance
,
1998,
Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272).
[5]
Laurence L. George,et al.
The Statistical Analysis of Failure Time Data
,
2003,
Technometrics.
[6]
A. Marie Vans,et al.
Program understanding behavior during debugging of large scale software
,
1997,
ESP '97.
[7]
Thomas M. Pigoski,et al.
Software maintenance training: Transition experiences
,
1993,
1993 Conference on Software Maintenance.
[8]
Anneliese Amschler Andrews,et al.
Comprehension processes during large scale maintenance
,
1994,
Proceedings of 16th International Conference on Software Engineering.
[9]
Bill Curtis,et al.
By the way, did anyone study any real programmers?
,
1986
.
[10]
Hans Christian Benestad,et al.
How to Recruit Professionals as Subjects in Software Engineering Experiments
,
2005
.
[11]
Marc Roper,et al.
A Comparative Evaluation of Program Comprehension Measures
,
2000
.