Queue-based cost evaluation of mental simulation process in program comprehension

We present a method to estimate the cost of mental (hand) simulation of programs. In mental simulation, human short term memory is extensively used to recall and memorize values of variables. When the simulation reaches a variable reference, the simulation can be performed easily if the value is still remembered. However, if not, we have to backtrack the simulation until the value is obtained, which is time consuming. Taking the above observation into consideration, we first present a model, called virtual mental simulation model (VMSM), which exploits a queue representing short term memory. The VMSM takes one of the abstract processes recall or backtrack, depending on whether the variable is currently stored in the queue or not. Then, applying cost functions to the VMSM, we derive four dynamic metrics reflecting the cost of mental simulation. In our empirical study, the proposed VMSM metrics reveal that the backtrack process for nonconstant variables gives a significant impact on the cost of mental simulation. Since the proposed method can be fully automated, it can provide a practical means to estimate the cost of mental simulation, which can be also used as a program comprehension measure.

[1]  David B. Bisant,et al.  Cognitive processes in software fault detection: A review and synthesis , 1993, Int. J. Hum. Comput. Interact..

[2]  I. Burnstein,et al.  A role for chunking and fuzzy reasoning in a program comprehension and debugging tool , 1997, Proceedings Ninth IEEE International Conference on Tools with Artificial Intelligence.

[3]  Norman E. Fenton,et al.  Measurement : A Necessary Scientific Basis , 2004 .

[4]  Banshi Dhar Chaudhary,et al.  Meaningfulness as a factor of program complexity , 1980, ACM '80.

[5]  Christian S. Collberg,et al.  Watermarking, Tamper-Proofing, and Obfuscation-Tools for Software Protection , 2002, IEEE Trans. Software Eng..

[6]  John S. Davis,et al.  Chunks: A basis for complexity measurement , 1984, Inf. Process. Manag..

[7]  Brian Henderson-Sellers,et al.  A conceptual model of cognitive complexity of elements of the programming process , 1995, Inf. Softw. Technol..

[8]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[9]  Egon Berghout,et al.  Proceedings. Ninth International Software Metrics Symposium , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[10]  Marc Roper,et al.  The role of comprehension in software inspection , 2000, J. Syst. Softw..

[11]  Marc Roper,et al.  A Comparative Evaluation of Program Comprehension Measures , 2000 .

[12]  N. Cowan The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[13]  Hausi A. Müller,et al.  Cognitive design elements to support the construction of a mental model during software exploration , 1999, J. Syst. Softw..

[14]  Hausi A. Müller,et al.  Cognitive design elements to support the construction of a mental model during software visualization , 1997, Proceedings Fifth International Workshop on Program Comprehension. IWPC'97.

[15]  James R. Larus,et al.  Optimally profiling and tracing programs , 1992, POPL '92.

[16]  Chris R. Douce Long Term Comprehension of Software Systems: A Methodology for Study , 2001, PPIG.