Measuring the difficulty of code comprehension tasks using software metrics

In this paper we report on an empirical study into the use of software metrics as a way of estimating the difficulty of code comprehension tasks. Our results indicate that software metrics can provide useful information about the difficulties inherent in code tracing in first year programming assessment. We conclude that software metrics may be a useful tool to assist in the design and selection of questions when setting an examination.

[1]  T. Dean Hendrix,et al.  The role of software measures and metrics in studies of program comprehension , 1999, ACM-SE 37.

[2]  Juergen Rilling,et al.  A cognitive complexity metric based on category learning , 2003, The Second IEEE International Conference on Cognitive Informatics, 2003. Proceedings..

[3]  Mark Guzdial,et al.  Assessing fundamental introductory computing concept knowledge in a language independent manner , 2010 .

[4]  Margaret Hamilton,et al.  A taxonomic study of novice programming summative assessment , 2009, ACE '09.

[5]  Mordechai Ben-Ari,et al.  Learning computer science concepts with scratch , 2010, ICER '10.

[6]  Ursula Fuller,et al.  Developing a computer science-specific learning taxonomy , 2007, ITiCSE-WGR '07.

[7]  Walter Kintsch,et al.  Toward a model of text comprehension and production. , 1978 .

[8]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .

[9]  Tony Clear,et al.  Salient elements in novice solutions to code writing problems , 2011, ACE 2011.

[10]  Katrin Becker,et al.  Measuring effectiveness of constructivist and behaviourist assignments in CS102 , 2003, ITiCSE '03.

[11]  Maurice H. Halstead,et al.  Elements of software science (Operating and programming systems series) , 1977 .

[12]  Tony Clear,et al.  An Australasian study of reading and comprehension skills in novice programmers, using the bloom and SOLO taxonomies , 2006 .

[13]  Angela Carbone,et al.  Reliably classifying novice programmer exam responses using the SOLO taxonomy , 2008 .

[14]  Raymond Lister,et al.  Not seeing the forest for the trees: novice programmers and the SOLO taxonomy , 2006, ITICSE '06.

[15]  Sue Fitzgerald,et al.  'Explain in plain English' questions: implications for teaching , 2012, SIGCSE '12.

[16]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[17]  Angela Carbone,et al.  Going SOLO to assess novice programmers , 2008, ITiCSE.

[18]  Kevin F. Collis,et al.  Evaluating the Quality of Learning: The SOLO Taxonomy , 1977 .

[19]  Anne Venables,et al.  A closer look at tracing, explaining and code writing skills in the novice programmer , 2009, ICER '09.

[20]  Tuomas Klemola.  Software comprehension : theory and metrics , 1998 .