On the Frequency of Words Used in Answers to Explain in Plain English Questions by Novice Programmers

Most previous research studies using Explain in Plain English questions have focussed on categorising the answers of novice programmers according to the SOLO taxonomy, and/or the relationship between explaining code and writing code. In this paper, we study the words used in the explanations of novice programmers. Our data is from twelve Explain in plain English questions presented to over three hundred students in an exam at the end of the students' first semester of programming. For each question, we compare the frequency of certain words used in correct answers, between students who scored a perfect twelve on all the Explain in plain English questions and students with lower scores. We report a number of statistically significant differences in word frequency between the students who answered all questions correctly and students who did not. The students who answered all twelve questions correctly tended to be more precise, more comprehensive, and more likely to choose words not explicitly in the code, but instead words that are an abstraction beyond the code.

[1]  Sue Fitzgerald,et al.  Ability to 'explain in plain english' linked to proficiency in computer-based programming , 2012, ICER '12.

[2]  Judy Kay,et al.  On the reliability of classifying programming tasks using a neo-piagetian theory of cognitive development , 2012, ICER '12.

[3]  Angela Carbone,et al.  Going SOLO to assess novice programmers , 2008, SIGCSE 2008.

[4]  Raymond Lister,et al.  Swapping as the Hello World of relational reasoning: replications, reflections and extensions , 2012, ACE 2012.

[5]  Raymond Lister,et al.  Not seeing the forest for the trees: novice programmers and the SOLO taxonomy , 2006, ITICSE '06.

[6]  Raymond Lister,et al.  Early relational reasoning and the novice programmer: swapping as the hello world of relational reasoning , 2011, ACE 2011.

[7]  Raymond Lister,et al.  A qualitative think aloud study of the early neo-piagetian stages of reasoning in novice programmers , 2013, ACE '13.

[8]  Benedict du Boulay,et al.  Some Difficulties of Learning to Program , 1986 .

[9]  Simon,et al.  Explaining program code: giving students the answer helps - but only just , 2011, ICER.

[10]  Kyungil Kim,et al.  Psychological Research using Linguistic Inquiry and Word Count (LIWC) and Korean Linguistic Inquiry and Word Count (KLIWC) Language Analysis Methodologies , 2015 .

[11]  Raymond Lister,et al.  Longitudinal think aloud study of a novice programmer , 2014, ACE.

[12]  Colin J. Fidge,et al.  Further evidence of a relationship between explaining, tracing and writing skills in introductory programming , 2009, ITiCSE.

[13]  Raymond Lister,et al.  Relationships between reading, tracing and writing skills in introductory programming , 2008, ICER '08.

[14]  H. J. Arnold.  Introduction to the Practice of Statistics , 1990 .

[15]  Anne Venables,et al.  A closer look at tracing, explaining and code writing skills in the novice programmer , 2009, ICER '09.

[16]  Kyungil Kim,et al.  How do experts and novices differ? Relation versus attribute and thinking versus feeling in language use. , 2011 .

[17]  J. Pennebaker,et al.  The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods , 2010 .

[18]  Susan Bergin,et al.  Automated assessment in CS1 , 2006 .

[19]  Raymond Lister,et al.  Some empirical results for neo-Piagetian reasoning in novice programmers and the relationship to code explanation questions , 2012, ACE 2012.

[20]  Michelene T. H. Chi,et al.  How knowledge is structured and used by expert and novice children , 1986 .

[21]  Benedict duBoulay,et al.  Some Difficulties of Learning to Program , 1986 .

[22]  Ruven E. Brooks,et al.  Towards a Theory of the Comprehension of Computer Programs , 1983, Int. J. Man Mach. Stud..

[23]  Tony Clear,et al.  An Australasian study of reading and comprehension skills in novice programmers, using the bloom and SOLO taxonomies , 2006 .