On the reliability of classifying programming tasks using a neo-piagetian theory of cognitive development

Recent research has proposed Neo-Piagetian theory as a useful way of describing the cognitive development of novice programmers. Neo-Piagetian theory may also be a useful way to classify materials used in learning and assessment. If Neo-Piagetian coding of learning resources is to be useful then it is important that practitioners can learn it and apply it reliably. We describe the design of an interactive web-based tutorial for Neo-Piagetian categorization of assessment tasks. We also report an evaluation of the tutorial's effectiveness, in which twenty computer science educators participated. The average classification accuracy of the participants on each of the three Neo-Piagetian stages were 85%, 71% and 78%. Participants also rated their agreement with the expert classifications, and indicated high agreement (91%, 83% and 91% across the three Neo-Piagetian stages). Self-rated confidence in applying Neo-Piagetian theory to classifying programming questions before and after the tutorial were 29% and 75% respectively. Our key contribution is the demonstration of the feasibility of the Neo-Piagetian approach to classifying assessment materials, by demonstrating that it is learnable and can be applied reliably by a group of educators. Our tutorial is freely available as a community resource.

[1]  John Hamer,et al.  The PeerWise system of student contributed assessment questions , 2008, ACE '08.

[2]  Sue Fitzgerald,et al.  Ability to 'explain in plain english' linked to proficiency in computer-based programming , 2012, ICER '12.

[3]  Yifat Ben-David Kolikant,et al.  “So my program doesn’t run!” Definition, origins, and practical expressions of students’ (mis)conceptions of correctness , 2008, Comput. Sci. Educ..

[4]  Jeff Kramer,et al.  Is abstraction the key to computing? , 2007, CACM.

[5]  K. H. Stauder,et al.  Psychology of the Child , 1959 .

[6]  Orit Hazzan,et al.  How Students Attempt to Reduce Abstraction in the Learning of Mathematics and in the Learning of Computer Science , 2003, Comput. Sci. Educ..

[7]  Angela Carbone,et al.  Going SOLO to assess novice programmers , 2008, ITiCSE.

[8]  Judy Kay,et al.  An architecture for systematic tracking of skills and competence level progression in computer science , 2011 .

[9]  David Ginat,et al.  Hasty design, futile patching and the elaboration of rigor , 2007, ITiCSE.

[10]  Raymond Lister,et al.  Some empirical results for neo-Piagetian reasoning in novice programmers and the relationship to code explanation questions , 2012, ACE 2012.

[11]  Barry L. Kurtz,et al.  Investigating the relationship between the development of abstract reasoning and performance in an introductory programming class , 1980, SIGCSE '80.

[12]  Roger Azevedo,et al.  Self-efficacy and prior domain knowledge: to what extent does monitoring mediate their relationship with hypermedia learning? , 2009 .

[13]  Sergio Morra,et al.  Cognitive Development : Neo-Piagetian Perspectives , 2012 .

[14]  Judy Kay,et al.  Coming to terms with Bloom: an online tutorial for teachers of programming fundamentals , 2012, ACE 2012.

[15]  Michelene T. H. Chi,et al.  Eliciting Self-Explanations Improves Understanding , 1994, Cogn. Sci..

[16]  Raymond Lister,et al.  Concrete and other neo-Piagetian forms of reasoning in the novice programmer , 2011, ACE 2011.

[17]  Errol Thompson,et al.  Bloom's taxonomy for CS assessment , 2008, ACE '08.