Learning Loops: A Replication Study Illuminates Impact of HS Courses

A recent study about the effectiveness of subgoal labeling in an introductory computer science programming course both supported previous research and produced some puzzling results. In this study, we replicate the experiment with a different student population to determine if the results are repeatable. We also gave the experimental task to students in a follow-on course to explore if they had indeed mastered the programming concept. We found that the previous puzzling results were repeated. In addition, for the novice programmers, we found a statistically significant difference in performance based on whether the student had previous programming courses in high school. However, this performance difference disappears in a follow-on course after all students have taken an introductory computer science programming course. The results of this study have implications for how quickly students are evaluated for mastery of knowledge and how we group students in introductory programming courses.

[1]  Elliot Soloway,et al.  PROUST: Knowledge-Based Program Understanding , 1984, IEEE Transactions on Software Engineering.

[2]  Roland Brünken,et al.  Cognitive Load Theory: THEORY , 2010 .

[3]  Fred Paas,et al.  Cognitive load, cues, and task selection in learning probability calculus , 2015 .

[4]  Sharon J. Derry,et al.  Computer-Based Examples Designed to Encourage Optimal Example Processing: A Study Examining the Impact of Sequentially Presented, Subgoal-Oriented Worked Examples 1 , 2000 .

[5]  Mark G. Simkin,et al.  What best predicts computer proficiency? , 1989, CACM.

[6]  Janet Rountree,et al.  Interacting factors that predict success and failure in a CS1 course , 2004, ITiCSE-WGR '04.

[7]  Kate Ehrlich,et al.  Empirical Studies of Programming Knowledge , 1984, IEEE Transactions on Software Engineering.

[8]  Beth Simon,et al.  Evaluating a new exam question: Parsons problems , 2008, ICER '08.

[9]  Elsa Eiriksdottir,et al.  Procedural Instructions, Principles, and Examples , 2011, Hum. Factors.

[10]  R. Catrambone.  Improving examples to improve transfer to novel problems , 1994, Memory & cognition.

[11]  Patricia Haden,et al.  Parson's programming puzzles: a fun and effective learning tool for first programming courses , 2006 .

[12]  T. Gog,et al.  Development of an instrument for measuring different types of cognitive load , 2013, Behavior Research Methods.

[13]  R. Atkinson.  Optimizing learning from examples using animated pedagogical agents. , 2002 .

[14]  R. Catrambone.  The subgoal learning model: Creating better examples so that students can solve novel problems. , 1998 .

[15]  G HrynickTobias.  Undergraduate Course Descriptions , 2015 .

[16]  John Sweller,et al.  Cognitive Load Theory , 2020, Encyclopedia of Education and Information Technologies.

[17]  Susan Palmiter,et al.  Animated Demonstrations for Learning Procedural Computer-Based Tasks , 1993, Hum. Comput. Interact..

[18]  S. Derry,et al.  Learning from Examples: Instructional Principles from the Worked Examples Research , 2000 .

[19]  Mark Guzdial,et al.  Subgoal-labeled instructional material improves performance and transfer in learning to develop mobile applications , 2012, ICER '12.

[20]  R. Catrambone.  Generalizing Solution Procedures Learned From Examples , 1996 .

[21]  Raymond Lister,et al.  Relationships between reading, tracing and writing skills in introductory programming , 2008, ICER '08.

[22]  Frederick W. B. Li,et al.  No tests required: comparing traditional and dynamic predictors of programming success , 2014, SIGCSE.

[23]  Richard Catrambone,et al.  Improving problem solving performance in computer-based learning environments through subgoal labels , 2014, L@S.

[24]  James L. Silver,et al.  Predicting success in a first programming course , 1982, SIGCSE '82.

[25]  Ronan G. Reilly,et al.  Programming: factors that influence success , 2005, SIGCSE '05.

[26]  Jens Bennedsen,et al.  An investigation of potential success factors for an introductory model-driven programming course , 2005, ICER '05.

[27]  Michael Johnstone,et al.  The Significance of Learning Style with Respect to Achievement in First Year Programming Students , 2010, 2010 21st Australian Software Engineering Conference.

[28]  Slava Kalyuga.  Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction , 2007 .

[29]  Wied Ruijssenaars,et al.  Encyclopedia of the Sciences of Learning , 2012 .

[30]  Brenda Cantwell Wilson,et al.  Contributing to success in an introductory computer science course: a study of twelve factors , 2001, SIGCSE '01.

[31]  Mark Guzdial,et al.  Subgoals, Context, and Worked Examples in Learning Computing Problem Solving , 2015, ICER.

[32]  Garry L. White,et al.  An Empirical Investigation of the Relationship between Success in Mathematics and Visual Programming Courses , 2003, J. Inf. Syst. Educ..

[33]  Dianne Hagan,et al.  Does it help to have some programming experience before beginning a computing degree program? , 2000, ITiCSE '00.

[34]  J. Sweller,et al.  Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions , 2005 .

[35]  Matthew W. Lewis,et al.  Self-Explonations: How Students Study and Use Examples in Learning to Solve Problems , 1989, Cogn. Sci..

[36]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[37]  R. Bjork.  Memory and metamemory considerations in the training of human beings. , 1994 .

[38]  Alexander Renkl,et al.  Learning From Examples: Fostering Self-Explanations in Computer-Based Learning Environments , 2002, Interact. Learn. Environ..

[39]  Susan Wiedenbeck,et al.  Factors affecting the success of non-majors in learning to program , 2005, ICER '05.

[40]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[41]  Richard Catrambone,et al.  Aiding Transfer in Statistics: Examining the Use of Conceptually Oriented Equations and Elaborations during Subgoal Learning. , 2003 .

[42]  Philip R. Ventura,et al.  Identifying predictors of success for an objects-first CS1 , 2005, Comput. Sci. Educ..

[43]  Margaret Hamilton,et al.  Predictors of success in a first programming course , 2006 .

[44]  J. Sweller.  Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load , 2010 .