Evaluation of Parsons Problems with Menu-Based Self-Explanation Prompts in a Mobile Python Tutor

The overarching goal of our project is to design effective learning activities for PyKinetic, a smartphone Python tutor. In this paper, we present a study using a variant of Parsons problems we designed for PyKinetic. Parsons problems contain randomized code which needs to be re-ordered to produce the desired effect. In our variant of Parsons problems, students were asked to complete the missing part(s) of some lines of code (LOCs), and rearrange the LOCs to match the problem description. In addition, we added menu-based Self-Explanation (SE) prompts. Students were asked to self-explain concepts related to incomplete LOCs they solved. Our hypotheses were: (H1) PyKinetic would be successful in supporting learning; (H2) menu-based SE prompts would result in further learning benefits; (H3) students with low prior knowledge (LP) would learn more from our Parsons problems in comparison to those with high prior knowledge (HP). We found that the participants’ scores on the post-test improved, thus showing evidence of learning in PyKinetic. The experimental group participants, who had SE prompts, showed improved learning in comparison to the control group. Further analyses revealed that LP students improved more than HP students and the improvement is even more pronounced for LP learners who self-explained. The contributions of our work are a) pedagogically-guided design of Parsons problems with SE prompts used on smartphones, b) showing that our Parsons problems are effective in supporting learning and c) our Parsons problems with SE prompts are especially effective for students with low prior knowledge.

[1]  Richard E. Mayer,et al.  Adding self-explanation prompts to an educational computer game , 2014, Comput. Hum. Behav..

[2]  Chung-Yuan Hsu,et al.  Facilitating Third Graders’ Acquisition of Scientific Concepts through Digital Game-Based Learning: The Effects of Self-Explanation Principles , 2012 .

[3]  Ivica Boticki,et al.  Teaching and learning computer science sorting algorithms with mobile devices: A case study , 2013, Comput. Appl. Eng. Educ..

[4]  Petri Ihantola,et al.  Two-Dimensional Parson's Puzzles: The Concept, Tools, and First Observations , 2011, J. Inf. Technol. Educ. Innov. Pract..

[5]  Amruth N. Kumar,et al.  Epplets: A Tool for Solving Parsons Puzzles , 2018, SIGCSE.

[6]  Beth Simon,et al.  Evaluating a new exam question: Parsons problems , 2008, ICER '08.

[7]  Antonija Mitrovic,et al.  Learning with intelligent tutors and worked examples: selecting learning activities adaptively leads to better learning outcomes than a fixed curriculum , 2016, User Modeling and User-Adapted Interaction.

[8]  M. Ferguson-Hessler,et al.  Studying Physics Texts: Differences in Study Processes Between Good and Poor Performers , 1990 .

[9]  Jason Chen,et al.  Distractors in Parsons Problems Decrease Learning Efficiency for Young Novice Programmers , 2016, ICER.

[10]  Cheryl I. Johnson,et al.  Applying the self-explanation principle to multimedia learning in a computer-based game-like environment , 2010, Comput. Hum. Behav..

[11]  Michelene T. H. Chi,et al.  Eliciting Self-Explanations Improves Understanding , 1994, Cogn. Sci..

[12]  Martin Ebner,et al.  It's in your pocket: A MOOC about programming for kids and the role of OER in teaching and learning contexts , 2018 .

[13]  M. W. van Someren,et al.  The think aloud method: a practical approach to modelling cognitive processes , 1994 .

[14]  Roya Hosseini,et al.  Program Construction Examples in Computer Science Education: From Static Text to Adaptive and Engaging Learning Technology , 2018 .

[15]  Antonija Mitrovic,et al.  Learning with Engaging Activities via a Mobile Python Tutor , 2017, AIED.

[16]  Ruth Wylie,et al.  The Self-Explanation Principle in Multimedia Learning , 2014 .

[17]  Antonija Mitrovic,et al.  Investigating the effects of learning activities in a mobile Python tutor for targeting multiple coding skills , 2018, Res. Pract. Technol. Enhanc. Learn..

[18]  Dean Mohamedally,et al.  RoboRun: A gamification approach to control flow learning for young students with TouchDevelop , 2013, ArXiv.

[19]  Vincent Aleven,et al.  An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor , 2002, Cogn. Sci..

[20]  Michelene T. H. Chi,et al.  Self-Explanations: How Students Study and Use Examples in Learning To Solve Problems. Technical Report No. 9. , 1987 .

[21]  M. Chi,et al.  The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes , 2014 .

[22]  Vincent Aleven,et al.  Evaluating the Effectiveness of a Tutorial Dialogue System for Self-Explanation , 2004, Intelligent Tutoring Systems.

[23]  Petri Ihantola,et al.  A mobile learning application for parsons problems with automatic feedback , 2012, Koli Calling.

[24]  Stuart K Garner,et al.  An Exploration of How a Technology-Facilitated Part-Complete Solution Method Supports the Learning of Computer Programming , 2007 .

[25]  Alexander Renkl,et al.  Assisting self-explanation prompts are more effective than open prompts when learning with multiple representations , 2009 .

[26]  Erkki Sutinen,et al.  Design, development, and evaluation of a mobile learning application for computing education , 2017, Education and Information Technologies.

[27]  Leon E. Winslow,et al.  Programming pedagogy—a psychological overview , 1996, SGCS.

[28]  Junfei Zhang,et al.  Design of a Microlecture Mobile Learning System Based on Smartphone and Web Platforms , 2015, IEEE Transactions on Education.

[29]  Lauri Malmi,et al.  How do students solve parsons programming problems?: an analysis of interaction traces , 2012, ICER '12.

[30]  Antonija Mitrovic,et al.  A comparison of different types of learning activities in a mobile Python tutor , 2017 .

[31]  Antonija Mitrovic,et al.  Investigating the Effectiveness of Menu-Based Self-explanation Prompts in a Mobile Python Tutor , 2017, AIED.

[32]  Ton de Jong,et al.  The effects of directive self-explanation prompts to support active processing of multiple representations in a simulation-based learning environment , 2011, J. Comput. Assist. Learn..

[33]  E MayerRichard,et al.  Applying the self-explanation principle to multimedia learning in a computer-based game-like environment , 2010 .

[34]  Raymond Lister,et al.  Relationships between reading, tracing and writing skills in introductory programming , 2008, ICER '08.

[35]  Kyungbin Kwon,et al.  Self-Explanation Prompts on Problem-Solving Performance in an Interactive Learning Environment. , 2011 .

[36]  Barbara Ericson,et al.  Solving parsons problems versus fixing and writing code , 2017, Koli Calling.

[37]  Vincent Aleven,et al.  The Need for Tutorial Dialog to Support Self-Explanation , 2000 .

[38]  Patricia Haden,et al.  Parson's programming puzzles: a fun and effective learning tool for first programming courses , 2006 .

[39]  Petri Ihantola,et al.  How to study programming on mobile touch devices: interactive Python code exercises , 2013, Koli Calling '13.

[40]  Timothy J. Nokes-Malach,et al.  Effectiveness of holistic mental model confrontation in driving conceptual change , 2012 .

[41]  Tobias Lauer,et al.  A study of algorithm animations on mobile devices , 2007, SIGCSE.

[42]  Vincent Aleven,et al.  Successful learning with multiple graphical representations and self-explanation prompts. , 2015 .

[43]  Antonija Mitrovic,et al.  Adaptive Problem Selection in a Mobile Python Tutor , 2018, UMAP.

[44]  Hussein Suleman,et al.  Design and Use of Static Scaffolding Techniques to Support Java Programming on a Mobile Phone , 2016, ITiCSE.

[45]  Robert McCartney,et al.  Naturally occurring data as research instrument: analyzing examination responses to study the novice programmer , 2010, SGCS.

[46]  Roland Brünken,et al.  Cognitive Load Theory: Frontmatter , 2010 .

[47]  Barbara Ericson,et al.  Subgoals Help Students Solve Parsons Problems , 2016, SIGCSE.

[48]  John R. Anderson.  Acquisition of cognitive skill. , 1982 .