Discriminating Programming Strategies in Scratch: Making the Difference between Novice and Experienced Programmers

Nowadays, block-based programming environments are often used to offer a gentle introduction to learning a programming language. However, an assessment of students' programming skills based on the results of a programming task is not sufficient to determine all areas students are struggling with. We therefore introduce a learning analytics approach of measuring and evaluating the programming sequences of students that program with Scratch 3. With our measurement framework, it is possible to record, store and analyze programming sequences done on a publicly-available, instrumented Scratch 3 environment. Changes in the programming sequence are categorized regarding the used block types and types of program change. We conducted an exploratory programming trial with lower and upper secondary school students to investigate small-scale programming strategies in the recorded programming sequences. Our goals are to identify students in need of support and to identify recurring patterns used by students successful in the trial. Clustering with k-means makes it possible to identify struggling students based on both interacted block types and types of program changes. Recurring patterns in the programming sequences of successful students show that small-scale programming strategies are very diverse.

[1]  Alexander Serebrenik,et al.  How do Scratch Programmers Name Variables and Procedures? , 2017, 2017 IEEE 17th International Working Conference on Source Code Analysis and Manipulation (SCAM).

[2]  Kathryn T. Stolee,et al.  Smells in block-based programming languages , 2016, 2016 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[3]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[4]  Andreas Bollin,et al.  The Bebras Contest in Austria - Do Personality, Self-concept and General Interests Play an Influential Role? , 2018, ISSEP.

[5]  Raymond Lister,et al.  Concrete and other neo-Piagetian forms of reasoning in the novice programmer , 2011, ACE 2011.

[6]  Gregorio Robles,et al.  Automatic detection of bad programming habits in scratch: A preliminary study , 2014, 2014 IEEE Frontiers in Education Conference (FIE) Proceedings.

[7]  Raymond Lister,et al.  Longitudinal think aloud study of a novice programmer , 2014, ACE.

[8]  Diana Franklin,et al.  Hairball: lint-inspired static analysis of scratch projects , 2013, SIGCSE '13.

[9]  Mordechai Ben-Ari,et al.  Habits of programming in scratch , 2011, ITiCSE '11.

[10]  Francisco J. García-Peñalvo,et al.  Clickstream for learning analytics to assess students' behavior with Scratch , 2019, Future Gener. Comput. Syst..

[11]  Andreas Grillenberger,et al.  Real-Time Data Analyses in Secondary Schools Using a Block-Based Programming Language , 2017, ISSEP.

[12]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[13]  Seymour Papert,et al.  Mindstorms: Children, Computers, and Powerful Ideas , 1981 .

[14]  Shuchi Grover,et al.  Measuring Student Learning in Introductory Block-Based Programming: Examining Misconceptions of Loops, Variables, and Boolean Logic , 2017, SIGCSE.

[15]  Michail N. Giannakos,et al.  Discovering children's competences in coding through the analysis of Scratch projects , 2018, 2018 IEEE Global Engineering Education Conference (EDUCON).

[16]  Frederick W. B. Li,et al.  Failure rates in introductory programming revisited , 2014, ITiCSE '14.

[17]  Richard T. White,et al.  Book Review: Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome) , 1983 .

[18]  Felienne Hermans,et al.  How Kids Code and How We Know: An Exploratory Study on the Scratch Repository , 2016, ICER.

[19]  MICHAL ARMONI,et al.  From Scratch to “Real” Programming , 2015, TOCE.

[20]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .

[21]  Adam S. Carter,et al.  IDE-Based Learning Analytics for Computing Education , 2017, ACM Trans. Comput. Educ..

[22]  Marcelo Worsley,et al.  Programming Pluralism: Using Learning Analytics to Detect Patterns in the Learning of Computer Programming , 2014, Journal of the Learning Sciences.

[23]  Raymond Lister,et al.  Toward a Developmental Epistemology of Computer Programming , 2016, WiPSCE.

[24]  Andreas Bollin,et al.  Quantifying Patterns and Programming Strategies in Block-Based Programming Environments , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion).

[25]  Mordechai Ben-Ari,et al.  Learning computer science concepts with scratch , 2010, ICER '10.

[26]  Jürgen Börstler,et al.  Educational Data Mining and Learning Analytics in Programming: Literature Review and Case Studies , 2015, ITiCSE-WGR.

[27]  Kostas Kontogiannis,et al.  Evaluation experiments on the detection of programming patterns using software metrics , 1997, Proceedings of the Fourth Working Conference on Reverse Engineering.

[28]  Gregorio Robles,et al.  Is My Game OK Dr. Scratch?: Exploring Programming and Computational Thinking Development via Metrics in Student-Designed Serious Games for STEM , 2019, IDC.

[29]  Kenneth R. Koedinger,et al.  Learning Curve Analysis for Programming: Which Concepts do Students Struggle With? , 2016, ICER.

[30]  Viera K. Proulx,et al.  Programming patterns and design patterns in the introductory computer science course , 2000, SIGCSE '00.

[31]  Felienne Hermans,et al.  Do code smells hamper novice programming? A controlled experiment on Scratch programs , 2016, 2016 IEEE 24th International Conference on Program Comprehension (ICPC).