Classification of strategies for solving programming problems using AoI sequence analysis

This eye tracking study examines participants' visual attention when solving algorithmic problems in the form of programming problems. The stimuli consisted of a problem statement, example output, and a set of multiple-choice questions regarding variables, data types, and operations needed to solve the programming problems. We recorded eye movements of students and performed an Area of Interest (Aol) sequence analysis to identify reading strategies in terms of participants' performance and visual effort. Using classical eye tracking metrics and a visual Aol sequence analysis we identified two main groups of participants---effective and ineffective problem solvers. This indicates that diversity of participants' mental schemas leads to a difference in their performance. Therefore, identifying how participants' reading behavior varies at a finer level of granularity warrants further investigation.

[1]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.

[2]  Peter C.-H. Cheng,et al.  A Survey on the Usage of Eye-Tracking in Computer Programming , 2018, ACM Comput. Surv..

[3]  Michael Burch,et al.  Visualization of Eye Tracking Data: A Taxonomy and Survey , 2017, Comput. Graph. Forum.

[4]  Kate Ehrlich,et al.  Empirical Studies of Programming Knowledge , 1984, IEEE Transactions on Software Engineering.

[5]  Françoise Détienne,et al.  An Empirically-Derived Control Structure for the Process of Program Understanding , 1990, Int. J. Man Mach. Stud..

[6]  Kshitij Sharma,et al.  Evidence for Programming Strategies in University Coding Exercises , 2018, EC-TEL.

[7]  Jana Holsanova,et al.  Reading or scanning? A study of newspaper and net paper reading. , 2003 .

[8]  Elliot Soloway,et al.  Learning to program = learning to construct mechanisms and explanations , 1986, CACM.

[9]  Robert S. Rist Schema Creation in Programming , 1989, Cogn. Sci..

[10]  Magdalena Andrzejewska,et al.  Analysis of the Strategy Used to Solve Algorithmic Problem: A Case Study Based on Eye Tracking Research , 2017 .

[11]  Elliot Soloway,et al.  A Goal/Plan Analysis of Buggy Pascal Programs , 1985, Hum. Comput. Interact..

[12]  Joseph H. Goldberg,et al.  Computer interface evaluation using eye movements: methods and constructs , 1999 .

[13]  Yann-Gaël Guéhéneuc,et al.  Eye-Tracking Metrics in Software Engineering , 2015, 2015 Asia-Pacific Software Engineering Conference (APSEC).

[14]  Daniel C. Richardson,et al.  Looking To Understand: The Coupling Between Speakers' and Listeners' Eye Movements and Its Relationship to Discourse Comprehension , 2005, Cogn. Sci..

[15]  Anabela Gomes,et al.  Learning to program - difficulties and solutions , 2007 .

[16]  Silvia Wen-Yu Lee,et al.  A review of using eye-tracking technology in exploring learning from 2000 to 2012 , 2013 .

[17]  Juha Sorva,et al.  Towards an Analysis of Program Complexity From a Cognitive Perspective , 2018, ICER.

[18]  Kate Ehrlich,et al.  Cognitive strategies and looping constructs: an empirical study , 1983, CACM.

[19]  Tony Clear,et al.  An introduction to program comprehension for computer science educators , 2010, ITiCSE-WGR '10.

[20]  Martha E. Crosby,et al.  How do we read algorithms? A case study , 1990, Computer.