EMIP: The eye movements in programming dataset

Abstract A large dataset that contains the eye movements of N=216 programmers of different experience levels captured during two code comprehension tasks is presented. Data are grouped in terms of programming expertise (from none to high) and other demographic descriptors. Data were collected through an international collaborative effort that involved eleven research teams across eight countries on four continents. The same eye tracking apparatus and software was used for the data collection. The Eye Movements in Programming (EMIP) dataset is freely available for download. The varied metadata in the EMIP dataset provides fertile ground for the analysis of gaze behavior and may be used to make novel insights about code comprehension.

[1]  Umesh Rajashekar,et al.  DOVES: a database of visual eye movements. , 2009, Spatial vision.

[2]  Alexander Pollatsek,et al.  Eye Movements During Reading , 2008 .

[3]  Andrew A. Berlin,et al.  Compiling scientific code using partial evaluation , 1990, Computer.

[4]  Agostino Gibaldi,et al.  Evaluation of the Tobii EyeX Eye tracking controller and Matlab toolkit for research , 2016, Behavior Research Methods.

[5]  Yann-Gaël Guéhéneuc,et al.  Eye-Tracking Metrics in Software Engineering , 2015, 2015 Asia-Pacific Software Engineering Conference (APSEC).

[6]  Peter C.-H. Cheng,et al.  A Survey on the Usage of Eye-Tracking in Computer Programming , 2018, ACM Comput. Surv..

[7]  Jonathan I. Maletic,et al.  An Eye Tracking Study on camelCase and under_score Identifier Styles , 2010, 2010 IEEE 18th International Conference on Program Comprehension.

[8]  Kristien Ooms,et al.  Accuracy and precision of fixation locations recorded with the low-cost Eye Tribe tracker in different experimental set- ups , 2015 .

[9]  Andrew Begel,et al.  Using psycho-physiological measures to assess task difficulty in software development , 2014, ICSE.

[10]  Matthias Bethge,et al.  DeepGaze II: Reading fixations from deep features trained on object recognition , 2016, ArXiv.

[11]  Manuela Chessa,et al.  The Perspective Geometry of the Eye: Toward Image-Based Eye-Tracking , 2012 .

[12]  Andrew Begel,et al.  Eye tracking in computing education , 2014, ICER '14.

[13]  Martha E. Crosby,et al.  How do we read algorithms? A case study , 1990, Computer.

[14]  Andrew Begel,et al.  Eye Movements in Code Reading: Relaxing the Linear Order , 2015, 2015 IEEE 23rd International Conference on Program Comprehension.

[15]  Agostino Gibaldi,et al.  The saccade main sequence revised: A fast and repeatable tool for oculomotor analysis , 2020, Behavior research methods.

[16]  Alan C. Bovik,et al.  Foveated Analysis and Selection of Visual Fixations in Natural Scenes , 2006, 2006 International Conference on Image Processing.

[17]  Akito Monden,et al.  Analyzing individual performance of source code review using reviewers' eye movement , 2006, ETRA.

[18]  Dawn J. Lawrie,et al.  The impact of identifier style on effort and comprehension , 2012, Empirical Software Engineering.

[19]  Yann-Gaël Guéhéneuc,et al.  A practical guide on conducting eye tracking studies in software engineering , 2020, Empirical Software Engineering.

[20]  Markku Tukiainen,et al.  An eye-tracking methodology for characterizing program comprehension processes , 2006, ETRA.

[21]  Zahra Hakimi,et al.  SET: a pupil detection method using sinusoidal approximation , 2015, Front. Neuroeng..

[22]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[23]  Yann-Gaël Guéhéneuc,et al.  A systematic literature review on the usage of eye-tracking in software engineering , 2015, Inf. Softw. Technol..

[24]  Bonita Sharif,et al.  An eye-tracking study assessing the comprehension of c++ and Python source code , 2014, ETRA.

[25]  Bonita Sharif,et al.  Towards automating fixation correction for source code , 2016, ETRA.

[26]  Andrew Begel,et al.  Improving Communication Between Pair Programmers Using Shared Gaze Awareness , 2017, CHI.

[27]  Roman Bednarik,et al.  The Role of Extrafoveal Vision in Source Code Comprehension , 2017, Perception.