The Influence of Different AOI Models in Source Code Comprehension Analysis

In source code comprehension, researchers frequently analyze eye movement data in order to find patterns to test hypotheses. For such an analysis, fixations are often assigned to objects of a stimulus, like tokens, code lines or code regions, through areas of interest (AOI). There are multiple ways to define an AOI which can have a broad impact on the analysis of the eye movement data. In this paper, we analyzed the impact of choosing two different AOI models, both as extreme points from a methodical point of view. We propose one AOI model and compare it to the AOI model from the EMIP dataset. Within the data, we found indications, that one AOI model captures less AOI transitions compared to the other model. A qualitative investigation showed, that some of these AOI transitions can be important for understanding the viewing and comprehension strategy of the participant. In conclusion, we argue, that every researcher should report the chosen AOI model and the particular AOI definitions used in a study, to understand their research. Additionally, we suggest a simple algorithm to test, which AOI model configuration will capture a majority of data points.