An eye-tracking investigation of visual search strategies and test performance of L1 and L2 listening test takers

Through the use of eye-tracking technology and a while-listening performance (WLP) test, this study examined the differences in gaze behaviors and measured listening performances on test items (across various stages of the test) and compared them between native English-speaking (E-L1) and non-native English-speaking (E-L2) candidates. One hundred students from a public university in Singapore participated in the study. A series of Mann-Whitney U tests indicated that E-L1 candidates outperformed E-L2 candidates in the test with higher test scores. Using stringent data processing cutoffs (presence≥80% gaze data) and a series of non-parametric multivariate analyses, the study further found that the dynamicity of gaze behaviors on the test items across various stages of the test was similar between E-L1 and E-L2 candidates. However, there were distinctive differences in gaze behaviors between the two groups. For E-L1 candidates, none of the gaze behaviors on the test items across the different stages of the test predicted their overall listening test scores. In contrast, the overall listening test scores for E-L2 candidates was predicted by the average proportion of time that they had dwelled on the test items while simultaneously answering them and listening to the auditory text. The study is the first to show that keyword matching on the test items during the while-listening stage significantly contributes to WLP test performance for E-L2 candidates. These results suggest that the focal construct of the listening test is confounded by group-specific reading behaviors on the test items. In line with previous research, the use of the WLP test format for assessing second language listening comprehension is not recommended.