Eye Tracking Indicators of Reading Approaches in Text-Picture Comprehension

Despite numerous researches on reading as well as multimedia comprehension, reading approaches in text-picture comprehension have become a focus of research only rarely. The current experiment aims at exploring text-picture comprehension in different reading approaches with item solving. In a within subjects design using eye tracking, seventeen secondary school students processed our blended text and picture materials in three different ways. (1) Unguided processing with text and picture and without the question. (2) Information gathering to answer the question after the prior experience with text and pictures. (3) Comprehending text and pictures to solve the task with the prior information of the question. Eye tracking data showed that text and picture play different roles in comprehension in different reading approaches. The data suggest that (1) text plays the main role to construct the mental model in unguided spontaneous processing of text and picture. (2) Students seem to mainly rely on the picture as an external representation when trying to answer a question after the prior experience with the material. (3)Text and picture are both used heavily when students work out an answer with the prior experience with the question presented. The text likely plays a major part in guiding the processing of meaning, whereas the picture is used as an external representation for information retrieval. Our work provides a first step towards an Item Solving Model in Text-Picture Comprehension. It also provides pedagogical implications for learning in secondary school.