Reading behavior and the effect of embedded selfies in role-playing picture e-books: An eye-tracking investigation

Abstract Digital and interactive media platforms, such as e-books, are becoming important tools in reading and education. In particular, picture e-books can embed multimedia effects such as sound, animation or personalized images, with potential benefits for learning and engagement. However, little is known about how such e-books are read, and most designs remain untested. In this study, an innovative type of role-playing picture e-book entitled “The Prank in the Forest” was designed which provides three different role-playing approaches: emotive selfies that allow the reader to appear as one of the main characters and were varied according to the story development, a fixed selfie that was the same on each page, or no selfie. Sixty-five students were randomly assigned to the emotive selfie, fixed selfie, and no selfie groups. To understand students’ online reading processes, their eye movements were tracked. The results showed that the emotive selfies attracted attention to the main character and also promoted scanning between text and pictures, indicating a better integration of the written and pictorial information. The selfie design led to distinct scanpaths, and this was particularly true when emotive selfies were embedded which responded to the context. Self-report questionnaires of reading motivation and engagement demonstrated that this condition was also the most engaging design for readers. We conclude that emotive selfies can boost reading engagement since they encourage the reader to observe the development of the story from the role he/she chose to play, and we discuss how embodying the reader within the graphics can enhance role-play activities with the potential for improved educational outcomes. We suggest this emotive selfie role-playing design could be applied in e-book user interfaces to create more interaction and personal meaning for the readers.