Developing Evaluation Metrics for Active Reading Support

Reading academic literature in digital formats is becoming more and more of a normalcy for students, but designers of reading support tools do not share common, metrics for evaluating such tools. This paper introduces our work in developing an evaluation form which we call the aRSX (active Reading Support IndeX). The aRSX-form is a quantitative means for evaluating whether a specific software or hardware tool supports active, academic reading in a way that resonates with personal user experience and learning preferences in other words; whether the tool is practical and pleasant to use for the student who consumes academic literature. The paper presents the first and second iterations of the aRSX evaluation survey based on a preliminary exploratory experiment with 50 university students. The paper also describes how the evaluation form can be developed and used by designers of reading support tools.

[1]  Jennifer Pearson,et al.  Improving Placeholders in Digital Documents , 2008, ECDL.

[2]  James S Wolffsohn,et al.  Digital eye strain: prevalence, measurement and amelioration , 2018, BMJ Open Ophthalmology.

[3]  Katherine A. Brady,et al.  Is Scrolling Disrupting While Reading? , 2018, ICLS.

[4]  Amjad M. Abuloum,et al.  College Students' Usage of and Preferences for Print and Electronic Textbooks , 2019, Int. J. Emerg. Technol. Learn..

[5]  K. Hew,et al.  Students’ perceptions of the usefulness of an E-book with annotative and sharing capabilities as a tool for learning: a case study , 2014 .

[6]  Xue Bai,et al.  The Influence of E-book Format and Reading Device on Users’ Reading Experience: A Case Study of Graduate Students , 2016 .

[7]  Jane Vincent.  Students’ use of paper and pen versus digital media in university environments for writing and reading – a cross-cultural exploration , 2016 .

[8]  David J. Shernoff,et al.  Student engagement in high school classrooms from the perspective of flow theory. , 2003 .

[9]  Serap Kurbanoğlu,et al.  The Academic Reading Format International Study (ARFIS): Investigating Students Around the World , 2016, ECIL.

[10]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[11]  Andrew Hoskins,et al.  How we think: digital media and contemporary technogenesis , 2014 .

[12]  S. Kol,et al.  Enhancing Screen Reading Strategies , 2013 .

[13]  Ling Zhai,et al.  Comparison of reading performance on screen and on paper: A meta-analysis , 2018, Comput. Educ..

[14]  William Haseman,et al.  User attitude as a mediator of learning performance improvement in an interactive multimedia environment: an empirical investigation of the degree of interactivity and learning styles , 2001, Int. J. Hum. Comput. Stud..

[15]  K. Lafreniere,et al.  Social Media, Texting, and Personality: A Test of the Shallowing Hypothesis , 2017 .

[16]  Kara Sage,et al.  Reading from print, computer, and tablet: Equivalent learning in the digital age , 2019, Education and Information Technologies.

[17]  Andrew Dillon,et al.  Reading From Paper Versus Reading From Screen , 1988, Comput. J..

[18]  Rakefet Ackerman,et al.  Don't throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension , 2018, Educational Research Review.

[19]  Pierre-Majorique Léger,et al.  How Learner Experience and Types of Mobile Applications Influence Performance: The Case of Digital Annotation , 2019, Computers in the Schools.

[20]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[21]  Luanne Freund,et al.  The effects of textual environment on reading comprehension: Implications for searching as learning , 2016, J. Inf. Sci..

[22]  Marie-France Plassard,et al.  Functional requirements for bibliographic records : final report , 2013 .

[23]  Joanna Wolfe,et al.  Annotations and the collaborative digital library: Effects of an aligned annotation interface on student argumentation and reading strategies , 2008, Int. J. Comput. Support. Collab. Learn..

[24]  J M Gardiner,et al.  Changes in memory awareness during learning: the acquisition of knowledge by psychology undergraduates. , 1997, Journal of experimental psychology. General.

[25]  Anne Mangen,et al.  Reading linear texts on paper versus computer screen: Effects on reading comprehension , 2013 .

[26]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[27]  Amanda J. Rockinson-Szapkiw,et al.  Electronic versus traditional print textbooks: A comparison study on the influence of university students' learning , 2013, Comput. Educ..

[28]  Catherine C. Marshall,et al.  Annotation: from paper books to the digital library , 1997, DL '97.

[29]  G. Haddock,et al.  The medium can influence the message: Print-based versus digital reading influences how people process different types of written information. , 2019, British journal of psychology.

[30]  P. Alexander,et al.  Reading Across Mediums: Effects of Reading Digital and Print Texts on Comprehension and Calibration , 2017 .

[31]  Nicole Johnston,et al.  University Students’ Engagement with Textbooks in Print and E-book Formats , 2019 .

[32]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[33]  R. Devellis.  Classical Test Theory , 2006, Medical care.

[34]  Kara D. Sage,et al.  Pacing, Pixels, and Paper: Flexibility in Learning Words from Flashcards , 2016, J. Inf. Technol. Educ. Res..

[35]  Jennifer Pearson,et al.  The Digital Reading Desk: A lightweight approach to digital note-taking , 2012, Interact. Comput..

[36]  Jennifer Pearson,et al.  Designing for Digital Reading , 2013, Designing for Digital Reading.

[37]  Jakob Nielsen,et al.  How to Conduct a Heuristic Evaluation , 2006 .

[38]  E. Balint.  Memory and consciousness. , 1987, The International journal of psycho-analysis.

[39]  Ágústa Pálsdóttir.  Advantages and disadvantages of printed and electronic study material: perspectives of university students , 2019, Inf. Res..

[40]  Nicholas Chen,et al.  Designing a multi-slate reading environment to support active reading activities , 2012, TCHI.

[41]  Celine Latulipe,et al.  Quantifying the Creativity Support of Digital Tools through the Creativity Support Index , 2014, ACM Trans. Comput. Hum. Interact..

[42]  Jo-Anne LeFevre,et al.  Cognitive load in hypertext reading: A review , 2007, Comput. Hum. Behav..

[43]  Hock-Hai Teo,et al.  An empirical study of the effects of interactivity on web user attitude , 2003, Int. J. Hum. Comput. Stud..

[44]  David J. Shernoff,et al.  Student engagement in high school classrooms from the perspective of flow theory. , 2003 .

[45]  Christopher Lim,et al.  Book or screen, pen or keyboard? A cross-cultural sociological analysis of writing and reading habits basing on Germany, Italy and the UK , 2016, Telematics Informatics.