Identification of Reading Difficulties by a Digital Game-Based Assessment Technology

Computerized game-based assessment (GBA) system for screening reading difficulties may provide substantial time and cost benefits over traditional paper-and-pencil assessment while providing means also to individually adapt learning content in educational games. To study the reliability and validity of a GBA system to identify struggling readers performing below a standard deviation from mean in paper-and-pencil test either in raw scores and grade-normative scores, a large-scale study with first to fourth grade students (N = 723) was conducted, where GBA was administrated as a group test by tablet devices. Overall, the results indicated that the GBA can be successfully used to identify students with reading difficulties with acceptable reliability. Although the reliability of the results were at a very good level overall, the identification was even better in the reading fluency than in reading accuracy and in terms of raw scores than in grade-normative scores. These findings are the first to demonstrate the promise of GBA in assessing reading skills reliably and in a cost-efficient manner in classrooms. Furthermore, the developed GBA is directly applicable to an educational game for successfully supporting reading development of learners with varying levels of reading skill.

[1]  Peter Tymms,et al.  Identifying reading problems with computer-adaptive assessments , 2007, J. Comput. Assist. Learn..

[2]  R. Bennett.  The Changing Nature of Educational Assessment , 2015 .

[3]  J. Mckechnie,et al.  Students’ perceptions of their school environments and the relationship with educational outcomes , 2011, Educational and Child Psychology.

[4]  Jeffrey P. Bigham,et al.  Dytective: towards detecting dyslexia across languages using an online game , 2016, W4A.

[5]  Douglas Fuchs,et al.  The Construct and Predictive Validity of a Dynamic Assessment of Young Children Learning to Read: Implications for RTI Frameworks , 2011, Journal of learning disabilities.

[6]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[7]  Fatma Susilawati Mohamad,et al.  Heuristic Evaluation Of i-Dyslex Tool for Dyslexia Screening , 2017 .

[8]  Neal M. Kingston.  Comparability of Computer- and Paper-Administered Multiple-Choice Tests for K–12 Populations: A Synthesis , 2008 .

[9]  Heikki Lyytinen,et al.  This Reprint May Differ from the Original in Pagination and Typographic Detail. the Graphogame Method: the Theoretical and Methodological Background of the Technology-enhanced Learning Environment for Learning to Read the Graphogame Method: the Theoretical and Methodological Background of the Techno , 2022 .

[10]  Paige C. Pullen,et al.  Reading Fluency Assessment and Instruction: What, Why, and How? , 2005, The Reading Teacher.

[11]  Veronica Ng,et al.  The computerised-based Lucid Rapid Dyslexia Screening for the identification of children at risk of dyslexia: A Singapore study , 2011, Educational and Child Psychology.

[12]  E. Walker,et al.  Diagnostic and Statistical Manual of Mental Disorders , 2013 .

[13]  Chris Singleton,et al.  Computer‐based cognitive assessment and the development of reading , 2000 .

[14]  P. H. Seymour,et al.  Foundation literacy acquisition in European orthographies. , 2003, British journal of psychology.

[15]  Catalina V. Mourgues-Codern,et al.  Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence , 2018, Front. Psychol..

[16]  Heikki Lyytinen,et al.  Heterogeneity in adult dyslexic readers: Relating processing skills to the speed and accuracy of oral text reading , 2001 .

[17]  L. Fuchs,et al.  Selecting At-Risk Readers in First Grade for Early Intervention: A Two-Year Longitudinal Study of Decision Rules and Procedures , 2006 .

[18]  Hong Jiao,et al.  Comparability of Computer-Based and Paper-and-Pencil Testing in K–12 Reading Assessments , 2008 .

[19]  S. Crewther,et al.  Preliminary validation of FastaReada as a measure of reading fluency , 2015, Front. Psychol..

[20]  Dara Ramalingam,et al.  How Can the Use of Data from Computer-Delivered Assessments Improve the Measurement of Twenty-First Century Skills? , 2018 .

[21]  Timo Ahonen,et al.  The Nature of and Factors Related to Reading Difficulties Among Adolescents in a Transparent Orthography , 2013 .

[22]  Yang Liu,et al.  An introduction to decision tree modeling , 2004 .

[23]  G. Gillon,et al.  Predicting reading outcomes in the classroom using a computer-based phonological awareness screening and monitoring assessment (Com-PASMA) , 2014, International journal of speech-language pathology.

[24]  Juha-Matti Latvala,et al.  Embedding Preschool Assessment Methods into Digital Learning Games to Predict Early Reading Skills , 2017 .

[25]  Min Li,et al.  Oral Reading Fluency Assessment: Issues of Construct, Criterion, and Consequential Validity , 2010 .

[26]  Ron Dumont,et al.  Woodcock-Johnson III Tests Of Cognitive Abilities , 2014 .

[27]  O. Bueno,et al.  Contrasting group analysis of Brazilian students with dyslexia and good readers using the computerized reading and writing assessment battery “BALE” , 2014, Front. Psychol..

[28]  The Predictive Validity of a Computer-Adaptive Assessment of Kindergarten and First-Grade Reading Skills , 2015 .

[29]  Andrew J. Martin,et al.  Computer-Adaptive Testing: Implications for Students’ Achievement, Motivation, Engagement, and Subjective Test Experience , 2018 .

[30]  Pascale Colé,et al.  French normative data on reading and related skills from EVALEC, a new computerized battery of tests (end Grade 1, Grade 2, Grade 3, and Grade 4) , 2005 .

[31]  A. Magnan,et al.  Computer-based assessment of reading ability and subtypes of readers with reading comprehension difficulties: a study in French children from G2 to G9 , 2018, European Journal of Psychology of Education.

[32]  Jack M Fletcher,et al.  Evidence-Based Assessment of Learning Disabilities in Children and Adolescents , 2005, Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53.

[33]  K. Eklund,et al.  Teachers’ Ability to Identify Children at Early Risk for Reading Difficulties in Grade 1 , 2018 .

[34]  Alicia Sandman,et al.  Dynamic indicators of basic early literacy skills , 2010 .

[35]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[36]  Samuel Greiff,et al.  Computer-generated log-file analyses as a window into students' minds? A showcase study based on the PISA 2012 assessment of problem solving , 2015, Comput. Educ..

[37]  Athanassios Protopapas,et al.  Validation of Unsupervised Computer-Based Screening for Reading Disability in Greek Elementary Grades 3 and 4 , 2008 .

[38]  Carlo Perrotta,et al.  Game-Based Learning: Latest Evidence and Future Directions , 2013 .

[39]  H. Catts,et al.  The Use of a Dynamic Screening of Phonological Awareness to Predict Risk for Reading Disabilities in Kindergarten Children , 2011, Journal of learning disabilities.

[40]  Moslem Yousefi,et al.  A Procedural Content Generation-Based Framework for Educational Games: Toward a Tailored Data-Driven Game for Developing Early English Reading Skills , 2018 .

[41]  Heikki Lyytinen,et al.  Literacy Skill Development of Children with Familial Risk for Dyslexia through Grades , 2014 .