Automatic assessment of expressive oral reading

We investigated the automatic assessment of expressive children's oral reading of grade level text passages using a standardized rubric. After a careful review of the reading literature and a close examination of the rubric, we designed a novel set of prosodic and lexical features to characterize fluent expressive reading. A number of complementary sources of information were used to design the features, each of them motivated by research on different components of reading fluency. Features are connected to the child's reading rate, to the presence and number of pauses, filled-pauses and word-repetitions, the correlation between punctuation marks and pauses, the length of word groupings, syllable stress and duration and the location of pitch peaks and contours. The proposed features were evaluated on a corpus of 783 one-minute reading sessions from 313 students reading grade-leveled passages without assistance (cold unassisted reading). Experimental results show that the proposed lexical and prosodic features provide complementary information and are able to capture the characteristics of expressive reading. The results showed that on both the 2-point and the 4-point expressiveness scales, computer-generated ratings of expressiveness agreed with human raters better than the human raters agreed with each other. The results of the study suggest that automatic assessment of expressive oral reading can be combined with automatic measures of word accuracy and reading rate to produce an accurate multidimensional estimate of children's oral reading ability.

[1]  Linda Adkins Ford,et al.  The Relationship of Dynamic Indicators of Basic Early Literacy Skills (DIBELS) Oral Reading Fluency and the Terra Nova, 2nd Ed. Performance on Ohio Grade 3 Reading Achievement Assessment , 2008 .

[2]  G. Logan.  Toward an instance theory of automatization. , 1988 .

[3]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[4]  Ronald Rosenfeld,et al.  The CMU Statistical Language Modeling Toolkit and its use in the 1994 ARPA CSR Evaluation , 1995 .

[5]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[6]  Paula J Schwanenflugel,et al.  Prosody of Syntactically Complex Sentences in the Oral Reading of Young Children. , 2006, Journal of educational psychology.

[7]  Michelle K. Hosp,et al.  Oral Reading Fluency as an Indicator of Reading Competence: A Theoretical, Empirical, and Historical Analysis , 2001 .

[8]  D. Borsboom.  Educational Measurement (4th ed.) , 2009 .

[9]  A Wichmann,et al.  Prosodic Characteristics of Skilled Reading: Fluency and Expressiveness in 8—10-year-old Readers , 2002, Language and speech.

[10]  Daniel Jurafsky,et al.  The detection of emphatic words using acoustic and lexical features , 2005, INTERSPEECH.

[11]  Jack Mostow,et al.  Adapting a duration synthesis model to rate children's oral reading prosody , 2010, INTERSPEECH.

[12]  Steven A. Stahl,et al.  Fluency: A review of developmental and remedial practices. , 2003 .

[13]  Rebekah George Benjamin,et al.  Text Complexity and Oral Reading Prosody in Young Readers , 2010 .

[14]  Mark R. Shinn,et al.  Advanced applications of curriculum-based measurement , 1998 .

[15]  Timothy V. Rasinski,et al.  Assessing Reading Fluency. , 2004 .

[16]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[17]  Douglas G. Altman,et al.  Practical statistics for medical research , 1990 .

[18]  Min Li,et al.  Oral Reading Fluency Assessment: Issues of Construct, Criterion, and Consequential Validity , 2010 .

[19]  C. Spearman.  The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[20]  Catherine E. Snow,et al.  Preventing reading difficulties in young children , 1998 .

[21]  R. Cole,et al.  THE OGI KIDS’ SPEECH CORPUS AND RECOGNIZERS , 2000 .

[22]  Timothy V. Rasinski,et al.  Reading Fluency: More Than Automaticity? More Than a Concern for the Primary Grades? , 2009 .

[23]  Andreas Oranje,et al.  Fourth-Grade Students Reading Aloud: NAEP 2002 Special Study of Oral Reading. The Nation's Report Card. NCES 2006-469. , 2005 .

[24]  Alicia Sandman,et al.  Dynamic indicators of basic early literacy skills , 2010 .

[25]  Daniel Bolaños.  The Bavieca open-source speech recognition toolkit , 2012, 2012 IEEE Spoken Language Technology Workshop (SLT).

[26]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[27]  Rupal Patel,et al.  Displaying prosodic text to enhance expressive oral reading , 2011, Speech Commun..

[28]  D. Langenberg.  Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction , 2000 .

[29]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[30]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[31]  Steven A. Stahl,et al.  Becoming a Fluent Reader: Reading Skill and Prosodic Features in the Oral Reading of Young Readers. , 2004, Journal of educational psychology.

[32]  Elmar Nöth,et al.  "Of all things the measure is man" automatic classification of emotions and inter-labeler consistency [speech-based emotion recognition] , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[33]  Chih-Jen Lin,et al.  Feature Ranking Using Linear SVM , 2008, WCCI Causation and Prediction Challenge.

[34]  S. Vaughn,et al.  A Synthesis of Research on Effective Interventions for Building Reading Fluency with Elementary Students with Learning Disabilities , 2002, Journal of learning disabilities.

[35]  Deborah C. Simmons,et al.  The Importance and Decision-Making Utility of a Continuum of Fluency-Based Indicators of Foundational Reading Skills for Third-Grade High-Stakes Outcomes , 2001 .

[36]  György Szaszák,et al.  Using prosody to improve automatic speech recognition , 2010, Speech Commun..

[37]  Timothy V. Rasinski,et al.  Training teachers to attend to their students’ oral reading fluency , 1991 .

[38]  Jack Mostow,et al.  Automated Assessment of Oral Reading Prosody , 2009, AIED.

[39]  Descriptors Educational,et al.  of Educational Measurement , 1988 .

[40]  JAN HASBROUCK,et al.  636 スライディングモード補償機能を用いた振動抑制のための高速・高精度位置制御 , 2009 .

[41]  Paula J Schwanenflugel,et al.  A Longitudinal Study of the Development of Reading Prosody as a Dimension of Oral Reading Fluency in Early Elementary School Children. , 2008, Reading research quarterly.

[42]  Ronald A. Cole,et al.  FLORA: Fluent oral reading assessment of children's speech , 2011, TSLP.

[43]  Marie M. Clay,et al.  Juncture, pitch, and stress as reading behavior variables , 1971 .

[44]  Melanie R. Kuhn,et al.  Review of Research: Aligning Theory and Assessment of Reading Fluency: Automaticity, Prosody, and Definitions of Fluency , 2010 .

[45]  W. Chafe.  Punctuation and the Prosody of Written Language , 1988 .

[46]  Jack Mostow,et al.  Two methods for assessing oral reading prosody , 2011, TSLP.

[47]  Teri Wallace,et al.  Literature Synthesis on Curriculum-Based Measurement in Reading , 2007 .

[48]  Andreas Stolcke,et al.  Enriching speech recognition with automatic detection of sentence boundaries and disfluencies , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[49]  A. B.,et al.  SPEECH COMMUNICATION , 2001 .

[50]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[51]  Melanie R. Kuhn.  A Comparative Study of Small Group Fluency Instruction , 2005 .