Computationally Assessing Expert Judgments of Freewriting Quality

This study examines the relationship between the linguistic features of freewrites and human assessments of freewrite quality. Freewriting is a prewriting strategy that has received little experimental attention, particularly in terms of linguistic differences between high and low quality freewrites. This study builds upon the authors’ previous study, in which linguistic features of freewrites written by 9th and 11th grade students were included in a model of the freewrites’ quality (Weston, Crossley, & McNamara; 2010). The current study reexamines this model using a larger data set of freewrites. The results show that similar linguistic features reported in the Weston et al. model positively correlate with expert ratings in the new data set. Significant predictors in the current model of freewrite quality were total number of words and stem overlap. In addition, analyses suggest that 11th graders, as compared to 9th graders, wrote higher quality and longer freewrites. Overall, the results of this study support the conclusion that better freewrites are longer and more cohesive than poor freewrites.

[1]  Avon Crismore,et al.  Metadiscourse in Persuasive Writing , 1993 .

[2]  Say Wei Foo,et al.  Hidden Markov Model Based Visemes Recognition, Part II: Discriminative Approaches , 2009 .

[3]  D. McNamara,et al.  Working memory capacity and strategy use , 2001, Memory & cognition.

[4]  Scott Jarvis,et al.  vocd: A theoretical and empirical evaluation , 2007 .

[5]  M. Louwerse.  An analytic and cognitve parameterization of coherence relations , 2002 .

[6]  Arthur C. Graesser,et al.  Using LSA to Automatically Identify Givenness and Newness of Noun Phrases in Written Discourse , 2005 .

[7]  Nikos Nikolaidis,et al.  Robust Zero-Bit and Multi-Bit Audio Watermarking Using Correlation Detection and Chaotic Signals , 2008 .

[8]  Peter Elbow,et al.  Writing Without Teachers , 1973 .

[9]  M. Halliday.  NOTES ON TRANSITIVITY AND THEME IN ENGLISH. PART 2 , 1967 .

[10]  Andy P. Field,et al.  Discovering Statistics Using SPSS , 2000 .

[11]  Danielle S. McNamara,et al.  Handbook of latent semantic analysis , 2007 .

[12]  Philip M. McCarthy,et al.  Linguistic Features of Writing Quality , 2010 .

[13]  A. Murat Tekalp,et al.  Multimodal Speaker Identification Using Discriminative Lip Motion Features , 2009 .

[14]  Ruth E. Knudson.  Effects of Instructional Strategies on Children's Informational Writing. , 1989 .

[15]  Steve Hinkle,et al.  An Experimental Comparison of the Effects of Focused Freewriting and Other Study Strategies on Lecture Comprehension , 1990 .

[16]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[17]  A. Paivio.  Abstractness, imagery, and meaningfulness in paired-associate learning , 1965 .

[18]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[19]  Steven E. Stern,et al.  Computer Synthesized Speech Technologies: Tools for Aiding Impairment , 2010 .

[20]  Michael Wilson,et al.  MRC psycholinguistic database: Machine-usable dictionary, version 2.00 , 1988 .

[21]  Danielle S. McNamara,et al.  Towards A Computational Assessment of Freewriting Quality , 2010, FLAIRS.

[22]  Tapio Seppänen,et al.  Digital Audio Watermarking Techniques and Technologies: Applications and Benchmarks , 2007 .

[23]  M. Reynolds.  Freewriting's Origin , 1984 .

[24]  Arthur C. Graesser,et al.  Coh-Metrix: Analysis of text on cohesion and language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[25]  Charles A. Perfetti,et al.  The Acquisition of Reading Comprehension Skill , 2008 .

[26]  D. McNamara,et al.  A Linguistic Analysis of Simplified and Authentic Texts , 2007 .

[27]  David Malvern,et al.  Lexical Diversity and Language Development , 2004 .

[28]  Alan Wee-Chung Liew,et al.  Visual Speech Recognition: Lip Segmentation and Mapping , 2008 .

[29]  Walter Kintsch,et al.  Toward a model of text comprehension and production. , 1978 .

[30]  R. Logie,et al.  Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1,944 words , 1980 .

[31]  Philip M. McCarthy,et al.  MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment , 2010, Behavior research methods.

[32]  David Malvern,et al.  Lexical Diversity and Language Development: Quantification and Assessment , 2004 .

[33]  Ian Witten,et al.  Data Mining , 2000 .

[34]  Joseph K. Torgesen,et al.  Repeated reading and reading fluency in learning disabled children. , 1985 .

[35]  Ralf W. Schlosser,et al.  Are speech-generating devices viable AAC options for adults with intellectual disabilities? , 2010 .

[36]  Javier Alda,et al.  “Evaluator”: A Grading Tool for Spanish Learners , 2013 .