Visualizing Topic Flow in Students' Essays

Visualizing how the parts of a document relate to each other and producing automatically generated quality measures that people can understand are means that writers can use to improve the quality of their compositions. This paper presents a novel document visualization technique and a measure of quality based on the average semantic distance between parts of a document. We show how the visualization helps tutors mark essays more efficiently and reliably, and how a distance index calculated for the visualizations correlates with grades. The technique is further evaluated using three dimensionality reduction techniques. The results provide evidence that the degree of topic flow between consecutive sentences and paragraphs is related to essay quality.

[1]  Danielle S. McNamara,et al.  Handbook of latent semantic analysis , 2007 .

[2]  Jin-Tan Yang,et al.  Visualizing Argumentation -Software Tools for Collaborative and Educational Sense-Making , 2003, J. Educ. Technol. Soc..

[3]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[4]  D. McNamara,et al.  Cohesion, coherence, and expert evaluations of writing proficiency , 2010 .

[5]  Danielle S. McNamara,et al.  Computational Methods to Extract Meaning From Text and Advance Theories of Human Cognition , 2011, Top. Cogn. Sci..

[6]  Xin Liu,et al.  Generic text summarization using relevance measure and latent semantic analysis , 2001, SIGIR '01.

[7]  Philip M. McCarthy,et al.  Linguistic Features of Writing Quality , 2010 .

[8]  Jinhao Wang,et al.  Automated Essay Scoring Versus Human Scoring: A Comparative Study , 2007 .

[9]  Arthur C. Graesser,et al.  Coh-Metrix: Analysis of text on cohesion and language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[10]  E. Sutinen,et al.  Automatic assessment of the content of essays based on course materials , 2004, ITRE 2004. 2nd International Conference Information Technology: Research and Education.

[11]  Thomas Hofmann,et al.  Unsupervised Learning by Probabilistic Latent Semantic Analysis , 2004, Machine Learning.

[12]  Eileen Kintsch,et al.  Summary Street: Interactive Computer Support for Writing , 2004 .

[13]  Philippe Dessus.  An Overview of LSA-Based Systems for Supporting Learning and Teaching , 2009, AIED.

[14]  Stanislaw Osinski.  Improving Quality of Search Results Clustering with Approximate Matrix Factorisations , 2006, ECIR.

[15]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[16]  Chris North,et al.  Toward measuring visualization insight , 2006, IEEE Computer Graphics and Applications.

[17]  Kalina Yacef,et al.  Collaborative Writing Support Tools on the Cloud , 2011, IEEE Transactions on Learning Technologies.

[18]  Rafael A. Calvo,et al.  Glosser: Enhanced Feedback for Student Writing Tasks , 2008, 2008 Eighth IEEE International Conference on Advanced Learning Technologies.

[19]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[20]  J. Leeuw.  Applications of Convex Analysis to Multidimensional Scaling , 2000 .

[21]  Rafael A. Calvo,et al.  Analysing Semantic Flow in Academic Writing , 2009, AIED.

[22]  Timothy J. Beals.  Between Teachers and Computers: Does Text-Checking Software Really Improve Student Writing? , 1998 .

[23]  Rafael A. Calvo,et al.  Students' Conceptions of Tutor and Automated Feedback in Professional Writing , 2010 .

[24]  Karel Jezek,et al.  Two uses of anaphora resolution in summarization , 2007, Inf. Process. Manag..

[25]  Rhonda Liss,et al.  Effective Academic Writing , 2006 .

[26]  Rafael A. Calvo,et al.  Visualizing Paragraph Closeness for Academic Writing Support , 2009, 2009 Ninth IEEE International Conference on Advanced Learning Technologies.

[27]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[28]  Arthur C. Graesser,et al.  Select-a-Kibitzer: A Computer Tool that Gives Meaningful Feedback on Student Compositions , 2000, Interact. Learn. Environ..

[29]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.