Towards automated story analysis using participatory design

Involving a school teacher in the development of the intelligent writing tutor StoryStation allowed progress to be made on the problem of story classication. An experienced Scottish school-teacher developed a rating scale and guidelines for StoryStation's automated plot analysis agent for the story rewriting task. In this task, pupils rewrite a story in their own words, allowing them to devote their full attention to improving their writing technique instead of creating a new plot. If the pupil forgets or confuses parts of the plot, the software needs to be able to detect this so that it may alert the pupil or their teacher. Teacher participation in the creation of the rating scale guided both the development of the tools used to analyze the stories and the scope of the plot analysis agent. A teacher and a story-teller rated the corpus, and this scale was used to successfully train the agent to classify both ''good'' and ''poor'' stories. Classification of ''excellent'' and ''fair'' stories proved to be very difficult. A number of facets of story understanding are shown to be beyond the range of the automated plot analysis agent and the advantages and disadvantages of automated plot analysis are weighed, including social factors.