The Shape of Story: A Semiotic Artistic Visualization of a Communal Storytelling Experience

Telling stories is a central part of human culture. The development of computational systems that can understand and respond to human language is an integral part of AI research in general and narrative technologies in particular. In this paper, we describe a system that is able to understand human spoken English sentences and portray that understanding via a semiotic visual language in real-time. We then employ this system in a communal storytelling environment as part of an interactive art installation and create a medium for collaborative creative expression.

[1]  Charles J. Fillmore,et al.  THE CASE FOR CASE. , 1967 .

[2]  Penousal Machado,et al.  Generation of Concept-Representative Symbols , 2015, ICCBR.

[3]  Dirk Lindebaum.  Sapiens: A Brief History of Humankind - A Review , 2015 .

[4]  M. Minsky.  The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind , 2006 .

[5]  Vadim Bulitko,et al.  Interactive Narrative: An Intelligent Systems Approach , 2012, AI Mag..

[6]  Jonathan P. Rowe,et al.  Story-Based Learning: The Impact of Narrative on Learning Experiences and Outcomes , 2008, Intelligent Tutoring Systems.

[7]  Scott McCloud.  Understanding comics: the invisible art = Memahami komik / Scott McCloud; penerjemah S. Kinanti , 2001 .

[8]  Ruck Thawonmas,et al.  Comic Layout for Automatic Comic Generation from Game Log , 2008, ECS.

[9]  Alisha Panjwani,et al.  Constructing Meaning: Designing Powerful Story-Making Explorations for Children to Express with Tangible Computational Media , 2017, IDC.

[10]  F. Heider,et al.  An experimental study of apparent behavior , 1944 .

[11]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[12]  Matthew Guzdial,et al.  Visual Procedural Content Generation with an Artificial Abstract Artist , 2017 .

[13]  Yi-Zhe Song,et al.  Simple art as abstractions of photographs , 2013, CAE '13.

[14]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[15]  Lian Loke,et al.  Intimate Aesthetics and Facilitated Interaction , 2014, Interactive Experience in the Digital Age.

[16]  Andrés Gómez de Silva Garza,et al.  Automating Evolutionary Art in the Style of Mondrian , 2004, GECCO.

[17]  Brian Magerko,et al.  Digital Improvisational Theatre: Party Quirks , 2011, IVA.

[18]  Leonard Talmy,et al.  Force Dynamics in Language and Cognition , 1987, Cogn. Sci..

[19]  Simon Colton.  The painting fool in new dimensions , 2011 .

[20]  Dan Ventura,et al.  Finding Creativity in an Artificial Artist , 2013 .

[21]  Yvonne Rogers,et al.  Enticing People to Interact with Large Public Displays in Public Spaces , 2003, INTERACT.

[22]  Kerstin Dautenhahn,et al.  The origins of narrative: In search of the transactional format of narratives in humans and other social animals , 2002 .

[23]  Kang Zhang,et al.  Generation of Miro's Surrealism , 2016, VINCI.

[24]  Ariel Shamir,et al.  Generating comics from 3D interactive computer graphics , 2006, IEEE Computer Graphics and Applications.

[25]  Elise van den Hoven,et al.  StoryBeads: Preserving Indigenous Knowledge through Tangible Interaction Design , 2013, 2013 International Conference on Culture and Computing.

[26]  Brian Magerko,et al.  Empirically Studying Participatory Sense-Making in Abstract Drawing with a Co-Creative Cognitive Agent , 2016, IUI.

[27]  Kang Zhang,et al.  Generation of Kandinsky Art , 2016, Leonardo.

[28]  Heeyoung Lee,et al.  A Multi-Pass Sieve for Coreference Resolution , 2010, EMNLP.

[29]  Ernest Edmonds,et al.  On creative engagement , 2006 .

[30]  Michele D. Dickey.  Game Design Narrative for Learning: Appropriating Adventure Game Design Narrative Devices and Techniques for the Design of Interactive Learning Environments , 2006 .

[31]  Ken Perlin,et al.  Improving noise , 2002, SIGGRAPH.

[32]  Ana Paiva,et al.  So tell me what happened: turning agent-based interactive drama into comics , 2008, AAMAS.

[33]  Jennifer W. Glos.  Once Upon an Object... Computationally-Augmented Toys for Storytelling , 1998 .

[34]  Mitchell Harrop,et al.  Uncovering the Honeypot Effect: How Audiences Engage with Public Interactive Systems , 2016, Conference on Designing Interactive Systems.

[35]  Masaki Oshita.  Generating Animation from Natural Language Texts and Framework of Motion Database , 2009, 2009 International Conference on CyberWorlds.

[36]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[37]  Roger C. Schank,et al.  The fourteen primitive actions and their inferences. , 1973 .

[38]  M. Mateas,et al.  What is Narrative Intelligence ? , 1998 .

[39]  David S. Miall.  Experiencing narrative worlds: On the psychological activities of reading , 2000 .

[40]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[41]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[42]  R. Malina.  Aaron’s Code: Meta-Art, Artificial Intelligence and the Work of Harold Cohen by Pamela McCorduck (review) , 2017 .

[43]  P. Ekman.  An argument for basic emotions , 1992 .

[44]  Justine Cassell,et al.  StoryMat: a play space for collaborative storytelling , 1999, CHI Extended Abstracts.