Drawcto: A Multi-Agent Co-Creative AI for Collaborative Non-Representational Art

Non-representational art–such as works by Wassily Kandinsky, Joan Mitchell, Willem de Kooning, etc.–showcases diverse artistic expressions and challenges viewers with its interpretive open-endedness and lack of a clear mapping to our everyday reality. Human cognition and perception nonetheless aid us in making sense of, reasoning about, and discussing the perceptual features prevalent in such non-representational art. While there have been various Computational Creative systems capable of generating representational artwork, only a few existing Computational (Co)Creative systems for visual arts can produce nonrepresentational art. How would a co-creative AI that incorporates elements of the human visual perception theory be able to collaborate with a human in co-creating a nonrepresentational art? This paper explores this challenge in detail, describes potential machine learning and non-machine learning approaches for designing an AI agent and introduces a new web-based, multi-agent AI drawing application, called Drawcto, capable of co-creating non-representational artwork with human collaborators. Playing video games can be a highly creative activity, requiring individuals to engage in creative behaviors like content creation, collaborative building, problem solving, etc. (Green and Kaufman 2015; Blanco-Herrera, Gentile, and Rokkum 2019). A model of creativity within AI agents may support new forms of creative gameplay and new applications of AI in game spaces. Inspired by this potential, we focus on exploring a specific creative interaction modality that has its roots in popular sketch-based games like Pictionary or web games like Skribbl.io (mel 2011). Previous research with these games has been limited to the training and development of computationally creative agents (Bhunia et al. 2020; Sarvadevabhatla et al. 2018); our aim is to develop a co-creative system for co-creating non-objective visual art that seeks to invoke the properties of human-computer cocreativity in ways applicable to the study, creation, and play of digital and analog games involving creative aspects. Computational Creativity (CC) in the visual arts has gained attention since the early days of AARON (Cohen 1995). Over the years, researchers have developed various Copyright © 2021, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. algorithms and CC systems that are capable of (co)creating artworks in a specific artistic style (Gatys, Ecker, and Bethge 2016), identify and suggest conceptually or visually similar objects (Karimi et al. 2020), produce strokes or pixels to complete a sketch or an image semantically (Su et al. 2020; Iizuka, Simo-Serra, and Ishikawa 2017), etc. While many of these approaches have examined creating representational artwork (e.g., realistic or impressionistic presentations of real-world scenes and objects), little work has been done in exploring how a more abstract, non-representational work could be done by (or in collaboration with) AI–inspired by human cognition and perception–that can discuss its intention behind specific features and composition of the artwork with a human collaborator. In an attempt to address this, we present Drawcto a web-based, multi-agent AI drawing application capable of co-creating and discussing specific features of non-representational art with a human collaborator. People often use abstract art and non-representational art interchangeably to refer to the same painting style, yet there are crucial differences between the two terms (Ashmore 1955). Abstract artwork distorts the view of a familiar subject (i.e., a thing, face, body, place, etc.). For example, Picasso distorts a person’s face to show different views of the same figure within a single painting. The resulting artwork appears abstracted, but still, there are discernable features and structures intact from the original subject. Figure 1a and 1b show examples of abstract art. Non-representational art, on the other hand, doesn’t have a known object or a thing that the artwork is trying to depict. For non-representational art–also known as non-objective art–the artist only uses visual design elements like form, shape, color, line, etc., to express themselves. Non-representational art represents the spiritual, mystic, non-materialistic, experiential, or creative painting/thought process of the artist (Fingesten 1961), making it challenging to appreciate, contextualize, or understand. For example, Kandinsky’s non-objective compositions represent his emotional experience of listening to music, Mondrian’s paintings–which only contain straight lines and primary colors–represent “what is absolute, among the relativity of time and space” (Wallis 1960), Pollock’s artworks represent the action-painting process, in other words it depicts Figure 1: (From left to right)1a:Picasso’s painting (Picasso 1932), 1b:Klee’s painting (Klee 1922), 1c:Rangoli designs (Balaji 2018), 1d:Joan Mitchell’s composition (Bracket 1989) the forces that lead to its creation. Figure 1c and Figure 1d show examples of non-representational art. Nonrepresentational art generally is not preconceived; instead, it emerges from the artist’s in-the-moment interaction with the medium, reflection-in-design process (Schön 1983). Generating visually sensible content in such a dynamic scenario is the main challenge for developing an AI agent for co-creating non-representational art. We cannot simply train the agent to use object detection or classification to make sense of and generate new strokes as usually there are no recognizable objects. At the same time, we can’t generate random strokes as they would not be visually sensible. Therefore, developing an AI that can create various strokes based on its perceptual ability to understand and reason with the quality of strokes made by the human collaborator is the challenge we address in this research. We utilize the perceptual organization theory (or Gestalt theory) for the agent(s) to make sense of and generate new strokes while co-creating a non-representational artwork. Gestalt theory describes a finite set of rules that guide and aid the reasoning of our visual system. Some of the gestalt grouping principles are proximity, balance, continuity, similarity, etc. (Arnheim 1957). Previously, researchers have used perceptual organization theory for various applications like image segmentation, contour detection, shape parsing, etc. In this paper, we present work that attempts to circumnavigate the ”authoring bottleneck” commonly associated with co-creative systems (Csinger, Booth, and Poole 1995) by using perceptual theories (like Gestalt’s) to both bootstrap various learning/non-learning approaches to collaborative sketching as well as a basis for affording AI explainability. We have organized the paper as follows. We examine potential learning and non-learning approaches for developing an AI agent in Related Work. The System Design section describes the current version of Drawcto and explains each component in detail. In the Discussion section, we reflect on the present drawbacks of the three drawing agents. Finally, we share potential future avenues of research we have identified for Drawcto in Future Work.