Imagination Machines: A New Challenge for Artificial Intelligence

The aim of this paper is to propose a new overarching challenge for AI: the design of imagination machines. Imagination has been defined as the capacity to mentally transcend time, place, and/or circumstance. Much of the success of AI currently comes from a revolution in data science, specifically the use of deep learning neural networks to extract structure from data. This paper argues for the development of a new field called imagination science, which extends data science beyond its current realm of learning probability distributions from samples. Numerous examples are given in the paper to illustrate that human achievements in the arts, literature, poetry, and science may lie beyond the realm of data science, because they require abilities that go beyond finding correlations: for example, generating samples from a novel probability distribution different from the one given during training; causal reasoning to uncover interpretable explanations; or analogical reasoning to generalize to novel situations (e.g., imagination in art, representing alien life in a distant galaxy, understanding a story about talking animals, or inventing representations to model the large-scale structure of the universe). We describe the key challenges in automating imagination, discuss connections between ongoing research and imagination, and outline why automation of imagination provides a powerful launching pad for transforming AI. “Imagination is more important than knowledge. Knowledge is limited. Imagination encircles the world.” – Albert Einstein ‘Imagine there’s no countries. It isn’t hard to do. Nothing to kill or die for. And no religion too” – Song by John Lennon Artificial intelligence is poised to become the “electricity” of our age (Ng 2016), transforming industries across a wide spectrum of areas, from autonomous driving to voiceactivated virtual personal assistants. However, these successes of AI, powered by data science (Murphy 2013) and deep learning (Goodfellow, Bengio, and Courville 2016), may not be sufficient for AI to be capable of matching human capabilities in the long run. This paper focuses specifically on one core capability – imagination – and discusses why its automation may be fundamental to the continuing success of AI in the coming decades. Copyright c © 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Figure 1: Jean-Michel Basquiat’s untitled painting of a human skull sold recently at a New York auction for over 100 million dollars. Art is a paradigmatic example of the imaginative capacity of humans. The Oxford Handbook of the Development of Imagination defines imagination as the capacity to mentally transcend time, place, and/or circumstance (Taylor 2013). Einstein prized imagination because it enabled him to pose hypothetical questions, such as “What would the world look like if I rode a beam of light”, a question that led him to develop the revolutionary theory of special (and later, general) relativity. Imagination is a hallmark of counterfactual and causal reasoning (Pearl 2009). Imagination also provides the foundational basis for art (see Figure 1). Basquiat’s painting illustrates what is special about imagination in art: fidelity to the original is not the objective here, but rather the striking use of colors and textures to signify an illusion. In John Lennon’s famous song “Imagine”, he asks us to contemplate a world without countries, an abstraction of reality that is a hallmark of imaginative thinking. In Beethoven’s Pastoral symphony, each of the five movements portrays a particular aspect of nature, from the slow movement depicting the motion of a stream to the strenuous fourth movement depicting the arrival of a storm and thunder. Imagination plays a central role in the lives of children and adults. The runaway success of the Harry Potter series shows what a gifted writer can accomplish in holding the attention of children, highlighting the crucial role that make-believe plays in the formative years of children. Wonder Woman was the smash $1 billion Hollywood hit of the year, showing once again that the world of fantasy and imagination is one sure fire way to create a money making movie. Although imagination has attracted the attention of some researchers, the early work on this topic has been somewhat limited in scope (Alexander 2001), and more recent work has explored this topic in rather restricted situations (Pascanu et al. 2017; Elgamman et al. 2017). This brief paper summarizes several converging lines of argument as to why imagination machines constitutes a broad comprehensive research program that has the potential to transform AI in the next few decades. Imagination is one of the hallmarks of human intelligence (Asma 2017), an ability that manifests itself in children at a very young age, and prized by society in many endeavors, from art (see Figure 1) and literature to science. It represents an area largely ignored by most AI research, although tantalizing glimpses of the power of imagination are beginning to manifest themselves in different strands of current AI research, as will be discussed below. As work by the Nobel-prize winning economist Daniel Kahneman (with his late colleague, Amos Tversky) has shown, based on many empirical studies, human decision making does not conform to the maxims of expected utility theory. Faced with a “lottery” (a decision problem with several uncertain outcomes with different payoffs), human decision making often does not result in picking choices that have the maximum expected utility. Year after year, in state after state, millions of Americans buy lottery tickets, because they can “imagine” themselves winning and becoming rich, despite the vanishingly small probability of winning. Clearly, for many humans, imagination in this case (mis)guides their actions into violating the principle of maximizing expected utility. From Data Science to Imagination Science “A theory is not like an airline or bus timetable. We are not interested simply in the accuracy of its predictions. A theory also serves as a base for thinking. It helps us to understand what is going on by enabling us to organize our thoughts. Faced with a choice between a theory which predicts well but gives us little insight into how the system works and one which gives us this insight but predicts badly, I would choose the latter, and I am inclined to think that most economists would do the same.” – Ronald Coase, Nobel-prize winning economist. “I now take causal relationships to be the fundamental building blocks both of physical reality and of human understanding of that reality, and I regard probabilistic relationships as but the surface phenomena of the causal machinery that underlies and propels our understanding of the world”. – Judea Pearl, Causality. The ability to coax structure out of large datasets, particFigure 2: Generative Adversarial Networks (GANs) (Goodfellow et al. 2014) can create images from samples of a fixed distribution, but imaginative art such as Basquiat’s painting in Figure 1 require going beyond reproducing existing art. A variant of a GAN, called a “creative adversarial network” attempts to generate “novel” art (Elgamman et al. 2017), producing the images shown above. ularly for difficult to program tasks, such as computer vision, speech recognition, and high-performance game playing, has led to significant successes of machine learning in a variety of real-world tasks, particularly using deep learning approaches. Broadly speaking, machine learning or data science is the process of constructing a probability distribution from samples, or equivalently being able to generate new samples from given samples that fool an expert discriminator (Goodfellow et al. 2014). The fundamental difference between data science and imagination science is that the latter extends to realms far beyond the former: for example, imagination science addresses the problem of generating samples that are “novel”, meaning they come from a distribution different from the one used in training. Imagination science also addresses the problem of causal reasoning to uncover simple explanations for complex events, and uses analogical reasoning to understand novel situations. Can computers produce novel art like Basquiat’s painting? Recent work on a variant of a generative adversarial network called CAN (for Creative Adversarial Network) (see Figure 2) shows that computers can be trained to produce images that are both art, as well as differ from standard styles, like impressionism or cubism. While CANs are a useful step forward, building on Berlyne’s theory of novelty (Berlyne 1960), their architecture is currently specific to art, and not general enough to provide a computational framework for imagination. However, it does suggest one possible avenue to designing an Imagination Network architecture. Other extensions of GAN models, such as CycleGAN (Zhu et al. 2017), are suggestive, but such extensions are at present tailored to visual domains, and even in that circumscribed setting, only capable of specific generalizations (e.g., turning Monet styled watercolor paintings into what look like digital photographs of the original scene). Most machine learning is based on the discovery and exploitation of statistical correlations from data, including approaches using parametric graphical model representations (Murphy 2013) or kernel-based non-parametric representations (Schölkopf and Smola 2002), and most recently, nonlinear neural net based models (Goodfellow, Bengio, and Courville 2016). Correlation, as has been pointed out many times, is not causation, however, and causal reasoning is one of the primary hallmarks of human imaginative reasoning (Pearl 2009). One of the primary rationales for causal reasoning is the need to provide comprehensible explanations, which will become increasingly important as autonomous systems play an ever large