The “book problem” and its neural correlates

Presence research can tell us why we feel present in the real world and can experience presence while using virtual reality technology (and movies and games) but has strikingly less to say on why we feel present in the scenes described in a book. Just how is it that the wonderful tangible detail of the real world or the complexity of digital technology can be matched and even surpassed by a story in a paperback book? This paper identifies a range of potential neurological solutions to this problem (and the “real world” and “dream” problems for good measure). We consider Jeannerod’s neural simulation of action, Grush’s emulation theory of representation and Rizzolatti’s work on mirror neurons as being candidate solutions to the “book problem”. We conclude by observing that these potential solutions further underline the “purpose” of presence is to act in the world whether it is real, virtual or solely in our imaginations.

[1]  S M Kosslyn,et al.  Visual images preserve metric spatial information: evidence from studies of image scanning. , 1978, Journal of experimental psychology. Human perception and performance.

[2]  A. Paivio.  Mental Representations: A Dual Coding Approach , 1986 .

[3]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[4]  Victor Nell,et al.  The Psychology of Reading for Pleasure: Needs and Gratifications , 1988 .

[5]  C. Keysers,et al.  Social Neuroscience: Mirror Neurons Recorded in Humans , 2010, Current Biology.

[6]  R. Schank,et al.  Knowledge and Memory: The Real Story , 1995 .

[7]  G. Tindall.  Countries of the Mind: The Meaning of Place to Writers , 2011 .

[8]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[9]  Z. Pylyshyn.  The imagery debate: Analogue media versus tacit knowledge. , 1981 .

[10]  Marleen B. Schippers,et al.  Mapping the information flow from one brain to another during gestural communication , 2010, Proceedings of the National Academy of Sciences.

[11]  Giuseppe Riva,et al.  From the body to the tools and back: A general framework for presence in mediated interactions , 2012, Interact. Comput..

[12]  Jack W. Tsao,et al.  Mirrors in the Brain: How Our Minds Share Actions, Emotions, and Experience, Giacomo Rizzolatti, Corrado Sinigaglia, translated by Frances Anderson, Oxford University Press (2006), ISBN-13: 978-0-19-921798-4, 242 pages, $49.95 , 2009, Journal of the Neurological Sciences.

[13]  Christoph Klimmt,et al.  Exploring the Book Problem: Text Design, Mental Representations of Space, and Spatial Presence in Readers , 2004 .

[14]  Zenon W. Pylyshyn,et al.  Validating computational models: A critique of Anderson's indeterminacy of representation claim. , 1979 .

[15]  F. Biocca.  Can we resolve the book , the physical reality , and the dream state problems ? From the two-pole to a three-pole model of shifts in presence , 2003 .

[16]  David Zeltzer,et al.  Autonomy, Interaction, and Presence , 1992, Presence: Teleoperators & Virtual Environments.

[17]  Giuseppe Riva,et al.  The Layers of Presence: A Bio-cultural Approach to Understanding Presence in Natural and Mediated Environments , 2004, Cyberpsychology Behav. Soc. Netw..

[18]  G. Rizzolatti,et al.  Mirrors in the Brain: How Our Minds Share Actions and Emotions , 2007 .

[19]  Holger Regenbrecht,et al.  Embodied Presence in Virtual Environments , 1999, Visual Representations and Interpretations.

[20]  Theodore R. Sarbin,et al.  Narrative Psychology: The Storied Nature of Human Conduct , 1986 .

[21]  Rick Grush,et al.  The emulation theory of representation: Motor control, imagery, and perception , 2004, Behavioral and Brain Sciences.

[22]  Rosa María Baños,et al.  The Third Pole of the Sense of Presence: Comparing Virtual and Imagery Spaces , 2005, PsychNology J..

[23]  A. Revonsuo.  Inner Presence: Consciousness as a Biological Phenomenon , 2005 .

[24]  M. Jeannerod,et al.  Mentally simulated movements in virtual reality: does Fitt's law hold in motor imagery? , 1995, Behavioural Brain Research.

[25]  Melanie C. Green,et al.  Narrative impact : social and cognitive foundations , 2003 .

[26]  Marie-Laure Ryan,et al.  Narrative as Virtual Reality: Immersion and Interactivity in Literature and Electronic Media , 2000 .

[27]  Melanie C. Green,et al.  In the Mind's Eye Transportation-Imagery Model of Narrative Persuasion , 2002 .

[28]  A. Paivio.  Imagery and verbal processes , 1972 .

[29]  Jeffrey M. Zacks,et al.  Reading Stories Activates Neural Representations of Visual and Motor Experiences , 2009, Psychological science.

[30]  David Herman,et al.  Basic elements of narrative , 2009 .

[31]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[32]  J. Decety.  Do imagined and executed actions share the same neural substrate? , 1996, Brain research. Cognitive brain research.

[33]  J. V. Wood,et al.  On building, defending, and regulating the self : a psychological perspective , 2005 .

[34]  Daniel Erlacher,et al.  Do REM (lucid) dreamed and executed actions share the same neural substrate , 2008 .

[35]  M. Iacoboni.  Neural mechanisms of imitation , 2005, Current Opinion in Neurobiology.

[36]  Ray Paton,et al.  Visual Representations and Interpretations , 1999, Springer London.

[37]  M. Jeannerod.  Neural Simulation of Action: A Unifying Mechanism for Motor Cognition , 2001, NeuroImage.

[38]  Thomas W. Schubert,et al.  Five Theses on the Book Problem : Presence in Books , Film and VR , 2002 .

[39]  Anna Kostygina,et al.  Narrative impact. Social and cognitive foundations , 2003, Z. für Medienpsychologie.

[40]  Melanie C. Green,et al.  Transportation into narrative worlds: Implications for the self , 2005 .

[41]  A. Glenberg,et al.  What memory is for: Creating meaning in the service of action , 1997, Behavioral and Brain Sciences.

[42]  Thomas B. Sheridan,et al.  Musings on Telepresence and Virtual Presence , 1992, Presence: Teleoperators & Virtual Environments.

[43]  Zenon W. Pylyshyn,et al.  The Imagery Debate: Analogue Media Versus Tacit Knowledge , 1988 .

[44]  A. Paivio.  Perceptual comparisons through the mind’s eye , 1975, Memory & cognition.

[45]  A. Sirigu,et al.  The Mental Representation of Hand Movements After Parietal Cortex Damage , 1996, Science.

[46]  J. Mazziotta,et al.  Cortical mechanisms of human imitation. , 1999, Science.

[47]  M. Ravenel,et al.  Science and the Modern World , 1926 .