Ideomotor design: Using common coding theory to derive novel video game interactions

Recent experiments show video games have a range of positive cognitive effects, such as improvement in attention, spatial cognition and mental rotation, and also overcoming of cognitive disabilities such as fear of flying. Further, game environments are now being used to generate scientific discoveries, and bring about novel phenomenological effects, such as out-of-body experiences. These advances provide interesting interaction design possibilities for video games. However, since the cognitive mechanisms underlying these experimental effects are unknown, it is difficult to systematically derive novel systems and interaction designs based on these results. We review the emerging cognitive mechanism known as common coding (which proposes a common neural representation connecting execution, perception and imagination of movements), and outline how this mechanism could provide an integrated account of the cognitive effects of video games. We then illustrate, using two ongoing projects, how novel video game interaction designs could be derived by extending common coding theory.

[1]  Bruno Galantucci,et al.  An Experimental Study of the Emergence of Human Communication Systems , 2005, Cogn. Sci..

[2]  Per Capita,et al.  About the authors , 1995, Machine Vision and Applications.

[3]  W. Prinz,et al.  The imitative mind : development, evolution, and brain bases , 2002 .

[4]  P. Johnson-Laird,et al.  Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness , 1985 .

[5]  Romeo Chua,et al.  Perceptual-Motor Interaction: Some Implications For Hci , 2007 .

[6]  Sanjay Chandrasekharan,et al.  The Origin of Epistemic Structures and Proto-Representations , 2007, Adapt. Behav..

[7]  Carrie Heater,et al.  Being There: The Subjective Experience of Presence , 1992, Presence: Teleoperators & Virtual Environments.

[8]  D. Elliott,et al.  Movement Trajectories in the Presence of a Distracting Stimulus: Evidence for a Response Activation Model of Selective Reaching , 2004, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[9]  Sian L. Beilock,et al.  Expertise and its embodiment: Examining the impact of sensorimotor skill expertise on the representation of action-related text , 2006, Psychonomic bulletin & review.

[10]  Sanjay Chandrasekharan,et al.  Building to Discover: A Common Coding Model , 2009, Cogn. Sci..

[11]  Michael A. Arbib,et al.  Mirror neurons and imitation: A computationally guided review , 2006, Neural Networks.

[12]  J. Cutting,et al.  Recognizing the sex of a walker from a dynamic point-light display , 1977 .

[13]  Paul Clifton,et al.  Recognizing self in puppet controlled virtual avatars , 2010, Fun and Games '10.

[14]  Günther Knoblich,et al.  The Social Nature of Perception and Action , 2006 .

[15]  S. M. Hansen.  Where the action is. , 1985, Journal (National Association for Hospital Development (U.S.)).

[16]  Alain Berthoz,et al.  Walking on a line: A motor paradigm using rotation and reflection symmetry to study mental body transformations , 2009, Brain and Cognition.

[17]  J. Pratt,et al.  Playing an Action Video Game Reduces Gender Differences in Spatial Cognition , 2007, Psychological science.

[18]  G. Rizzolatti,et al.  Motor facilitation during action observation: a magnetic stimulation study. , 1995, Journal of neurophysiology.

[19]  H. Ehrsson.  The Experimental Induction of Out-of-Body Experiences , 2007, Science.

[20]  C. S. Green,et al.  Action video game modifies visual selective attention , 2003, Nature.

[21]  W. Prinz,et al.  Movement observation affects movement execution in a simple response task. , 2001, Acta psychologica.

[22]  Frank J. Bernieri,et al.  Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream , 2000 .

[23]  A. Goldman,et al.  The Shared Circuits Model: How Control, Mirroring and Simulation Can Enable Imitation, Deliberation, and Mindreading , 2007 .

[24]  A. Wohlschläger.  Mental object rotation and the planning of hand movements , 2001, Perception & psychophysics.

[25]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[26]  N. Katherine Hayles,et al.  How we became posthuman , 1999 .

[27]  D. Spalding.  The Principles of Psychology , 1873, Nature.

[28]  P Viviani,et al.  Perceptual anticipation in handwriting: The role of implicit motor competence , 2000, Perception & psychophysics.

[29]  J. Tambling.  How We Became Posthuman: Virtual Bodies in Cybernetics, Literature and Informatics by N. Katherine Hayles (review) , 2001, Modern Language Review.

[30]  Brenda Laurel,et al.  Computers as theatre , 1991 .

[31]  Cecilia Heyes,et al.  Imitation by association , 2005 .

[32]  M. Jeannerod.  The cognitive neuroscience of action , 1997, Trends in Cognitive Sciences.

[33]  Paul Clifton,et al.  Giving your self to the game: transferring a player's own movements to avatars using tangible interfaces , 2009, Sandbox@SIGGRAPH.

[34]  M. Brass,et al.  Imitation: is cognitive neuroscience solving the correspondence problem? , 2005, Trends in Cognitive Sciences.

[35]  Kerri L. Johnson,et al.  Swagger, sway, and sexuality: Judging sexual orientation from body motion and morphology. , 2007, Journal of personality and social psychology.

[36]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[37]  Jaime G. Carbonell,et al.  Interactive drama, art and artificial intelligence , 2002 .

[38]  William V. Wright,et al.  A Theory of Fun for Game Design , 2004 .

[39]  G. Rizzolatti,et al.  Visuomotor neurons: ambiguity of the discharge or 'motor' perception? , 2000, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[40]  A. J. Dijksterhuis,et al.  Why are we social animals: The high road to imitation as social glue , 2005 .

[41]  F. Strack,et al.  Motormouth: mere exposure depends on stimulus-specific motor simulations. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[42]  G. Aschersleben,et al.  The Theory of Event Coding (TEC): a framework for perception and action planning. , 2001, The Behavioral and brain sciences.

[43]  Katherine Isbister.  Better Game Characters by Design - A Psychological Approach , 2006, The Morgan Kaufmann series in interactive 3D technology.

[44]  G. Miller,et al.  Cognitive science. , 1981, Science.

[45]  P. Viviani,et al.  Biological movements look uniform: evidence of motor-perceptual interactions. , 1992, Journal of experimental psychology. Human perception and performance.

[46]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[47]  J. Cutting,et al.  Recognizing friends by their walk: Gait perception without familiarity cues , 1977 .

[48]  S. Turkle.  Life on the Screen: Identity in the Age of the Internet , 1997 .

[49]  Floris P. de Lange,et al.  Motor Imagery in Mental Rotation: An fMRI Study , 2002, NeuroImage.

[50]  M. Hegarty.  Mechanical reasoning by mental simulation , 2004, Trends in Cognitive Sciences.

[51]  S Hurley,et al.  Perspectives on imitation: From neuroscience to social science. Imitation, human development and culture. , 2005 .

[52]  Pablo Neruda,et al.  Rethinking Situatedness : Environment Structure in the Time of the Common Code , 2010 .

[53]  G. Rizzolatti,et al.  Speech listening specifically modulates the excitability of tongue muscles: a TMS study , 2002, The European journal of neuroscience.

[54]  Daniel L. Schwartz,et al.  Physically Distributed Learning: Adapting and Reinterpreting Physical Environments in the Development of Fraction Concepts , 2005, Cogn. Sci..

[55]  Sian L. Beilock,et al.  Embodied Preference Judgments , 2007, Psychological science.

[56]  Moore Hall Honolulu.  Simulated Action in an Embodied Construction Grammar , 2004 .

[57]  J. N. Bassili.  Facial motion in the perception of faces and of emotional expression. , 1978, Journal of experimental psychology. Human perception and performance.

[58]  S. Aglioti,et al.  Selective impairment of hand mental rotation in patients with focal hand dystonia. , 2005, Brain : a journal of neurology.

[59]  P. Viviani,et al.  The effect of movement velocity on form perception: Geometric illusions in dynamic displays , 1989, Perception & psychophysics.

[60]  Wolfgang Prinz,et al.  Why don't we perceive our brain states? , 1992 .

[61]  Nancy J. Nersessian,et al.  Creating Scientific Concepts , 2008 .

[62]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[63]  T. Metzinger,et al.  Video Ergo Sum: Manipulating Bodily Self-Consciousness , 2007, Science.

[64]  Vittorio Gallese,et al.  The eyes, the hand, and the mind: behavioral and neurophysiological aspects of social cognition , 2001 .

[65]  N. Nersessian.  The Cognitive Basis of Science: The cognitive basis of model-based reasoning in science , 2002 .

[66]  Chris McNorgan,et al.  An attractor model of lexical conceptual processing: simulating semantic priming , 1999, Cogn. Sci..

[67]  B. Rothbaum,et al.  Virtual reality exposure therapy and standard (in vivo) exposure therapy in the treatment of fear of flying. , 2006, Behavior therapy.

[68]  Mark Wexler,et al.  Depth perception by the active observer , 2005, Trends in Cognitive Sciences.

[69]  Paul Dourish,et al.  Where the action is , 2001 .

[70]  Raymond W. Gibbs,et al.  Real and Imagined Body Movement Primes Metaphor Comprehension , 2007, Cogn. Sci..

[71]  Romeo Chua,et al.  Within- and between-nervous-system inhibition of return: Observation is as good as performance , 2007, Psychonomic bulletin & review.