Developing a pedagogical framework for designing a multisensory serious gaming environment

The importance of multisensory interaction for learning has increased with improved understanding of children’s sensory development, and a flourishing interest in embodied cognition. The potential to foster new forms of multisensory interaction through various sensor, mobile and haptic technologies is promising in providing new ways for young children to engage with key mathematical concepts. However, designing effective learning environments for real world classrooms is challenging, and requires a pedagogically, rather than technologically, driven approach to design. This paper describes initial work underpinning the development of a pedagogical framework, intended to inform the design of a multisensory serious gaming environment. It identifies the theoretical basis of the framework, illustrates how this informs teaching strategies, and outlines key technology research driven perspectives and considerations important for informing design. An initial table mapping mathematical concepts to design, a framework of considerations for design, and a process model of how the framework will continue to be developed across the design process are provided.

[1]  Michael F. Young,et al.  Our Princess Is in Another Castle , 2012 .

[2]  Jennifer Suh,et al.  Developing Students’ Representational Fluency Using Virtual and Physical Algebra Balances , 2007 .

[3]  Dor Abrahamson,et al.  The Enactive Roots of STEM: Rethinking Educational Design in Mathematics , 2015 .

[4]  Sam Sellar,et al.  Framing pedagogies: A review of frameworks and research designed to promote effective approaches to teaching and learning. Report for Department of Education and Children's Services, South Australia. , 2007 .

[5]  Jasmine Y. Ma.  Multi-Party, Whole-Body Interactions in Mathematical Activity , 2017 .

[6]  Yvonne Rogers,et al.  Collaboration and interference: awareness with mice or touch input , 2008, CSCW.

[7]  Sara Price,et al.  Action and representation in tangible systems: implications for design of learning interactions , 2010, TEI '10.

[8]  Maurice Merleau-Ponty.  Phenomenology of Perception , 1964 .

[9]  Dennis Charsky,et al.  From Edutainment to Serious Games: A Change in the Use of Game Characteristics , 2010, Games Cult..

[10]  James C. Lester,et al.  Enhancing 5th graders' science content knowledge and self-efficacy through game-based learning , 2011, Comput. Educ..

[11]  Celia Hoyles,et al.  Building mathematical knowledge with programming: insights from the ScratchMaths project , 2016 .

[12]  Jacob Buur,et al.  Getting a grip on tangible interaction: a framework on physical space and social interaction , 2006, CHI.

[13]  Terry Winograd,et al.  Understanding computers and cognition - a new foundation for design , 1987 .

[14]  Dor Abrahamson,et al.  Toward an embodied-interaction design framework for mathematical concepts , 2011, IDC.

[15]  Barbara King,et al.  Learning angles through movement: Critical actions for developing understanding in an embodied activity , 2014 .

[16]  S. Gallagher.  How the body shapes the mind , 2005 .

[17]  Scott R. Klemmer,et al.  How bodies matter: five themes for interaction design , 2006, DIS '06.

[18]  Barbara Tversky,et al.  Visualizing space, time, and agents: production, performance, and preference , 2011, Cognitive Processing.

[19]  P. Vorderer,et al.  Serious games : mechanisms and effects , 2009 .

[20]  Monica Gori,et al.  WeDRAW: using multisensory serious games to explore concepts in primary mathematics , 2017 .

[21]  Mijung Kim,et al.  CHILDREN’S GESTURES AND THE EMBODIED KNOWLEDGE OF GEOMETRY , 2011 .

[22]  M. Csíkszentmihályi,et al.  The Concept of Flow , 2014 .

[23]  S. Goldin-Meadow,et al.  Gesturing Gives Children New Ideas About Math , 2009, Psychological science.

[24]  M. Besson,et al.  Music and Dyslexia: A New Musical Training Method to Improve Reading and Related Disorders , 2016, Front. Psychol..

[25]  Paul Dourish,et al.  Where the action is , 2001 .

[26]  L. Barsalou.  Grounded cognition. , 2008, Annual review of psychology.

[27]  Sara Price,et al.  Designing for physical-digital correspondence in tangible learning environments , 2009, IDC.

[28]  B. Bederson,et al.  Designing storytelling technologies to encouraging collaboration between young children , 2000, CHI.

[29]  C. O'Malley,et al.  The Effects of Physical Manipulatives on Children's Numerical Strategies , 2016 .

[30]  David W. Johnson,et al.  Learning Together and Alone. Cooperative, Competitive, and Individualistic Learning. Fourth Edition. , 1991 .

[31]  Kai Pata,et al.  Pedagogy-driven design of digital learning ecosystems , 2014, Comput. Sci. Inf. Syst..

[32]  David Starr-Glass,et al.  Participation in Online Distance Learning Environments: Proxy, Sign, or a Means to an End? , 2016 .

[33]  Andrea A. Anderson,et al.  The Community Builder's Approach to Theory of Change , 2006 .

[34]  Alissa Nicole Antle,et al.  Button Matrix: How Tangible Interfaces can Structure Physical Experiences for Learning , 2015, TEI.

[35]  Ella Shoval,et al.  Using mindful movement in cooperative learning while learning about angles , 2011 .

[36]  J. Decety,et al.  The power of simulation: Imagining one's own and other's behavior , 2006, Brain Research.

[37]  David C. Burr,et al.  Young Children Do Not Integrate Visual and Haptic Form Information , 2008, Current Biology.

[38]  G. Lakoff,et al.  Where mathematics comes from : how the embodied mind brings mathematics into being , 2002 .

[39]  Geetha B. Ramani,et al.  Playing linear numerical board games promotes low-income children's numerical development. , 2008, Developmental science.

[40]  Iman C. Chahine,et al.  The impact of using multiple modalities on students’ acquisition of fractional knowledge: An international study in embodied mathematics across semiotic cultures , 2013 .

[41]  Marc Prensky,et al.  Digital game-based learning , 2000, CIE.

[42]  Alissa Nicole Antle,et al.  The CTI framework: informing the design of tangible systems for children , 2007, TEI.

[43]  Dor Abrahamson,et al.  Learning Is Moving in New Ways: The Ecological Dynamics of Mathematics Education , 2016 .

[44]  Daniel L. Schwartz,et al.  Physically Distributed Learning: Adapting and Reinterpreting Physical Environments in the Development of Fraction Concepts , 2005, Cogn. Sci..

[45]  Barbara L. Grabowski,et al.  Gameplaying for maths learning: cooperative or not? , 2007, Br. J. Educ. Technol..

[46]  Luis Mateus Rocha,et al.  Introduction to the Special Issue: Embodied and Situated Cognition , 2005, Artificial Life.

[47]  M. Gori.  Multisensory Integration and Calibration in Children and Adults with and without Sensory and Motor Disabilities. , 2015, Multisensory research.

[48]  Sara Price,et al.  Mobile Experiences of Historical Place: A Multimodal Analysis of Emotional Engagement , 2016 .

[49]  Harold Bekkering,et al.  Spatial Interferences in Mental Arithmetic: Evidence from the Motion–Arithmetic Compatibility Effect , 2014, Quarterly journal of experimental psychology.

[50]  A. Gagatsis,et al.  The role of gestures in making connections between space and shape aspects and their verbal representations in the early years: findings from a case study , 2014 .

[51]  Pierre Dillenbourg,et al.  Tangible vs. virtual representations: when tangibles benefit the training of spatial skills , 2012, NordiCHI.

[52]  Sara Price,et al.  A representation approach to conceptualizing tangible learning environments , 2008, TEI.

[53]  Mitchell J. Nathan,et al.  Embodiment in Mathematics Teaching and Learning: Evidence From Learners' and Teachers' Gestures , 2012 .

[54]  Fernando Almeida e Costa,et al.  Embodied and Situated Cognition , 2005 .

[55]  Michael Gasser,et al.  The Development of Embodied Cognition: Six Lessons from Babies , 2005, Artificial Life.

[56]  F. Garbarini,et al.  At the root of embodied cognition: Cognitive science meets neurophysiology , 2004, Brain and Cognition.

[57]  Eva Hornecker,et al.  Of pages and paddles: Children's expectations and mistaken interactions with physical-digital tools , 2009, Interact. Comput..

[58]  A. Clark,et al.  The Extended Mind , 1998, Analysis.

[59]  Yeonjeong Park,et al.  A Pedagogical Framework for Mobile Learning: Categorizing Educational Applications of Mobile Technologies into Four Types. , 2011 .

[60]  Roy D. Pea,et al.  A walk on the WILD side How wireless handhelds may change computer-supported collaborative learning , 2002 .

[61]  A. Nicolopoulou,et al.  Play, Cognitive Development, and the Social World: Piaget, Vygotsky, and Beyond , 1993 .

[62]  Markus Hohenwarter,et al.  Combination of dynamic geometry, algebra and calculus in the software system GeoGebra , 2004 .

[63]  Timothy C. Bell,et al.  Enthusing & inspiring with reusable kinaesthetic activities , 2009, ITiCSE.

[64]  Margaret Wilson,et al.  Six views of embodied cognition , 2002, Psychonomic bulletin & review.

[65]  Chris Rasmussen,et al.  When the Classroom Floor Becomes the Complex Plane: Addition and Multiplication as Ways of Bodily Navigation , 2012 .

[66]  J. Piaget,et al.  The principles of genetic epistemology , 1972 .

[67]  L. S. Vygotksy.  Mind in society: the development of higher psychological processes , 1978 .

[68]  Autumn B. Hostetter,et al.  Visible embodiment: Gestures as simulated action , 2008, Psychonomic bulletin & review.

[69]  Christian Heath,et al.  Configuring 'Interactivity' , 2008 .

[70]  Wilfried Admiraal,et al.  Mobile game-based learning in secondary education: engagement, motivation and learning in a mobile city game , 2009, J. Comput. Assist. Learn..