Towards methods for evaluating and communicating participatory design: A multimodal approach

Participatory Design (PD) has been proposed as a useful strategy to address pitfalls in the design of serious games for children with special needs. Nonetheless, methodological weaknesses in the analysis of the results of PD workshops may hinder its effectiveness in providing useful and robust design contributions and facilitating communication in multidisciplinary teams. To address this issue we propose the use of multimodal analysis to evaluate participants' contributions during PD workshops. We present an analysis that was applied in an informant workshop with autistic children, aimed at refining the design of a serious game based on Full-Body Interaction. Results show that multimodal analysis constitutes an effective and coherent method to capture and analyze users' contributions across a wide range of semiotic resources, thus extending the richness of insights that can be derived from a PD workshop and communicated to the rest of the team. Furthermore, the analysis allowed the identification of fundamental design questions, thus offering a robust empirical ground for supporting dialogue and reflection between multiple stakeholders. Use of Multimodal Analysis to analyze users' contributions in Participatory Design.Case study with 4 children in Autistic Spectrum Disorder.Multimodal analysis effectively grasped contributions across different resources.Usefulness for guiding design of Serious Games in interdisciplinary teams.

[1]  J. Bezemer.  Displaying Orientation in the Classroom: Students' Multimodal Responses to Teacher Instructions. , 2008 .

[2]  Manfred Tscheligi,et al.  Around the world in 8 workshops: investigating anticipated player experiences of children , 2014, CHI PLAY.

[3]  R. Poldrack,et al.  Reward processing in autism , 2010, Autism research : official journal of the International Society for Autism Research.

[4]  Panos Markopoulos,et al.  Head Up Games: combining the best of both worlds by merging traditional and digital play , 2010, Personal and Ubiquitous Computing.

[5]  Yvonne Rogers,et al.  Kids as informants: telling us what we didn't know or confirming what we knew already? , 1998 .

[6]  Laura Malinverni,et al.  Learning of Abstract Concepts through Full-Body Interaction: A Systematic Review , 2014, J. Educ. Technol. Soc..

[7]  Simon McCallum,et al.  Emotion Recognition for Mobile Devices with a Potential Use in Serious Games for Autism Spectrum Disorder , 2013, SGDA.

[8]  R. Breckinridge Church,et al.  Using Gesture and Speech to Capture Transitions in Learning , 1999 .

[9]  M. Hoepfl.  Choosing Qualitative Research: A Primer for Technology Education Researchers , 1997 .

[10]  C. Goodwin.  Action and embodiment within situated human interaction , 2000 .

[11]  Elizabeth Foss,et al.  Layered elaboration: a new technique for co-design with children , 2010, CHI.

[12]  Karin Slegers,et al.  Multimodal Analysis in Participatory Design with Children: A Primary School Case Study , 2015, CHI.

[13]  Andrew Large,et al.  Children in the information technology design process: A review of theories and their applications , 2004 .

[14]  Lorenza Mondada,et al.  Understanding as an embodied, situated and sequential achievement in interaction , 2011 .

[15]  Dion Hoe-Lian Goh,et al.  Strategies for designing effective psychotherapeutic gaming interventions for children and adolescents , 2008, Comput. Hum. Behav..

[16]  J. Einarsdóttir,et al.  Playschool in pictures: children’s photographs as a research method , 2005 .

[17]  Douglas A. Harper,et al.  Talking about pictures: A case for photo elicitation , 2002 .

[18]  Carey Jewitt,et al.  Multimodal methods for researching digital technologies , 2013 .

[19]  Hilary Johnson,et al.  Diversity for design: a framework for involving neurodiverse children in the technology design process , 2014, CHI.

[20]  C. Jewitt.  The Routledge handbook of multimodal analysis , 2014 .

[21]  Elisa Giaccardi,et al.  Embodied narratives: a performative co-design technique , 2012, DIS '12.

[22]  Anne Marie Piper,et al.  SIDES: A Cooperative Tabletop Computer Game for Social Skills Development , 2022 .

[23]  Eynat Gal,et al.  To enforce or not to enforce? The use of collaborative interfaces to promote social skills in children with high functioning autism spectrum disorder , 2013, Autism : the international journal of research and practice.

[24]  Mirjam Hauck,et al.  What are multimodal data and transcription , 2013 .

[25]  Juliane Jung,et al.  The Practice Of Everyday Life , 2016 .

[26]  Sue Cobb,et al.  Participatory design approach with children with autism , 2011 .

[27]  Sara Price,et al.  The role of touch in preschool children’s learning using iPad versus paper interaction , 2014, The Australian Journal of Language and Literacy.

[28]  Ling Xia,et al.  Eye tracking and online search: Lessons learned and challenges ahead , 2008, J. Assoc. Inf. Sci. Technol..

[29]  Christopher Frauenberger,et al.  Conversing through and about technologies: Design critique as an opportunity to engage children with autism and broaden research(er) perspectives , 2013, Int. J. Child Comput. Interact..

[30]  J. Hindmarsh,et al.  The Tacit Order of Teamwork: Collaboration and Embodied Conduct in Anesthesia , 2002 .

[31]  Anibal Gutierrez,et al.  Comparative Efficacy of LEAP, TEACCH and Non-Model-Specific Special Education Programs for Preschoolers with Autism Spectrum Disorders , 2014, Journal of autism and developmental disorders.

[32]  D Strickland,et al.  Virtual reality for the treatment of autism. , 1997, Studies in health technology and informatics.

[33]  K. Squire,et al.  Design-Based Research: Putting a Stake in the Ground , 2004 .

[34]  Feng Wang,et al.  Design-based research and technology-enhanced learning environments , 2005 .

[35]  Yvonne Rogers,et al.  Does he take sugar?: moving beyond the rhetoric of compassion , 2013, INTR.

[36]  B. Serrell.  Paying Attention: The Duration and Allocation of Visitors' Time in Museum Exhibitions , 1997 .

[37]  Janet C. Read,et al.  Integrating children's contributions in the interaction design process , 2012, Int. J. Arts Technol..

[38]  Michael J. Muller,et al.  Participatory design: the third space in HCI , 2002 .

[39]  Matthew S. Goodwin,et al.  Exploring speech therapy games with children on the autism spectrum , 2009, INTERSPEECH.

[40]  U. Frith,et al.  The Weak Coherence Account: Detail-focused Cognitive Style in Autism Spectrum Disorders , 2006, Journal of autism and developmental disorders.

[41]  Andy Clark,et al.  Embodiment and the Philosophy of Mind , 1998, Royal Institute of Philosophy Supplement.

[42]  Sara Price,et al.  Animated Diagrams: How effective are explicit dynamics for learners? , 2002 .

[43]  Sara Price,et al.  A multimodal approach to examining 'embodiment' in tangible learning environments , 2013, TEI '13.

[44]  A. Jaworski,et al.  Gesture and movement in tourist spaces , 2009 .

[45]  Feng Yan,et al.  A SUNNY DAY: Ann and Ron's World an iPad Application for Children with Autism , 2011, SGDA.

[46]  T. F. McLaughlin,et al.  EDUCATIONAL GAMES: A TECHNIQUE TO ACCELERATE THE ACQUISITION OF READING SKILLS OF CHILDREN WITH LEARNING DISABILITIES , 2005 .

[47]  Susan Cox,et al.  Intention and meaning in young children's drawing , 2005 .

[48]  Laura Malinverni,et al.  Narrative-based elicitation: orchestrating contributions from experts and children , 2014, CHI Extended Abstracts.

[49]  Christopher Frauenberger,et al.  Supporting the design contributions of children with autism spectrum conditions , 2012, IDC '12.

[50]  Allison Druin,et al.  The role of children in the design of new technology , 2002 .

[51]  Susan Goldin-Meadow,et al.  Learning through gesture. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[52]  J. Fuentes-Biggi,et al.  Guía de buena práctica para el tratamiento de los trastornos del espectro autista , 2006 .

[53]  Christina Whalen,et al.  Efficacy of TeachTown: Basics computer-assisted intervention for the Intensive Comprehensive Autism Program in Los Angeles Unified School District , 2010, Autism : the international journal of research and practice.

[54]  Lennart E. Nacke,et al.  Gamification : Toward a Definition , 2022 .

[55]  M. Rutter,et al.  The role of eye contact in goal detection: Evidence from normal infants and children with autism or mental handicap , 1992, Development and Psychopathology.

[56]  Steve Harrison,et al.  Making epistemological trouble: Third-paradigm HCI as successor science , 2011, Interact. Comput..

[57]  C. Frauenberger,et al.  Designing technology for children with special needs: bridging perspectives through participatory design , 2011 .

[58]  A. Clark,et al.  Listening to and involving young children: a review of research and practice , 2005 .

[59]  Sue Cobb,et al.  Participatory design with children with autism , 2010 .

[60]  Liyan Song,et al.  Digital Game-Based Learning , 2014 .

[61]  C. Jewitt.  Different approaches to multimodality , 2009 .

[62]  Caroline Hummels,et al.  Move to get moved: a search for methods, tools and knowledge to design for expressive and rich movement-based interaction , 2007, Personal and Ubiquitous Computing.

[63]  Laura Malinverni,et al.  Participatory design methods to define educational goals for full-body interaction , 2014, Advances in Computer Entertainment.

[64]  Eleonora Bilotta,et al.  An emotional learning environment for subjects with Autism Spectrum Disorder , 2013, 2013 International Conference on Interactive Collaborative Learning (ICL).

[65]  Laura Malinverni,et al.  Participatory design strategies to enhance the creative contribution of children with special needs , 2014, IDC.

[66]  Jane Jorgenson,et al.  Accessing Children's Perspectives Through Participatory Photo Interviews , 2009 .

[67]  John P Rae,et al.  The interactional use of eye-gaze in children with autism spectrum disorders , 2014 .

[68]  Gaston Bachelard,et al.  The Formation of the Scientific Mind: A Contribution to a Psychoanalysis of Objective Knowledge , 2002 .

[69]  Christopher Frauenberger,et al.  In pursuit of rigour and accountability in participatory design , 2015, Int. J. Hum. Comput. Stud..

[70]  Charles Goodwin,et al.  Embodied Interaction in the Material World: An , 2022 .

[71]  Charles Goodwin,et al.  Embodied Interaction: Language And Body In The Material World , 2014 .

[72]  M. Resnick.  Rethinking Learning in the Digital Age , 2002 .