A Method Impact Assessment Framework for User Experience Evaluations with Children

Based upon a review of the literature, this paper presents a Method Impact Assessment Framework. Theoretically synthesized, the framework offers five dimensions: (1) the role of the child, (2) the user experience construct, (3) system, (4) epistemological perspective, (5) Practical and Ethical Concerns. Although other dimensions could have been construed, these were judged to be the most pertinent to understanding evaluation methods with children. The framework thus provides a critical lens in which evaluation methods can be assessed by the Children Computer Interaction (CCI) Community to inform method selection.

[1]  Sheila Greene,et al.  Exploring Meaning in Interviews with Children , 2005 .

[2]  Mary Kellett,et al.  ‘Just teach us the skills please, we'll do the rest’: empowering ten‐year‐olds as active researchers , 2004 .

[3]  Gavin Sim,et al.  Empowering Teenagers to Perform a Heuristic Evaluation of a Game , 2014, BCS HCI.

[4]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[5]  Shaowen Bardzell,et al.  Interaction criticism: a proposal and framework for a new discipline of hci , 2008, CHI Extended Abstracts.

[6]  Gerison Lansdown,et al.  Promoting Children's Participation in Democratic Decision-Making , 2001 .

[7]  Wolmet Barendregt,et al.  Identifying usability and fun problems in a computer game during first use and after some practice , 2006, Int. J. Hum. Comput. Stud..

[8]  Bieke Zaman,et al.  A value sensitive design approach to parental software for young children , 2015, IDC.

[9]  Robert C. Williges,et al.  Criteria For Evaluating Usability Evaluation Methods , 2003, Int. J. Hum. Comput. Interact..

[10]  David Gauntlett.  Creative Explorations: New Approaches to Identities and Audiences , 2007 .

[11]  Marshall W. Fishwick,et al.  Emotional Design: Why We Love (or Hate) Everyday Things , 2004 .

[12]  Gavin Sim,et al.  Investigating children's opinions of games: Fun Toolkit vs. This or That , 2012, IDC '12.

[13]  Malcolm Hill,et al.  Children’s Voices on Ways of Having a Voice , 2006 .

[14]  Jodi Forlizzi,et al.  Understanding experience in interactive systems , 2004, DIS '04.

[15]  Sarah Diefenbach,et al.  Needs, affect, and interactive products - Facets of user experience , 2010, Interact. Comput..

[16]  J. Russell.  Core affect and the psychological construction of emotion. , 2003, Psychological review.

[17]  Marc Hassenzahl,et al.  The Thing and I: Understanding the Relationship Between User and Product , 2005, Funology.

[18]  Jeffrey Bardzell.  Critical and Cultural Approaches to HCI , 2013 .

[19]  M. Gallagher,et al.  Methodological Immaturity in Childhood Research? , 2008 .

[20]  Gaetano Cascini,et al.  A framework for user experience, needs and affordances , 2014 .

[21]  P. Desmet,et al.  Framework of product experience , 2007 .

[22]  John C. McCarthy,et al.  Technology as experience , 2004, INTR.

[23]  E. Reed.  The Ecological Approach to Visual Perception , 1989 .

[24]  Hugh McLaughlin,et al.  Involving Young Service Users as Co-Researchers: Possibilities, Benefits and Costs , 2005 .

[25]  Horst Niesyto.  Youth research on video self‐productions reflections on a social‐aesthetic approach , 2000 .

[26]  Panos Markopoulos,et al.  On the assessment of usability testing methods for children , 2003, Interact. Comput..

[27]  Kori Inkpen,et al.  Three Important Research Agendas for Educational Multimedia: Learning, Children, and Gender , 1998 .

[28]  V. Morrow,et al.  The Ethics of Social Research with Children: An Overview† , 1996 .

[29]  Bieke Zaman,et al.  MemoLine: evaluating long-term UX with children , 2013, IDC.

[30]  Stuart MacFarlane,et al.  Using the fun toolkit and other survey methods to gather opinions in child computer interaction , 2006, IDC '06.

[31]  R. Hart.  Children's Participation: From Tokenism to Citizenship. Innocenti Essays No. 4. , 1992 .

[32]  Kirsten Risden,et al.  Guidelines for usability testing with children , 1997, INTR.

[33]  H. Freud.  Emotional Design Why We Love Or Hate Everyday Things , 2016 .

[34]  Manfred Tscheligi,et al.  A Formal Analysis of the ISO 9241-210 Definition of User Experience , 2015, CHI Extended Abstracts.

[35]  J. Read,et al.  Endurability, Engagement and Expectations: Measuring Children’s Fun , 2002 .

[36]  Virpi Roto,et al.  UX Curve: A method for evaluating long-term user experience , 2011, Interact. Comput..

[37]  Kasper Hornbæk,et al.  Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience , 2011, CHI.

[38]  A. Bell.  Designing and testing questionnaires for children , 2007 .

[39]  Vero Vanden Abeele,et al.  Measuring product liking in preschool children: An evaluation of the Smileyometer and This or That methods , 2013, Int. J. Child Comput. Interact..

[40]  J. Alexander,et al.  ‘Draw, write and tell’: A literature review and methodological development on the ‘draw and write’ research method , 2015 .

[41]  Simon Davis,et al.  A Systematic Review of Cybersickness , 2014, IE.

[42]  Virpi Roto,et al.  Understanding, scoping and defining user experience: a survey approach , 2009, CHI.

[43]  Thomas D. Lyon,et al.  Child Witnesses and the Oath , 2002 .

[44]  Greg Corness,et al.  Springboard: Designing Image Schema Based Embodied Interaction for an Abstract Domain , 2011, Whole Body Interaction.

[45]  Vero Vanden Abeele,et al.  Laddering with young children in User eXperience evaluations: theoretical groundings and a practical case , 2010, IDC.

[46]  Wolmet Barendregt,et al.  Development and evaluation of the problem identification picture cards method , 2008, Cognition, Technology & Work.

[47]  Janet C. Read.  Evaluating artefacts with children: age and technology effects in the reporting of expected and experienced fun , 2012, ICMI '12.

[48]  Janet C. Read,et al.  PLU-E: a proposed framework for planning and conducting evaluation studies with children , 2011, BCS HCI.

[49]  D. Kahneman,et al.  Well-being : the foundations of hedonic psychology , 1999 .

[50]  Alissa Nicole Antle,et al.  Child-based personas: need, ability and experience , 2008, Cognition, Technology & Work.

[51]  K. Hammond,et al.  Ethical Guidelines for Educational Research in the ELP , 2005 .

[52]  Sharlene Hesse-Biber,et al.  Handbook of emergent methods , 2008 .

[53]  Heidi Krömker,et al.  A framework to measure user experience of interactive online products , 2010, MB '10.

[54]  Panos Markopoulos,et al.  Assessing the applicability of the structured expert evaluation method (SEEM) for a wider age group , 2006, IDC '06.

[55]  Bieke Zaman.  Introduction and validation of a pairwise comparison scale for UX evaluations and benchmarking with preschoolers , 2009 .

[56]  Thomas W. Malone,et al.  Heuristics for designing enjoyable user interfaces: Lessons from computer games , 1982, CHI '82.

[57]  Gavin Sim,et al.  Evaluating Usability, Fun and Learning in Educational Software for Children , 2005 .

[58]  Janet C. Read,et al.  CHECk: a tool to inform and encourage ethical practice in participatory design with children , 2013, CHI Extended Abstracts.

[59]  João Barreiros,et al.  Adults' perception of children's height and reaching capability. , 2010, Acta psychologica.

[60]  Andrew Chin,et al.  UN Convention on the Rights of the Child , 1990 .

[61]  Michael G. Wyness.  Children’s participation and intergenerational dialogue: Bringing adults back into the analysis , 2013 .

[62]  Virpi Roto,et al.  Towards a shared definition of user experience , 2008, CHI Extended Abstracts.

[63]  Jodi Forlizzi,et al.  The Product Ecology: Understanding Social Product Use and Supporting Design Culture , 2008 .

[64]  Allison Druin,et al.  The design of children's technology , 1998 .

[65]  Samantha Punch,et al.  Research with Children , 2002 .