Measuring product liking in preschool children: An evaluation of the Smileyometer and This or That methods

Abstract This article provides a detailed evaluation of two summative evaluation methods, This or That and Smileyometer. The methods’ reliability and validity were examined, using a sample of 113 preschoolers (ages from 33 to 90 months). The results show considerable promise for This or That for measuring preferences across entertainment products. This or That resulted in reliable and valid responses in preschool children aged four and older. Nevertheless, low reliability and validity scores were obtained for This or That in preschoolers younger than four years old. As for the Smileyometer results, preschoolers’ responses clearly indicated an overrepresentation of extreme positive scores that were inconsistent with their actual product preferences. Consequently, the validity of these young children’s responses evoked through the Smileyometer was not supported.

[1]  S. Denham,et al.  Mother-child dialogue about emotions and preschoolers' emotional competence. , 1995, Genetic, social, and general psychology monographs.

[2]  H. Sweeting,et al.  Children, Health and Social Order (Book). , 1997 .

[3]  Gavin Sim,et al.  Investigating children's opinions of games: Fun Toolkit vs. This or That , 2012, IDC '12.

[4]  Allison Druin,et al.  The design of children's technology , 1998 .

[5]  M. Hayes.  Research with children: perspectives and practices , 2001 .

[6]  Jean-Xavier Guinard,et al.  Sensory and consumer testing with children , 2000 .

[7]  R. Popper,et al.  Conducting Sensory Research with Children , 2005 .

[8]  J. Jacko,et al.  The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications , 2002 .

[9]  Janet C. Read,et al.  Evaluating Children's Interactive Products: Principles and Practices for Interaction Designers , 2008 .

[10]  Sara R. Jaeger,et al.  Comparison of five common acceptance and preference methods , 2008 .

[11]  V. Morrow,et al.  The Ethics of Social Research with Children: An Overview† , 1996 .

[12]  P. Darbyshire,et al.  Multiple methods in qualitative research with children: more insight or just more? , 2005 .

[13]  Bojana Lobe.  Integration of Online Research Methods , 2011 .

[14]  Howard B. Lee,et al.  A first course in factor analysis , 1973 .

[15]  P. Vorderer,et al.  Enjoyment: At the Heart of Media Entertainment , 2004 .

[16]  Nicky Britten,et al.  Hearing children's voices: methodological issues in conducting focus groups with children aged 7-11 years , 2002 .

[17]  Richard L. Hazlett,et al.  Measuring emotional valence to understand the user's experience of software , 2007, Int. J. Hum. Comput. Stud..

[18]  Bieke Zaman,et al.  Introducing a Pairwise Comparison Scale for UX Evaluations with Preschoolers , 2009, INTERACT.

[19]  Juan Pablo Hourcade,et al.  Interaction Design and Children , 2008, Found. Trends Hum. Comput. Interact..

[20]  Sarah J Neill,et al.  Research with children: a critical review of the guidelines , 2005, Journal of child health care : for professionals working with children in the hospital and community.

[21]  Seymour Sudman,et al.  Response Effects in Surveys: A Review and Synthesis , 1974 .

[22]  P. H. Miller,et al.  Young children's understanding of the psychological causes of behavior: A review. , 1989 .

[23]  J. Bryant,et al.  Playing video games: Motives, responses, and consequences. , 2012 .

[24]  Karen A. Machleit,et al.  Measuring preschool children's attitude , 1990 .

[25]  Leslie Haddon,et al.  Researching children's experiences: online across countries:issues and problems in methodology , 2007 .

[26]  J. Read,et al.  Endurability, Engagement and Expectations: Measuring Children’s Fun , 2002 .

[27]  Gavin Sim,et al.  Assessing usability and fun in educational software , 2005, IDC '05.

[28]  Maria K. E. Lahman.  always Othered , 2008 .

[29]  William V. Wright,et al.  A Theory of Fun for Game Design , 2004 .

[30]  N. Borgers,et al.  Children as Respondents in Survey Research: Cognitive Development and Response Quality 1 , 2000 .

[31]  C. Forrest,et al.  Elementary school-aged children's reports of their health: A cognitive interviewing study , 2004, Quality of Life Research.

[32]  Kirsten Risden,et al.  Evaluating computer game concepts with children , 2004, IDC '04.

[33]  A. Bosenberg,et al.  Validation of a six‐graded faces scale for evaluation of postoperative pain in children , 2003, Paediatric anaesthesia.

[34]  A. Ant Ozok,et al.  SURVEY DESIGN AND IMPLEMENTATION IN HCI , 2015 .

[35]  Joy L. Johnson,et al.  Interviewing Young Children: Explicating Our Practices and Dilemmas , 2005, Qualitative health research.

[36]  Ann Mahon,et al.  Researching Children: Methods and Ethics , 1996 .

[37]  Stuart MacFarlane,et al.  Using the fun toolkit and other survey methods to gather opinions in child computer interaction , 2006, IDC '06.

[38]  Colin MacDougall,et al.  Extending new paradigm childhood research: meeting the challenges of including younger children , 2005 .

[39]  D. Wong,et al.  Pain in children: comparison of assessment scales. , 1988, The Oklahoma nurse.

[40]  Howard R. Moskowitz,et al.  Sensory and Consumer Research in Food Product Design and Development , 2006 .

[41]  Jason W. Osborne,et al.  Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. , 2005 .

[42]  I. Ajzen,et al.  Predicting and Changing Behavior: The Reasoned Action Approach , 2009 .

[43]  Jayne Taylor,et al.  Doing Research with Children , 1999 .

[44]  E. Köster,et al.  Measuring food liking in children: a comparison of non verbal methods , 1999 .

[45]  C. Pole,et al.  Researching Children and Fashion , 2007 .

[46]  Janet C. Read,et al.  Validating the Fun Toolkit: an instrument for measuring children’s opinions of technology , 2008, Cognition, Technology & Work.

[47]  C. Chambers,et al.  Developmental differences in children's use of rating scales. , 2002, Journal of pediatric psychology.