Visual attention by consumers to check-all-that-apply questions: Insights to support methodological development

The use by consumers of simplifying response strategies when answering check-all-that-apply (CATA) questions for sensory characterization cannot be eliminated. However, ways to motivate consumers to cognitively engage more in the processing of CATA questions and provide their full attention to the task may be welcome. In this context, the aim of the present work was to evaluate patterns of visual attention to CATA questions by consumers and use the insights gained to advance knowledge about CATA question design. A consumer study was carried out in which participants were asked to evaluate two product sets with five samples each (potato chips or plain crackers) and to answer a check-all-that-apply question composed of 20 sensory terms, which was presented on a computer screen. Half of the consumers received the CATA question with the terms in a fixed order, whereas for the other half of participants the order of terms was different for each sample (i.e., simulating within participants balancing of term presentation order). While consumers completed the CATA tasks their eye movements were recorded using a remote eye-tracker. Results showed that the first time that consumers read the CATA question (i.e., for sample 1), they processed the terms from left to right and from top to bottom. In general, they considered the whole list of terms and then selected those that applied to the sample. As the task progressed consumers changed how they answered the CATA question, making fewer and shorter eye fixations to complete the task. Presenting the terms in different order for each sample significantly increased the total number and duration of fixations, which indicates higher cognitive effort. Implications for the design of CATA questions and suggestions for further research are discussed.

[1]  R. Fisher.  Statistical methods for research workers , 1927, Protoplasma.

[2]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[3]  Gastón Ares,et al.  Check-all-that-apply (CATA) responses elicited by consumers: Within-assessor reproducibility and stability of sensory product characterizations , 2013 .

[4]  K. Mcwatters,et al.  A SYSTEMATIC APPROACH TO PREDICTION OF SNACK CHIP ACCEPTABILITY UTILIZING DISCRIMINANT FUNCTIONS BASED ON INSTRUMENTAL MEASUREMENTS , 1995 .

[5]  Jonathan Evans.  The heuristic-analytic theory of reasoning: Extension and evaluation , 2006, Psychonomic bulletin & review.

[6]  N. Hatton.  ASKING QUESTIONS , 1979, The Medical journal of Australia.

[7]  Jonathan Evans.  Dual-processing accounts of reasoning, judgment, and social cognition. , 2008, Annual review of psychology.

[8]  Rik Pieters,et al.  Attention Capture and Transfer in Advertising: Brand, Pictorial, and Text-Size Effects , 2004 .

[9]  J. Cacioppo,et al.  The need for cognition. , 1982 .

[10]  Daniel Kahneman,et al.  Availability: A heuristic for judging frequency and probability , 1973 .

[11]  R. Fisher,et al.  Statistical Methods for Research Workers. , 1955 .

[12]  Peter V. Miller,et al.  Research on Interviewing Techniques , 1981 .

[13]  Jon A. Krosnick,et al.  The Impact of Cognitive Sophistication and Attitude Importance on Response-Order and Question-Order Effects , 1992 .

[14]  S. Jaeger,et al.  Investigation of bias of hedonic scores when co-eliciting product attribute information using CATA questions , 2013 .

[15]  Jolene D. Smyth,et al.  Comparing Check-All and Forced-Choice Question Formats in Web Surveys , 2006 .

[16]  Norman M. Bradburn,et al.  DO RESPONDENTS REALLY “MARK ALL THAT APPLY” ON SELF-ADMINISTERED QUESTIONS? , 1994 .

[17]  Gastón Ares,et al.  Sensory profiling, the blurred line between sensory and consumer science. A review of novel methods for product characterization , 2012 .

[18]  R. Pieters,et al.  A Review of Eye-Tracking Research in Marketing , 2008 .

[19]  Cleo D. Redline,et al.  Eye-Movement Analysis: A New Tool for Evaluating the Design of Visually Administered Instruments (Paper and Web) By , 2001 .

[20]  S. Jaeger,et al.  Check-all-that-apply questions: Influence of attribute order on sensory product characterization , 2013 .

[21]  Gastón Ares,et al.  Investigation of the number of consumers necessary to obtain stable sample and descriptor configurations from check-all-that-apply (CATA) questions , 2014 .

[22]  S. Tirkkonen-Condit,et al.  Think-Aloud Protocols , 2006 .

[23]  S. Jaeger,et al.  CATA questions for sensory product characterization: Raising awareness of biases , 2013 .

[24]  Davide Giacalone,et al.  Comparison of three sensory profiling methods based on consumer perception: CATA, CATA with intensity and Napping® , 2014 .

[25]  W. S. Peavler,et al.  Incentive effects and pupillary changes in association learning. , 1969, Journal of experimental psychology.

[26]  J. Edward Russo,et al.  Eye Fixations Can Save the World: a Critical Evaluation and a Comparison Between Eye Fixations and Other Information Processing Methodologies , 1978 .

[27]  C. J. Findlay,et al.  Experimental consideration for the use of check‐all‐that‐apply questions to describe the sensory properties of orange juices , 2013 .

[28]  Roger Tourangeau,et al.  Eye-Tracking Data: New Insights on Response Order Effects and Other Cognitive Shortcuts in Survey Responding. , 2008, Public opinion quarterly.

[29]  G. Hough,et al.  SENSORY DESCRIPTIVE ANALYSIS, SENSORY ACCEPTABILITY AND EXPECTATION STUDIES ON BISCUITS WITH REDUCED ADDED SALT AND INCREASED FIBER , 2009 .

[30]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.

[31]  Shelly Chaiken,et al.  The heuristic-systematic model in its broader context. , 1999 .

[32]  Zata M Vickers,et al.  EFFECT OF QUESTIONNAIRE DESIGN AND THE NUMBER OF SAMPLES TASTED ON HEDONIC RATINGS , 1993 .

[33]  Gastón Ares,et al.  Are consumer profiling techniques equivalent for some product categories? The case of orange-flavoured powdered drinks , 2011 .

[34]  D. Kahneman.  Thinking, Fast and Slow , 2011 .

[35]  Robert J. K. Jacob,et al.  Eye tracking in human-computer interaction and usability research : Ready to deliver the promises , 2002 .

[36]  Gastón Ares,et al.  CONSUMERS' TEXTURE PERCEPTION OF MILK DESSERTS. II – COMPARISON WITH TRAINED ASSESSORS' DATA , 2012 .

[37]  Young-Seung Lee,et al.  The application of check-all-that-apply (CATA) consumer profiling to preference mapping of vanilla ice cream and its comparison to classical external preference mapping , 2010 .

[38]  F. Pedreschi,et al.  Comparative study of physical and sensory properties of pre-treated potato slices during vacuum and atmospheric frying , 2009 .

[39]  S. Epstein,et al.  Individual differences in intuitive-experiential and analytical-rational thinking styles. , 1996, Journal of personality and social psychology.

[40]  H. Macfie,et al.  Effect of question order on sensory perception and preference in central location trials , 1997 .

[41]  Edward B. Manoukian,et al.  Mathematical nonparametric statistics , 1986 .

[42]  J. Krosnick,et al.  AN EVALUATION OF A COGNITIVE THEORY OF RESPONSE-ORDER EFFECTS IN SURVEY MEASUREMENT , 1987 .

[43]  David LaBerge,et al.  Attentional Processing: The Brain's Art of Mindfulness , 1995 .

[44]  Lars Kaczmirek,et al.  Seeing through the eyes of the respondent: an eye-tracking study on survey question comprehension , 2011 .

[45]  Norbert Schwarz,et al.  A Cognitive Model of Response-Order Effects in Survey Measurement , 1992 .

[46]  K. Rayner,et al.  Eye movements in reading words and sentences , 2007 .

[47]  Gastón Ares,et al.  APPLICATION OF A CHECK‐ALL‐THAT‐APPLY QUESTION TO THE DEVELOPMENT OF CHOCOLATE MILK DESSERTS , 2010 .

[48]  Daniel M. Ennis,et al.  Analysis and Thurstonian Scaling of Applicability Scores , 2013 .