Accounting for diversity in subjective judgments

In this paper we argue against averaging as a common practice in the analysis of subjective attribute judgments, both across and within subjects. Previous work has raised awareness of the diversity between individuals' perceptions. In this paper it will furthermore become apparent that such diversity can also exist within a single individual, in the sense that different attribute judgments from a subject may reveal different, complementary, views. A Multi-Dimensional Scaling approach that accounts for the diverse views on a set of stimuli is proposed and its added value is illustrated using published data. We will illustrate that the averaging analysis provides insight to only 1/6th of the total number of attributes in the example dataset. The proposed approach accounts for more than double the information obtained from the average model, and provides richer and semantically diverse views on the set of stimuli.

[1]  JAN-BENEDICT STEENKAMP,et al.  Attribute Elicitation in Marketing Research: A Comparison of Three Procedures , 1997 .

[2]  David Frohlich,et al.  Beauty constructs for MP3 players , 2007 .

[3]  Daniel Ullrich,et al.  To do or not to do: Differences in user experience and retrospective judgments depending on the presence or absence of instrumental goals , 2007, Interact. Comput..

[4]  Marc Hassenzahl,et al.  The Interplay of Beauty, Goodness, and Usability in Interactive Products , 2004, Hum. Comput. Interact..

[5]  Paul E. Green,et al.  Multidimensional Scaling: Concepts and Applications , 1989 .

[6]  Marc Hassenzahl,et al.  From mental effort to perceived usability: transforming experiences into summary assessments , 2004, CHI EA '04.

[7]  Marc Hassenzahl,et al.  Capturing Design Space From a User Perspective: The Repertory Grid Technique Revisited , 2000, Int. J. Hum. Comput. Interact..

[8]  Marc Hassenzahl,et al.  Eine gruppenspezifische Repertory Grid Analyse der wahrgenommenen Attraktivität von Universitätswebsites , 2007, Mensch & Computer.

[9]  Evangelos Karapanos,et al.  Do knobs have character?: exploring diversity in users' inferences , 2008, CHI Extended Abstracts.

[10]  H. V. Trijp,et al.  Consumer research in the early stages of new product development: a critical review of methods and techniques , 2005 .

[11]  Evangelos Karapanos,et al.  The quantitative side of the Repertory Grid Technique: some concerns , 2009, ArXiv.

[12]  M. Hassenzahl,et al.  AESTHETICS IN INTERACTIVE PRODUCTS: CORRELATES AND CONSEQUENCES OF BEAUTY , 2008 .

[13]  Einar Breivik,et al.  Elicitation of product attributes in an evaluation context: A comparison of three elicitation techniques , 2003 .

[14]  P. Jordan.  Designing Pleasurable Products: An Introduction to the New Human Factors , 2000 .

[15]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[16]  Evangelos Karapanos,et al.  Characterizing the Diversity in Users' Perceptions , 2007, INTERACT.

[17]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[18]  James O. Ramsay.  Matfit: A fortran subroutine for comparing two matrices in a subspace , 1990 .

[19]  Florian N. Egger,et al.  "Trust me, I'm an online vendor": towards a model of trust for e-commerce system design , 2000, CHI Extended Abstracts.

[20]  Daniel Fallman,et al.  In Romance with the Materials of Mobile Interaction : A Phenomenological Approach to the Design of Mobile Information Technology , 2003 .

[21]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[22]  Evangelos Karapanos,et al.  User experience over time , 2008, CHI Extended Abstracts.

[23]  David R. Anderson,et al.  Understanding AIC and BIC in Model Selection , 2004 .

[24]  Niels Asger Nielsen,et al.  A comparison of five elicitation techniques for elicitation of attributes of low involvement products , 1999 .

[25]  Stephen Boyd Davis,et al.  Constructing a Player-Centred Definition of Fun for Video Games Design , 2004, BCS HCI.

[26]  C. Osgood,et al.  The Measurement of Meaning , 1958 .

[27]  M. Sarstedt.  A review of recent approaches for capturing heterogeneity in partial least squares path modelling , 2008 .

[28]  Jean-Bernard Martens.  Image Technology Design: A Perceptual Approach , 2003 .

[29]  Kasper Hornbæk,et al.  Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? , 2000, CHI.

[30]  Wijnand A. IJsselsteijn,et al.  Keeping in touch with the family: home and away with the ASTRA awareness system , 2004, CHI EA '04.

[31]  Richard Bell,et al.  A manual for repertory grid technique , 1977 .

[32]  Marc Hassenzahl,et al.  Analysis of web sites with the repertory grid technique , 2001, CHI Extended Abstracts.

[33]  John Zimmerman,et al.  User experience over time: an initial framework , 2009, CHI.

[34]  Bo N. Schenkman,et al.  Aesthetics and preferences of web pages , 2000, Behav. Inf. Technol..

[35]  Marc Hassenzahl.  Character grid : a simple repertory grid technique for web site analysis and evaluation , 2003 .