Measuring subjectively experienced time in usability and user experience testing scenarios

Abstract Duration perception influences decision and evaluation processes in daily life. Nevertheless, subjectively experienced time is not commonly considered in usability and user experience (UX) testing scenarios. In this paper, we introduce an easy way to measure subjectively experienced time (SXT) and integrate it into a framework of UX, both theoretically and empirically. In two studies ( N = 62 and N = 80 ), we show that SXT can be evaluated by users after solving tasks on a website. Moreover, we show that it correlates substantially with components of UX but that it is not redundant to common UX measures. Additionally, we show that varying aspects of the website (usability) and the evaluation task (think aloud) show similar effects on both SXT and UX measures. Our results suggest that SXT is partly based on the estimated clock time as distortions in estimations translate into subjectively experienced time. We conclude that users perceive time during an ongoing interaction with a technical device, that they evaluate it, and integrate this evaluation into their UX evaluation. Hence, the presented way of measuring SXT can be handy for UX practitioners to understand how objective temporal characteristics of the interaction translate into UX, especially when designing temporal aspects of interactions to elicit a positive UX.

[1]  S. Droit-Volet,et al.  "Time flies in the presence of angry faces"… depending on the temporal task used! , 2011, Acta psychologica.

[2]  Lassi A. Liikkanen,et al.  Designing interactive systems for the experience of time , 2013, DPPI.

[3]  Anshu Agarwal,et al.  Beyond usability: evaluating emotional response as an integral part of the user experience , 2009, CHI Extended Abstracts.

[4]  Denis Cousineau,et al.  Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson's method , 2005 .

[5]  D. Mahar,et al.  Investigating users' intuitive interaction with complex artefacts. , 2010, Applied ergonomics.

[6]  M. Treisman.  Temporal discrimination and the indifference interval. Implications for a model of the "internal clock". , 1963, Psychological monographs.

[7]  S. W. Brown,et al.  Time perception and attention: The effects of prospective versus retrospective paradigms and task demands on perceived duration , 1985, Perception & psychophysics.

[8]  E. Bigand,et al.  Time flies with music whatever its emotional valence. , 2010, Acta psychologica.

[9]  P. Trahanias,et al.  Passage of time judgments in everyday life are not related to duration judgments except for long durations of several minutes. , 2017, Acta psychologica.

[10]  Marc Hassenzahl,et al.  MARC HASSENZAHL CHAPTER 3 The Thing and I: Understanding the Relationship Between User and Product , 2003 .

[11]  Natalie Woletz,et al.  Combining methods to evaluate mobile usability , 2007, Mobile HCI.

[12]  Madalina Sucala,et al.  Psychological Time: Interval Length Judgments and Subjective Passage of Time Judgments , 2011 .

[13]  Tingting Zhao,et al.  Dual Verbal Elicitation: The Complementary Use of Concurrent and Retrospective Reporting Within a Usability Test , 2013, Int. J. Hum. Comput. Interact..

[14]  Chatpong Tangmanee,et al.  Comparison of Perceived Waiting Time Between Two Lengths of Progress Indicator and Two Styles of Graphics Animation With Perceived Uncertainty as a Covariate , 2018, 2018 Seventh ICT International Student Project Conference (ICT-ISPC).

[15]  Thomas S. Tullis,et al.  Beyond the Usability Lab: Conducting Large-scale Online User Experience Studies , 2009 .

[16]  R M Church,et al.  Scalar Timing in Memory , 1984, Annals of the New York Academy of Sciences.

[17]  William N. Venables,et al.  Modern Applied Statistics with S , 2010 .

[18]  Steven C. Seow.  Designing and Engineering Time: The Psychology of Time Perception in Software , 2008 .

[19]  Martina Ziefle,et al.  How older adults meet complexity: Aging effects on the usability of different mobile phones , 2005, Behav. Inf. Technol..

[20]  Morten Hertzum,et al.  Thinking Aloud in the Presence of Interruptions and Time Constraints , 2013, Int. J. Hum. Comput. Interact..

[21]  Morten Hertzum,et al.  Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload? , 2009, Behav. Inf. Technol..

[22]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[23]  B. Kahn,et al.  How Tolerable is Delay? Consumers’ Evaluations of Internet Web Sites after Waiting , 1998 .

[24]  Leif D. Nelson,et al.  You’re Having Fun When Time Flies , 2010, Psychological science.

[25]  Kasper Hornbæk,et al.  Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? , 2000, CHI.

[26]  G. A. Mendelsohn,et al.  Affect grid : A single-item scale of pleasure and arousal , 1989 .

[27]  Amitava Chattopadhyay,et al.  Waiting for the Web: How Screen Color Affects Time Perception , 2004 .

[28]  S. Droit-Volet,et al.  Emotional time distortions: The fundamental role of arousal , 2012, Cognition & emotion.

[29]  Andrew N. K. Chen,et al.  Can Online Wait Be Managed? The Effect of Filler Interfaces and Presentation Modes on Perceived Waiting Time Online , 2012, MIS Q..

[30]  J. Wearden,et al.  Experience Sampling Methodology reveals similarities in the experience of passage of time in young and elderly adults. , 2015, Acta psychologica.

[31]  D. Altman,et al.  Statistics notes: Calculating correlation coefficients with repeated observations: Part 1—correlation within subjects , 1995 .

[32]  Marc Hassenzahl,et al.  User experience (UX): towards an experiential perspective on product quality , 2008, IHM '08.

[33]  R. Block,et al.  Prospective and retrospective duration judgments: A meta-analytic review , 1997, Psychonomic bulletin & review.

[34]  Noam Tractinsky,et al.  Assessing dimensions of perceived visual aesthetics of web sites , 2004 .

[35]  Morten Hertzum,et al.  Thinking Aloud Influences Perceived Time , 2015, Hum. Factors.

[36]  Joachim Meyer,et al.  Economic and subjective measures of the perceived value of aesthetics and usability , 2006, TCHI.

[37]  M. Thüring,et al.  Usability, aesthetics and emotions in human–technology interaction , 2007 .

[38]  D. Zakay.  Models of psychological time revisited* , 2011 .

[39]  Mark D. Griffiths,et al.  Experiences of Time Loss among Videogame Players: An Empirical Study , 2007, Cyberpsychology Behav. Soc. Netw..

[40]  N. Tractinsky,et al.  What is beautiful is usable , 2000, Interact. Comput..

[41]  Gabor Aranyi,et al.  Testing a model of user‐experience with news websites , 2016, J. Assoc. Inf. Sci. Technol..

[42]  André J. Szameitat,et al.  Behavioral and emotional consequences of brief delays in human-computer interaction , 2009, Int. J. Hum. Comput. Stud..

[43]  Andreas Sonderegger,et al.  The influence of design aesthetics in usability testing: effects on user performance and perceived usability. , 2010, Applied ergonomics.

[44]  John Zimmerman,et al.  Measuring the dynamics of remembered experience over time , 2010, Interact. Comput..

[45]  Scott W. Brown.  Attentional resources in timing: Interference effects in concurrent temporal and nontemporal working memory tasks , 1997, Perception & psychophysics.

[46]  J. Wearden,et al.  Passage of Time Judgments Are Not Duration Judgments: Evidence from a Study Using Experience Sampling Methodology , 2016, Front. Psychol..

[47]  Michael Minge,et al.  Hedonic and pragmatic halo effects at early stages of User Experience , 2018, Int. J. Hum. Comput. Stud..

[48]  Scott A. W. Brown.  Time and attention: Review of the literature , 2008 .

[49]  Richard D. Morey,et al.  Confidence Intervals from Normalized Data: A correction to Cousineau (2005) , 2008 .

[50]  M. Thüring,et al.  Developing and Validating an English Version of the meCUE Questionnaire for Measuring User Experience , 2016 .

[51]  M. Masson,et al.  A linear mixed model analysis of masked repetition priming , 2010 .

[52]  E. Bigand,et al.  Music, emotion, and time perception: the influence of subjective emotional valence and arousal? , 2013, Front. Psychol..

[53]  Nigel Bevan,et al.  Extending Quality in Use to Provide a Framework for Usability Measurement , 2009, HCI.

[54]  Kai-Christoph Hamborg,et al.  The Interplay between Usability and Aesthetics: More Evidence for the "What Is Usable Is Beautiful" Notion , 2014, Adv. Hum. Comput. Interact..

[55]  Eva Hudlicka,et al.  To feel or not to feel: The role of affect in human-computer interaction , 2003, Int. J. Hum. Comput. Stud..

[56]  Thea van der Geest,et al.  What is beautiful is useful - Visual appeal and expected information quality , 2009 .

[57]  Peter C. Verhoef,et al.  Erim Report Series Research in Management Consumer Perception and Evaluation of Waiting Time: a Field Experiment Bibliographic Data and Classifications , 2022 .

[58]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[59]  Sanford Weisberg,et al.  An R Companion to Applied Regression , 2010 .

[60]  D G Altman,et al.  Correlation, regression, and repeated data. , 1994, BMJ.

[61]  Michael Minge,et al.  meCUE - Ein modularer Fragebogen zur Erfassung des Nutzungserlebens , 2013, MuC.

[62]  P. Hancock,et al.  How cognitive load affects duration judgments: A meta-analytic review. , 2010, Acta psychologica.

[63]  M. Reed,et al.  A watched pot sometimes boils: a study of duration experience. , 1980, Acta psychologica.

[64]  A. Baddeley.  Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[65]  Virpi Roto,et al.  Understanding, scoping and defining user experience: a survey approach , 2009, CHI.

[66]  P. Trahanias,et al.  The explicit judgment of long durations of several minutes in everyday life: Conscious retrospective memory judgment and the role of affects? , 2018, PloS one.

[67]  Douglas G. Altman,et al.  Statistics Notes: Correlation, regression, and repeated data , 1994 .

[68]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[69]  J. Gibbon.  Scalar expectancy theory and Weber's law in animal timing. , 1977 .