The evolving emotional experience with portable interactive devices

Emotions play a significant role in people’s lives, including interactions with portable devices. The research aimed to understand the evolving emotional experience between people and portable interactive devices (PIDs). Activity Theory was the theoretical framework used to contextualise the research approach and findings. Two longitudinal experiments were conducted investigating emotional experiences with PIDs over six months. Experiment 1 focused on media / entertainment PIDs while Experiment 2 focused on medical / health PIDs. Mixed research methods consisting of diaries, interviews and codiscovery sessions were used to collect data. Results identified that more social interactions were experienced with media PIDs than medical PIDs. Different Task Categories, and their emotional responses, were also revealed including Features, Functional, Mediation and Auxiliary Categories. Functional and Mediation categories were characterised as overall positive while Features and Auxiliary Categories were characterised as overall negative. Further, the consequences of Negative Personal and Social interactions on the overall emotional experience were determined. For media PIDs, Negative Social experiences adversely impacted the evolving emotional experience. For medical PIDs, both Negative Social and Negative Personal experiences adversely impacted the evolving emotional experience. As a result of the findings the Designing for Evolving Emotional Experience framework was developed, outlining principles to promote positive, and avoid negative, emotional experiences with PIDs. Contributions to knowledge from the research include methodological contributions, advancing understanding of emotional experiences with PIDs, expanding the taxonomy of emotional interactions with PIDs and broadening emotion design theory and principles. The thesis concludes with an outline of implications to design research, design and related fields, future research potentials, as well as the positive contributions to designing for meaningful and enjoyable experiences in everyday life.

[1]  B. Fredrickson.  The role of positive emotions in positive psychology. The broaden-and-build theory of positive emotions. , 2001, The American psychologist.

[2]  Eric Bergman,et al.  Information appliances and beyond : interaction design for consumer products , 2000 .

[3]  Paul Lukowicz,et al.  AMON: a wearable multiparameter medical monitoring and alert system , 2004, IEEE Transactions on Information Technology in Biomedicine.

[4]  Mark Blythe,et al.  Funology:From Usability to Enjoyment , 2003 .

[5]  Alethea L. Blackler,et al.  Emotional experience with portable health devices , 2010 .

[6]  Joseph E LeDoux.  The emotional brain , 1996 .

[7]  Mark van 't Hooft,et al.  Anywhere Anytime Learning with Wireless Mobile Devices , 2009 .

[8]  Bruce Hanington.  Designing Pleasurable Products: An Introduction to the New Human Factors by Patrick W. Jordan (London: Taylor and Francis, 2000) 216 pages, illustrated, $65.00 (hardback) ISBN 0748408444 , 2002, Design Issues.

[9]  D. Ariely.  Combining experiences over time: the effects of duration, intensity changes and on‐line measurements on retrospective pain evaluations , 1998 .

[10]  Martin C. Maguire,et al.  Context of Use within usability activities , 2001, Int. J. Hum. Comput. Stud..

[11]  Paul Dourish,et al.  Where the action is , 2001 .

[12]  Stefan Poslad,et al.  Ubiquitous Computing: Smart Devices, Environments and Interactions , 2009 .

[13]  J. Stainer,et al.  The Emotions , 1922, Nature.

[14]  Barbara A. Lafferty.  Emotional Branding: The New Paradigm for Connecting Brands to People , 2001 .

[15]  B. Joseph Pine,et al.  The Experience Economy , 1999 .

[16]  Emile H. L. Aarts,et al.  True Visions: The Emergence of Ambient Intelligence , 2008 .

[17]  John O'Shaughnessy,et al.  The marketing power of emotion , 2002 .

[18]  Dong-Seok Lee,et al.  Escape from Designers' Dilemma on Creeping Featurism , 2006 .

[19]  Emil Jovanov,et al.  Medical Monitoring Applications for Wearable Computing , 2004, Comput. J..

[20]  P. Johnson-Laird,et al.  Towards a Cognitive Theory of Emotions , 1987 .

[21]  W. Keith Edwards,et al.  At Home with Ubiquitous Computing: Seven Challenges , 2001, UbiComp.

[22]  Izak Benbasat,et al.  An Empirical Study of Computer System Learning: Comparison of Co-Discovery and Self-Discovery Methods , 1997, Inf. Syst. Res..

[23]  G. Fricchione.  Descartes’ Error: Emotion, Reason and the Human Brain , 1995 .

[24]  B. Nardi.  Activity theory and human-computer interaction , 1995 .

[25]  P. Ekman,et al.  What is Meant by Calling Emotions Basic , 2011 .

[26]  P. Kleinginna,et al.  A categorized list of emotion definitions, with suggestions for a consensual definition , 1981 .

[27]  David Gauntlett,et al.  TV Living: Television, Culture and Everyday Life , 1999 .

[28]  Martin Maguire.  Applying Evaluation Methods to Future Digital TV Services , 2002 .

[29]  A. Blackler,et al.  Categorising emotional experiences with portable interactive devices , 2011 .

[30]  A. Blackler,et al.  Emotional experience with portable interactive devices , 2009 .

[31]  Daniel Kahneman,et al.  Evaluation by Moments: Past and Future , 2002 .

[32]  Liam J. Bannon,et al.  Beyond the Interface: Encountering Artifacts in Use , 1989 .

[33]  A. Weale,et al.  Strategies of Qualitative Inquiry, 2nd edition , 2005 .

[34]  P. Griffiths,et al.  Philosophy and the Emotions: Basic Emotions, Complex Emotions, Machiavellian Emotions , 2003 .

[35]  Boris E. R. de Ruyter,et al.  New research perspectives on Ambient Intelligence , 2009, J. Ambient Intell. Smart Environ..

[36]  John C. McCarthy,et al.  Technology as experience , 2004, INTR.

[37]  Context Sensitive Interactive Systems Design : A Framework for Representation of contexts , 2003 .

[38]  A. Ryle.  Object relations theory and activity theory: a proposed link by way of the procedural sequence model. , 1991, The British journal of medical psychology.

[39]  Tracy J. Mayne.  Emotions and health. , 2001 .

[40]  Henry Lieberman,et al.  Out of context: Computer systems that adapt to, and learn from, context , 2000, IBM Syst. J..

[41]  Bill Tomlinson,et al.  PersonalSoundtrack: context-aware playlists that adapt to user pace , 2006, CHI Extended Abstracts.

[42]  D. Watson,et al.  Toward a consensual structure of mood. , 1985, Psychological bulletin.

[43]  Ann Blandford,et al.  Emotion and technology: an empirical study , 2008 .

[44]  Bernd H. Schmitt.  Experiential marketing : how to get customers to sense, feel, think, act, and relate to your company and brands , 1999 .

[45]  P. Jordan,et al.  Pleasure With Products : Beyond Usability , 2002 .

[46]  Leonard M. Jessup,et al.  The relevance of social issues in ubiquitous computing environments , 2002, CACM.

[47]  Evangelos Karapanos,et al.  On the retrospective assessment of users' experiences over time: memory or actuality? , 2010, CHI Extended Abstracts.

[48]  Rjoè,et al.  Activity theory as a framework for analyzing and redesigning work , 2005 .

[49]  John Herbert,et al.  CAMMD: Context-Aware Mobile Medical Devices , 2006, J. Univers. Comput. Sci..

[50]  Daniel Kahneman,et al.  Experiences extended across time: Evaluation of moments and episodes , 1992 .

[51]  P. Desmet,et al.  Framework of product experience , 2007 .

[52]  Michael Kipp,et al.  Designing Emotions , 2011, KI - Künstliche Intelligenz.

[53]  Leena Arhippainen,et al.  Empirical Evaluation of User Experience in two Adaptive Mobile Application Prototypes , 2003 .

[54]  Astrid M. Oddershede,et al.  Perception of Mobile Technology Provision in Health Service , 2008 .

[55]  John Zimmerman,et al.  User experience over time: an initial framework , 2009, CHI.

[56]  Ann Blandford,et al.  Conceptualising user hedonic experience , 2004 .

[57]  Youngjin Yoo,et al.  Computing in Everyday Life: A Call for Research on Experiential Computing , 2010, MIS Q..

[58]  A. Ortony,et al.  What's basic about basic emotions? , 1990, Psychological review.

[59]  Marsha L. Richins.  Measuring Emotions in the Consumption Experience , 1997 .

[60]  John Dixon,et al.  The Design of Future Things , 2010 .

[61]  D. Norman.  Emotional design : why we love (or hate) everyday things , 2004 .

[62]  A. Blackler,et al.  Positive emotional experiences with portable interactive products , 2012 .

[63]  Richard Fries,et al.  Handbook of Medical Device Design , 2000 .

[64]  Sam Bucolo,et al.  Driving Experience and the Effect of Challenging Interactions in High Traffic Context , 2004 .

[65]  Cato Alexander Bjørkli,et al.  Exploring the embodied-mind approach to user experience , 2002, NordiCHI '02.

[66]  Jack Ox.  Leonardo's Laptop: Human Needs and The New Computing Technologies by Ben Shneiderman . MIT Press , Cambridge, MA, U.S.A. , 2002 . 269 pp., illus. Trade. ISBN: 0-262-19476-7. , 2003, Leonardo.

[67]  Witold Pedrycz,et al.  Ambient Intelligence, Wireless Networking, And Ubiquitous Computing , 2006 .

[68]  Gordon H. Bower,et al.  Affect, memory, and social cognition. , 2000 .

[69]  C.C.M. Hummels,et al.  Gestural design tools: Prototypes, experiments and scenarios , 2000 .

[70]  Amic G. Ho,et al.  Emotion Design, Emotional Design, Emotionalize Design: A Review on Their Relationships from a New Perspective , 2012 .

[71]  Richard E. Boyatzis,et al.  Transforming Qualitative Information: Thematic Analysis and Code Development , 1998 .

[72]  Antti Oulasvirta,et al.  Understanding Mobile Contexts , 2003, Mobile HCI.

[73]  Peter Brown.  Context-awareness: some compelling applications , 2000 .

[74]  B. Fredrickson.  What Good Are Positive Emotions? , 1998, Review of general psychology : journal of Division 1, of the American Psychological Association.

[75]  Debora Viana Thompson,et al.  Defeating feature fatigue. , 2006, Harvard business review.

[76]  Gregory D. Abowd,et al.  A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Applications , 2001, Hum. Comput. Interact..

[77]  Anind K. Dey,et al.  Designing mediation for context-aware applications , 2005, TCHI.

[78]  Oya Demirbilek,et al.  Product design, semantics and emotional response , 2003, Ergonomics.

[79]  Jan Van den Bergh,et al.  Towards modeling context-sensitive interactive applications: the context-sensitive user interface profile (CUP) , 2005, SoftVis '05.

[80]  Paul Dourish,et al.  Seeking a Foundation for Context-Aware Computing , 2001, Hum. Comput. Interact..

[81]  D. Kahneman,et al.  Patients' memories of painful medical treatments: real-time and retrospective evaluations of two minimally invasive procedures , 1996, Pain.

[82]  Lucy Suchman.  Plans and situated actions: the problem of human-machine communication , 1987 .

[83]  Vesna Popovic,et al.  Driving : the emotional experience and automotive design , 2004 .

[84]  R. Reiner,et al.  Integrating a Portable Biofeedback Device into Clinical Practice for Patients with Anxiety Disorders: Results of a Pilot Study , 2008, Applied psychophysiology and biofeedback.

[85]  Suprateek Sarker,et al.  Understanding mobile handheld device use and adoption , 2003, CACM.

[86]  Upkar Varshney,et al.  Pervasive Healthcare and Wireless Health Monitoring , 2007, Mob. Networks Appl..

[87]  Kristina Höök,et al.  eMoto: emotionally engaging interaction , 2004, Personal and Ubiquitous Computing.

[88]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[89]  Na Li,et al.  The importance of affective quality , 2005, CACM.

[90]  Ruth Kikin-Gil,et al.  Affective is effective: how information appliances can mediate relationships within communities and increase one’s social effectiveness , 2006, Personal and Ubiquitous Computing.

[91]  Jane Fulton Suri.  Designing Experience: Whether to Measure Pleasure or Just Tune In? , 2002 .

[92]  Bonnie A. Nardi.  Concepts of Cognition and Consciousness: four voices , 1996 .

[93]  Peter C. Wright,et al.  Pervasive healthcare in lived experience: Thinking beyond the home , 2010, 2010 4th International Conference on Pervasive Computing Technologies for Healthcare.

[94]  D. Morley.  Television, Audiences and Cultural Studies , 1992 .

[95]  C. Izard.  The psychology of emotions , 1991 .

[96]  Jan Stage,et al.  New techniques for usability evaluation of mobile systems , 2004, Int. J. Hum. Comput. Stud..

[97]  Diane J. Cook,et al.  Author's Personal Copy Pervasive and Mobile Computing Ambient Intelligence: Technologies, Applications, and Opportunities , 2022 .

[98]  Dana Chisnell,et al.  Handbook of Usability Testing , 2009 .

[99]  K. Poels,et al.  How to Capture the Heart? Reviewing 20 Years of Emotion Measurement in Advertising , 2006 .

[100]  K. Scherer.  What are emotions? And how can they be measured? , 2005 .

[101]  Guy H. Walker,et al.  Human Factors Methods: A Practical Guide for Engineering and Design , 2012 .

[102]  Manfred Tscheligi,et al.  User Experience Evaluation in the Mobile Context , 2010, Mobile TV: Customizing Content and Experience.

[103]  K. Oatley.  Emotions: A Brief History , 2004 .

[104]  V. Mahajan,et al.  Delight by Design: The Role of Hedonic versus Utilitarian Benefits , 2008 .

[105]  E Hirt,et al.  Wearable devices for telemedicine applications , 2005, Journal of telemedicine and telecare.

[106]  N. Denzin.  The research act: A theoretical introduction to sociological methods , 1977 .

[107]  Andreas Krause,et al.  SenSay: a context-aware mobile phone , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[108]  Amos Tversky,et al.  Endowment and contrast in judgments of well-being. , 1991 .

[109]  John Rieman,et al.  The diary study: a workplace-oriented research tool to guide laboratory efforts , 1993, INTERCHI.

[110]  Clifford Nass,et al.  Future interfaces: social and emotional , 2002, CHI Extended Abstracts.

[111]  Jorge Frascara.  Cognition, Emotion, and Other Inescapable Dimensions of Human Experience. , 1999 .

[112]  Marc Hassenzahl,et al.  MARC HASSENZAHL CHAPTER 3 The Thing and I: Understanding the Relationship Between User and Product , 2003 .

[113]  Kristina Höök,et al.  Mirroring bodily experiences over time , 2009, CHI Extended Abstracts.

[114]  Leslie Haddon,et al.  Mobile Communications: An Introduction to New Media , 2009 .

[115]  A. Lymberis,et al.  Advanced Wearable Health Systems and Applications - Research and Development Efforts in the European Union , 2007, IEEE Engineering in Medicine and Biology Magazine.

[116]  Daniel J. Garland,et al.  Situation Awareness Analysis and Measurement , 2009 .

[117]  Martin W. Bauer,et al.  Qualitative researching with text, image and sound : a practical handbook , 2000 .

[118]  Gregory D. Abowd,et al.  Providing architectural support for building context-aware applications , 2000 .

[119]  Sarah Diefenbach,et al.  Needs, affect, and interactive products - Facets of user experience , 2010, Interact. Comput..

[120]  Tom Rodden,et al.  At home with the technology: an ethnographic study of a set-top-box trial , 1999, TCHI.

[121]  Colin Robson,et al.  Real World Research: A Resource for Social Scientists and Practitioner-Researchers , 1993 .

[122]  Kasper Hornbæk,et al.  Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience , 2011, CHI.

[123]  Vesna Popovic,et al.  Emotional driving experiences , 2008 .

[124]  Victor Kaptelinin,et al.  Computer-mediated activity: functional organs in social and developmental contexts , 1995 .

[125]  Mikael Wiberg,et al.  The Interaction Society: Practice, Theories and Supportive Technologies , 2005 .

[126]  Vesna Popovic,et al.  The evaluation of qualitative methods selection in the field of design and emotion , 2010 .

[127]  Saul Greenberg,et al.  Context as a Dynamic Construct , 2001, Hum. Comput. Interact..

[128]  Marc Hassenzahl,et al.  The Effect of Perceived Hedonic Quality on Product Appealingness , 2001, Int. J. Hum. Comput. Interact..

[129]  B. Fredrickson.  Extracting meaning from past affective experiences: The importance of peaks, ends, and specific emotions , 2000 .

[130]  Hsin-Hsi Lai,et al.  Expression modes used by consumers in conveying desire for product form: A case study of a car , 2006 .

[131]  J. Russell.  Core affect and the psychological construction of emotion. , 2003, Psychological review.

[132]  Harri Kiljander,et al.  Evolution and usability of mobile phone interaction styles , 2004 .

[133]  Leysia Palen,et al.  Discovery and Integration of Mobile Communications in Everyday Life , 2001, Personal and Ubiquitous Computing.

[134]  Lisa Haney,et al.  Applying User-Centered Design to Mobile Application Development , 2005 .

[135]  D. Goleman.  Emotional Intelligence: Why It Can Matter More Than IQ , 1995 .

[136]  D. Kahneman,et al.  Duration neglect in retrospective evaluations of affective episodes. , 1993, Journal of personality and social psychology.

[137]  D. Kahneman,et al.  When More Pain Is Preferred to Less: Adding a Better End , 1993 .

[138]  Albrecht Schmidt,et al.  There is more to context than location , 1999, Comput. Graph..

[139]  Roland T. Rust,et al.  Feature Fatigue: When Product Capabilities Become Too Much of a Good Thing , 2005 .

[140]  Scott W. Campbell,et al.  A cross-cultural comparison of perceptions and uses of mobile telephony , 2007, New Media Soc..

[141]  Charles Lee Isbell,et al.  Impact of user context on song selection , 2008, ACM Multimedia.

[142]  C. Nass,et al.  Emotion in human-computer interaction , 2002 .

[143]  Caroline Hummels,et al.  Beauty in usability : forget about ease of use! , 2002 .

[144]  N. Bolger,et al.  Diary methods: capturing life as it is lived. , 2003, Annual review of psychology.

[145]  L. F. Barrett,et al.  Handbook of Emotions , 1993 .

[146]  Rainer Stiefelhagen,et al.  Computers in the Human Interaction Loop , 2009, Human-Computer Interaction Series.

[147]  Daniel Ronzani,et al.  Ubiquitous Computing and Communication Journal Volume 4 Number 2 Page9 the Battle of Concepts: Ubiquitous Computing, Pervasive Computing and Ambient Intelligence in Mass Media , 2022 .

[148]  Roger Jianxin Jiao,et al.  A Kansei mining system for affective design , 2006, Expert Syst. Appl..

[149]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[150]  P. Jordan.  Human factors for pleasure in product use. , 1998, Applied ergonomics.

[151]  Upkar Varshney,et al.  Using Wireless Networks for Enhanced Monitoring of Patients , 2004, AMCIS.

[152]  D. Silverman.  Doing Qualitative Research , 2009 .

[153]  Winifred Gallagher,et al.  The Power of Place: How Our Surroundings Shape Our Thoughts, Emotions, and Actions , 1993 .

[154]  Personal, portable, pedestrian: Mobile phones in Japanese life. , 2006 .

[155]  James D. Hollan,et al.  A diary study of mobile information needs , 2008, CHI.

[156]  Liam J. Bannon,et al.  A pilgrim's progress: From cognitive science to cooperative design , 1990, AI & SOCIETY.

[157]  Kaisa Väänänen,et al.  Analysing user experience of personal mobile products through contextual factors , 2010, MUM.

[158]  E. Diener,et al.  Subjective well-being. The science of happiness and a proposal for a national index. , 2000, The American psychologist.

[159]  Ichiro Kawachi,et al.  Positive emotion and health: going beyond the negative. , 2005, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[160]  J. Campos,et al.  A Darwinian Legacy to Understanding Human Infancy , 2003, Annals of the New York Academy of Sciences.

[161]  Dan Ariely,et al.  On the making of an experience: the effects of breaking and combining experiences on their overall evaluation , 2000 .

[162]  Daniel M. Gloyd.  Positive user experience and medical adherence , 2003, DPPI '03.

[163]  Hans-Werner Gellersen,et al.  Beyond Prototypes: Challenges in Deploying Ubiquitous Systems , 2002, IEEE Pervasive Comput..

[164]  Matt Jones,et al.  Mobile Interaction Design , 2006 .

[165]  G. Loewenstein.  Emotions in Economic Theory and Economic Behavior , 2000 .

[166]  Geri Gay.  Context-Aware Mobile Computing: Affordances of Space, Social Awareness, and Social Influence , 2009, Context-Aware Mobile Computing.

[167]  Sungmee Park,et al.  Enhancing the quality of life through wearable technology , 2003, IEEE Engineering in Medicine and Biology Magazine.

[168]  Rich Ling.  One can talk about common manners!': the use of mobile telephones in inappropriate situations , 1996 .

[169]  Nicholas V. Iuppa.  Interactive Design for New Media and the Web , 2001 .

[170]  Lynne Hamill,et al.  Mobile world : past, present and future , 2005 .

[171]  Sari Kujala,et al.  Capturing users’ perceptions of valuable experience and meaning , 2009 .