Review of unconventional user interfaces for emotional communication between long-distance partners

New form factors and user interfaces for computer-mediated communication are emerging. The possibilities to use these systems for emotional communication are interesting, and recent years have witnessed the appearance of a versatile range of prototypes. In this paper, we present the results of a systematic literature review on research addressing the design of systems with unconventional user interfaces for emotional communication, focusing on the use case of facilitating long-distance relationships. We reviewed a body of 150 papers resulting from a systematic search, further analysis scoping the body to 47 papers, containing altogether 52 prototypes that were relevant for our focus. We then analysed the characteristics affecting the interaction mediated by these systems and their user interfaces. We present the results related to the design attributes, e.g., form factors, modalities, and message types of the systems, as well as to the evaluation approaches. As salient findings, touch input and visual output are the most common interaction modalities in these systems, and their evaluations lack in-the-wild studies, especially on long-term usage.

[1]  Grace Ngai,et al.  Emotar: Communicating Feelings through Video Sharing , 2015, IUI.

[2]  Leon Adam Watts,et al.  It's neat to feel the heat: how can we hold hands at a distance? , 2012, CHI Extended Abstracts.

[3]  A. J. Bernheim Brush,et al.  BodyPods: Designing Posture Sensing Chairs for Capturing and Sharing Implicit Interactions , 2015, TEI.

[4]  Katherine M. Tsui,et al.  Towards measuring the quality of interaction: communication through telepresence robots , 2012, PerMIS.

[5]  Leon Adam Watts,et al.  The Magic Sock Drawer project , 2011, CHI Extended Abstracts.

[6]  Sarah Diefenbach,et al.  All You Need is Love: Current Strategies of Mediating Intimate Relationships through Technology , 2012, TCHI.

[7]  Alessandro Soro,et al.  Cooking Together at a Distance: Sustain Connectedness for Long Distance Families , 2017, CHI Extended Abstracts.

[8]  Lora Aroyo,et al.  PillowTalk: can we afford intimacy? , 2007, Tangible and Embedded Interaction.

[9]  Carman Neustaedter,et al.  In Your Eyes: Anytime, Anywhere Video and Audio Streaming for Couples , 2017, CSCW.

[10]  Gordon Cheng,et al.  Augmenting Affect from Speech with Generative Music , 2015, CHI Extended Abstracts.

[11]  Matthew Jennex,et al.  Anchor: connecting sailors to home , 2012, CHI EA '12.

[12]  Carman Neustaedter,et al.  Intimacy in long-distance relationships over video chat , 2012, CHI.

[13]  Sriram Subramanian,et al.  Emotions Mediated Through Mid-Air Haptics , 2015, CHI.

[14]  Tomoo Inoue,et al.  Virtually dining together in time-shifted environment: KIZUNA design , 2013, CSCW.

[15]  Carman Neustaedter,et al.  Communicating Through A Telepresence Robot: A Study of Long Distance Relationships , 2017, CHI Extended Abstracts.

[16]  Marc Hassenzahl,et al.  Sharing a Robotic Pet as a Maintenance Strategy for Romantic Couples in Long-Distance Relationships.: An Autobiographical Design Exploration , 2016, CHI Extended Abstracts.

[17]  Jonna Häkkilä,et al.  In the candle light: pervasive display concept for emotional communication , 2016, PerDis.

[18]  Anne Marie Piper,et al.  Exploring affective communication through variable-friction surface haptics , 2014, CHI.

[19]  Bongwon Suh,et al.  Now here or nowhere: conflict resolution strategies for intimate relationship in diverse geographical contexts , 2014, CSCW Companion '14.

[20]  Sarah Diefenbach,et al.  The whisper pillow: a study of technology-mediated emotional expression in close relationships , 2013, DPPI.

[21]  Carman Neustaedter,et al.  "MyEyes": The Design and Evaluation of First Person View Video Streaming for Long-Distance Couples , 2017, Conference on Designing Interactive Systems.

[22]  Luís Carriço,et al.  Can you feel it?: sharing heart beats with Augmento , 2012, AH '12.

[23]  Masahiko Inami,et al.  Emotional priming of mobile text messages with ring-shaped wearable device using color lighting and tactile expressions , 2014, AH.

[24]  William G. Griswold,et al.  CoupleVIBE: mobile implicit communication to improve awareness for (long-distance) couples , 2011, CSCW.

[25]  Antti Jylhä,et al.  Designing for Active Place Presence at Home: The Hole in Space Design Experiment , 2016, Conference on Designing Interactive Systems.

[26]  Youngwoo Park,et al.  BreathingFrame: An Inflatable Frame for Remote Breath Signal Sharing , 2015, TEI.

[27]  Albrecht Schmidt,et al.  A design space for ephemeral user interfaces , 2013, TEI '13.

[28]  Wijnand A. IJsselsteijn,et al.  Mediated social touch: a review of current research and future directions , 2006, Virtual Reality.

[29]  Koji Tsukada,et al.  CalMate: communication support system for couples using a calm avatar , 2012, UbiComp '12.

[30]  Carman Neustaedter,et al.  BeWithMe: An Immersive Telepresence System for Distance Separated Couples , 2017, CSCW Companion.

[31]  Koji Tsukada,et al.  InPhase: evaluation of a communication system focused on "happy coincidences" of daily behaviors , 2010, CHI.

[32]  Gijs Huisman,et al.  A touch of affect: mediated social touch and affect , 2012, ICMI '12.

[33]  Xuan Wang,et al.  CoDine: an interactive multi-sensory system for remote dining , 2011, UbiComp '11.

[34]  Leila Takayama,et al.  Presentation of (telepresent) self: On the double-edged effects of mirrors , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[35]  Jon Froehlich,et al.  Bear-with-me: an embodied prototype to explore tangible two-way exchanges of emotional language , 2013, CHI Extended Abstracts.

[36]  S. Sundar,et al.  Personalization versus Customization: The Importance of Agency, Privacy, and Power Usage , 2010 .

[37]  Dzmitry Tsetserukou,et al.  World's first wearable humanoid robot that augments our emotions , 2010, AH.

[38]  Carman Neustaedter,et al.  Flex-N-Feel: The Design and Evaluation of Emotive Gloves for Couples to Support Touch Over Distance , 2017, CSCW.

[39]  Katarzyna Stawarz,et al.  Silka: a domestic technology to mediate the threshold between connection and solitude , 2012, CHI EA '12.

[40]  Ryan Kelly,et al.  Demanding by Design: Supporting Effortful Communication Practices in Close Personal Relationships , 2017, CSCW.

[41]  Eve E. Hoggan,et al.  Pressages: augmenting phone calls with non-verbal messages , 2012, UIST '12.

[42]  Youngwoo Park,et al.  Wrigglo: shape-changing peripheral for interpersonal mobile communication , 2014, CHI.

[43]  Elisa Giaccardi,et al.  Exploring design opportunities for social intimacy through everyday objects and practices , 2013, UbiComp.

[44]  Robert Kowalski,et al.  cubble: a multi-device hybrid approach supporting communication in long-distance relationships , 2013, TEI '13.

[45]  Johanna Schmeer,et al.  Touch trace mirror: asynchronous, collaborative messaging as a concept for creating a relatedness experience , 2010, TEI.

[46]  Daniel Harrison,et al.  Feelybean: communicating touch over distance , 2012, CHI EA '12.

[47]  Leon Adam Watts,et al.  sleepyWhispers: sharing goodnights within distant relationships , 2012, UIST Adjunct Proceedings '12.

[48]  Sean A. Munson,et al.  Temperature sharing to support remote relationships , 2014, UbiComp Adjunct.

[49]  Carman Neustaedter,et al.  Puzzle Space: A Distributed Tangible Puzzle for Long Distance Couples , 2017, CSCW Companion.

[50]  Elizabeth D. Mynatt,et al.  Eating alone, together: new forms of commensality , 2012, GROUP.

[51]  Bruno Dumas,et al.  Designing for intimacy: how fashion design can address privacy issues in wearable computing , 2014, ISWC '14 Adjunct.

[52]  Darren Gergle,et al.  "Back and forth, back and forth": channel switching in romantic couple conflict , 2013, CSCW.

[53]  Adrian David Cheok,et al.  Kissenger: design of a kiss transmission device , 2012, DIS '12.

[54]  Luis Ceze,et al.  Augmenting Interpersonal Communication through Connected Lighting , 2017, CHI Extended Abstracts.

[55]  Leon Adam Watts,et al.  A design framework for mediated personal relationship devices , 2011, BCS HCI.

[56]  Jennifer Ann Rode,et al.  (Whose) value-sensitive design: a study of long- distance relationships in an Arabic cultural context , 2011, CSCW.

[57]  Youngwoo Park,et al.  How do couples use CheekTouch over phone calls? , 2012, CHI.