Social Glasses: Simulating Interactive Gaze for Visually Impaired People in Face-to-Face Communication

ABSTRACT Eye contact is crucial in social interactions, linking with sincerity and friendliness. However, blind people cannot see and make eye contact when they communicate with sighted people. It influences the involvement of blind people in blind-sighted conversations. Based on this context, we implemented Social glasses with an eye-tracking system, aiming to improve the communication quality between blind and sighted people in face-to-face conversations. Social glasses attempts to simulate the appropriate gaze for blind people, especially establishing the “eye contact” in blind-sighted conversations. To evaluate the impact of the interactive gaze displayed on the Social glasses, we performed dyadic-conversation tests under four experimental conditions (No Gaze, Constant Gaze, Random Gaze, and Interactive Gaze) for 40 participants. Quantitative results showed that the Interactive gaze has a positive impact on improving the communication quality between blind and sighted people, which were consistent with a qualitative analysis of the participants’ comments.

[1]  Beatriz Defez Garcia,et al.  Real-time assistance prototype — A new navigation aid for blind people , 2010, IECON 2010 - 36th Annual Conference on IEEE Industrial Electronics Society.

[2]  G.W.M. Rauterberg,et al.  Traces of Unconscious Mental Processes in Introspective Reports and Physiological Responses , 2015, PloS one.

[3]  Horst Arndt,et al.  Intergrammar: Toward an Integrative Model of Verbal, Prosodic, and Kinesic Choices in Speech , 1988 .

[4]  Mohammed Yeasin,et al.  Do you see what I see?: designing a sensory substitution device to access non-verbal modes of communication , 2013, ASSETS.

[5]  Jacqueline M. C. Smith,et al.  The role of gaze in impression formation. , 1975, The British journal of social and clinical psychology.

[6]  D. Rutter,et al.  Eye-contact as a chance product of individual looking: Implications for the intimacy model of argyle and dean , 1984 .

[7]  Sethuraman Panchanathan,et al.  iCARE interaction assistant: a wearable face recognition system for individuals with visual impairments , 2005, Assets '05.

[8]  Gerhard Weber,et al.  A tactile windowing system for blind users , 2010, ASSETS '10.

[9]  Racial Differences in Gaze Behavior During Conversations: Two Systematic Observational Studies , 1976 .

[10]  Mohammed E. Hoque,et al.  A google glass app to help the blind in small talk , 2014, ASSETS.

[11]  Dimitrios Tzovaras,et al.  Haptic representation of images for the blind and the visually impaired , 2005 .

[12]  Anton Leuski,et al.  The Effect of An Animated Virtual Character on Mobile Chat Interactions , 2015, HAI.

[13]  Matthias Rauterberg,et al.  Impact of simulated gaze gestures on social interaction for people with visual impairments , 2018 .

[14]  Yukiko Kikuchi,et al.  Cultural background modulates how we look at other persons’ gaze , 2013, International journal of behavioral development.

[15]  Edward T. Hall,et al.  A System for the Notation of Proxemic Behavior1 , 1963 .

[16]  P. Shaver,et al.  Adolescents' theories about the development of their relationships with parents. , 1985, Journal of personality and social psychology.

[17]  Dirk Heylen,et al.  CONTROLLING THE GAZE OF CONVERSATIONAL AGENTS , 2005 .

[18]  D.S. Hedin,et al.  Smartphone based face recognition tool for the blind , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[19]  Alistair A. Young,et al.  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 2017, MICCAI 2017.

[20]  Vincent B. Van Hasselt,et al.  Social adaptation in the blind , 1983 .

[21]  R. Kleck,et al.  Congruence between the indicative and communicative functions of eye contact in interpersonal relations. , 1968, The British journal of social and clinical psychology.

[22]  Mark A. Runco,et al.  Personality and Motivation , 2014 .

[23]  Anderson Rocha,et al.  A Kinect-Based Wearable Face Recognition System to Aid Visually Impaired Users , 2017, IEEE Transactions on Human-Machine Systems.

[24]  Hiroshi Ono,et al.  The Efects of Physical Deviance upon Face-to-Face Interaction , 1966 .

[25]  Shyamanta M. Hazarika,et al.  An insight into assistive technology for the visually impaired and blind people: state-of-the-art and future trends , 2017, Journal on Multimodal User Interfaces.

[26]  F. Biocca,et al.  Internal Consistency and Reliability of the Networked Minds Social Presence Measure , 2006 .

[27]  J. Cassell,et al.  Turn Taking versus Discourse Structure , 1999 .

[28]  M. Argyle,et al.  Gaze and Mutual Gaze , 1994, British Journal of Psychiatry.

[29]  Sethuraman Panchanathan,et al.  Assistive Technologies as Effective Mediators in Interpersonal Social Interactions for Persons with Visual Disability , 2010, ICCHP.

[30]  Norihiko Kato,et al.  Dual Mode Fingertip Guiding Manipulator for Blind Persons Enabling Passive/Active Line-Drawing Explorations , 2008, ICCHP.

[31]  Jun Hu,et al.  E-Gaze: Create Gaze Communication for People with Visual Disability , 2015, HAI.

[32]  A. Kendon.  Some functions of gaze-direction in social interaction. , 1967, Acta psychologica.

[33]  Jun Hu,et al.  Designing and Evaluating a Wearable Device for Accessing Gaze Signals from the Sighted , 2016, HCI.

[34]  Marianna Obrist,et al.  "I Always Wanted to See the Night Sky": Blind User Preferences for Sensory Substitution Devices , 2016, CHI.

[35]  M. Hersen,et al.  Assertiveness, Depression, and Social Support in Older Visually Impaired Adults , 1995 .

[36]  C. Kleinke.  Gaze and eye contact: a research review. , 1986, Psychological bulletin.

[37]  Jacob O. Wobbrock,et al.  In the shadow of misperception: assistive technology use and social interactions , 2011, CHI.

[38]  Shi Qiu,et al.  Understanding visually impaired people’s experiences of social signal perception in face-to-face communication , 2019, Universal Access in the Information Society.

[39]  Yugo Takeuchi,et al.  Coordinating turn-taking and talking in multi-party conversations by controlling robot's eye-gaze , 2014, The 23rd IEEE International Symposium on Robot and Human Interactive Communication.

[40]  Yan Zhao,et al.  Enhancing Emotion Recognition in VIPs with Haptic Feedback , 2016, HCI.

[41]  Hirotaka Osawa.  Emotional Cyborg: Human Extension with Agency for Emotional Labor , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[42]  Andrew Sears,et al.  Representing users in accessibility research , 2012, TACC.

[43]  R. Baumeister,et al.  The need to belong: desire for interpersonal attachments as a fundamental human motivation. , 1995, Psychological bulletin.

[44]  Michael Kipp,et al.  IGaze: Studying Reactive Gaze Behavior in Semi-immersive Human-Avatar Interactions , 2008, IVA.

[45]  Eva-Lotta Sallnäs,et al.  Audio makes a difference in haptic collaborative virtual environments , 2010, Interact. Comput..

[46]  Jun Hu,et al.  Exploring Social Interaction with Everyday Object based on Perceptual Crossing , 2016, HAI.

[47]  Mohammed Yeasin,et al.  Expression: A dyadic conversation aid using Google Glass for people who are blind or visually impaired , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[48]  Hirotaka Osawa,et al.  Wearable social prosthetics: supporting joint attention during communication with artificial eyes , 2017, SEMWEB.

[49]  Marc Cavazza,et al.  Discovering eye gaze behavior during human-agent conversation in an interactive storytelling application , 2010, ICMI-MLMI '10.

[50]  M. Argyle,et al.  EYE-CONTACT, DISTANCE AND AFFILIATION. , 1965, Sociometry.

[51]  P. Lindsay,et al.  Social inclusion of employees who are blind or low vision , 2011 .

[52]  Nigel J. Kemp,et al.  Social Interaction in Blind People: An Experimental Analysis , 1986 .

[53]  Arthur James Lowery,et al.  Advances in implantable bionic devices for blindness: a review , 2016, ANZ journal of surgery.

[54]  A. Aron,et al.  Inclusion of Other in the Self Scale and the structure of interpersonal closeness , 1992 .

[55]  Michael Frankfurter,et al.  A First Look At Communication Theory , 2016 .

[56]  Jun Hu,et al.  E-Gaze Glasses: Simulating Natural Gazes for Blind People , 2016, Tangible and Embedded Interaction.

[57]  D. Cooper,et al.  Some factors associated with popularity. , 1944 .

[58]  M. Argyle,et al.  The meaning of five patterns of gaze , 1974 .

[59]  James M. Coughlan,et al.  Crosswatch: A Camera Phone System for Orienting Visually Impaired Pedestrians at Traffic Intersections , 2008, ICCHP.

[60]  Yuichi Iizuka,et al.  CROSS‐CULTURAL VARIATIONS IN RELATIONSHIP RULES , 1986 .

[61]  Masayuki Numao,et al.  Predicting Levels of Rapport in Dyadic Interactions through Automatic Detection of Posture and Posture Congruence , 2011, 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing.

[62]  G.W.M. Rauterberg,et al.  Nonverbal Signals for Face-to-Face Communication between the Blind and the Sighted , 2015 .

[63]  Riitta Hari,et al.  Influence of Turn-Taking in a Two-Person Conversation on the Gaze of a Viewer , 2013, PloS one.

[64]  Frank Biocca,et al.  Toward a More Robust Theory and Measure of Social Presence: Review and Suggested Criteria , 2003, Presence: Teleoperators & Virtual Environments.