Using Human Eye Gaze Patterns as Indicators of Need for Assistance from a Socially Assistive Robot

With current growth in social robotics comes a need for well developed and fine tuned agents which respond to the user in a seamless and intuitive manner. Socially assistive robots in particular have become popular for their uses in care for older adults for medication adherence and socializing. Since eye gaze cues are important mediators in human-human interactions, we hypothesize that gaze patterns can be applied to human-robot interactions to identify when the user may need assistance. We reviewed videos (N = 16) of robot supported collaborative work to explore how recognition of gaze patterns for an assistive robot in the context of a medication management task can help predict when a user needs assistance. We found that mutual gaze is a better predictor than confirmatory request, gaze away, and goal reference. While eye gaze serves as an important indicator for need for assistance, it should be combined with other indicators, such as verbal cues or facial expressions to sufficiently represent assistance needed in the interaction and provide timely assistance.

[1]  Patrick Gebhard,et al.  Exploring a Model of Gaze for Grounding in Multimodal HRI , 2014, ICMI.

[2]  Takanori Shibata,et al.  Living With Seal Robots—Its Sociopsychological and Physiological Influences on the Elderly at a Care House , 2007, IEEE Transactions on Robotics.

[3]  Sean Andrist,et al.  Conversational Gaze Aversion for Humanlike Robots , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[4]  Bertrand Schneider,et al.  Real-time mutual gaze perception enhances collaborative learning and collaboration quality , 2013, International Journal of Computer-Supported Collaborative Learning.

[5]  Matthias Scheutz,et al.  Supporting Human Autonomy in a Robot-Assisted Medication Sorting Task , 2018, International Journal of Social Robotics.

[6]  C. Kleinke.  Gaze and eye contact: a research review. , 1986, Psychological bulletin.

[7]  Joyce Yue Chai,et al.  Embodied Collaborative Referring Expression Generation in Situated Human-Robot Interaction , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[8]  Takanori Shibata,et al.  Robot Therapy in a Care House - Change of Relationship among the Residents and Seal Robot during a 2-month Long Study , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[9]  A. Kendon,et al.  The consistency of gaze patterns in social interaction. , 1969, British journal of psychology.

[10]  C. Depp,et al.  Positive mental aging. , 2010, The American journal of geriatric psychiatry : official journal of the American Association for Geriatric Psychiatry.

[11]  Illah R. Nourbakhsh,et al.  The role of expressiveness and attention in human-robot interaction , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[12]  Brian Scassellati,et al.  The Grand Challenges in Socially Assistive Robotics , 2007 .

[13]  Bilge Mutlu,et al.  Using gaze patterns to predict task intent in collaboration , 2015, Front. Psychol..

[14]  Sean Andrist,et al.  Looking Coordinated: Bidirectional Gaze Mechanisms for Collaborative Interaction with Virtual Characters , 2017, CHI.

[15]  Brian Scassellati,et al.  Socially assistive robotics [Grand Challenges of Robotics] , 2007, IEEE Robotics & Automation Magazine.

[16]  B. Scassellati,et al.  Social eye gaze in human-robot interaction , 2017, J. Hum. Robot Interact..

[17]  Emmanuel Senft,et al.  Architecture for a Social Assistive Robot in Cardiac Rehabilitation , 2018, 2018 IEEE 2nd Colombian Conference on Robotics and Automation (CCRA).

[18]  Hideaki Kuzuoka,et al.  Prior-to-request and request behaviors within elderly day care: Implications for developing service robots for use in multiparty settings , 2007, ECSCW.

[19]  Anthony G. Pipe,et al.  Joint action understanding improves robot-to-human object handover , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Maja J Matarić,et al.  Socially Assistive Robotics for Post-stroke Rehabilitation Journal of Neuroengineering and Rehabilitation Socially Assistive Robotics for Post-stroke Rehabilitation , 2007 .

[21]  Takayuki Kanda,et al.  How to approach humans?-strategies for social robots to initiate interaction , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[22]  R. Riener,et al.  Journal of Neuroengineering and Rehabilitation Open Access Biofeedback for Robotic Gait Rehabilitation , 2022 .

[23]  Kristiina Jokinen,et al.  Gaze and Gesture Activity in Communication , 2009, HCI.

[24]  L. Billeci,et al.  Autism and social robotics: A systematic review , 2016, Autism research : official journal of the International Society for Autism Research.