iKnowU - Exploring the Potential of Multimodal AR Smart Glasses for the Decoding and Rehabilitation of Face Processing in Clinical Populations

This article presents an explorative study with a smart glasses application developed to help visually impaired individuals to identify faces and facial expressions of emotion. The paper discusses three experiments in which different patients, suffering from distinct pathologies impairing vision, tested our application. These preliminary studies demonstrate the feasibility and usefulness of visual prostheses for face and emotion identification, and offer novel and interesting directions for future wearable see-through devices.

[1]  Susan M. Downes,et al.  A Depth-Based Head-Mounted Visual Display to Aid Navigation in Partially Sighted Individuals , 2013, PloS one.

[2]  Rabia Jafri,et al.  Exploring the potential of eyewear-based wearable display devices for use by the visually impaired , 2014, 2014 3rd International Conference on User Science and Engineering (i-USEr).

[3]  Guangtao Zhai,et al.  An Augmented-Reality night vision enhancement application for see-through glasses , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[4]  Frode Eika Sandnes,et al.  What Do Low-Vision Users Really Want from Smart Glasses? Faces, Text and Perhaps No Glasses at All , 2016, ICCHP.

[5]  Joo-Hwee Lim,et al.  A Wearable Face Recognition System on Google Glass for Assisting Social Interactions , 2014, ACCV Workshops.

[6]  M. Seghier,et al.  A network of occipito-temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing. , 2003, Brain : a journal of neurology.

[7]  Kai Kunze,et al.  Eyewear computers for human-computer interaction , 2016, Interactions.

[8]  Bruno Rossion,et al.  Does Prosopagnosia Take the Eyes Out of Face Representations? Evidence for a Defect in Representing Diagnostic Facial Information following Brain Damage , 2005, Journal of Cognitive Neuroscience.

[9]  Rabia Jafri.  A GPU-accelerated real-time contextual awareness application for the visually impaired on Google’s project Tango device , 2016, The Journal of Supercomputing.

[10]  Rabia Jafri,et al.  Computer vision-based object recognition for the visually impaired in an indoors environment: a survey , 2013, The Visual Computer.

[11]  Bogdan Raducanu,et al.  New Opportunities for Computer Vision-Based Assistive Technology Systems for the Visually Impaired , 2014, Computer.

[12]  Joo-Hwee Lim,et al.  SocioGlass: social interaction assistance with face recognition on google glass , 2016 .

[13]  A. Horowitz,et al.  Challenges faced by older adults with vision loss: a qualitative study with implications for rehabilitation , 2012, Clinical rehabilitation.

[14]  Daniel McDuff,et al.  AFFDEX SDK: A Cross-Platform Real-Time Multi-Face Expression Recognition Toolkit , 2016, CHI Extended Abstracts.

[15]  Joo-Hwee Lim,et al.  Enhancing Social Interaction with Seamless Face Recognition on Google Glass: Leveraging opportunistic multi-tasking on smart phones , 2015, MobileHCI Adjunct.

[16]  Sethuraman Panchanathan,et al.  A Systematic Requirements Analysis and Development of an Assistive Device to Enhance the Social Interaction of People Who are Blind or Visually Impaired , 2008 .

[17]  Alex D. Hwang,et al.  An Augmented-Reality Edge Enhancement Application for Google Glass , 2014, Optometry and vision science : official publication of the American Academy of Optometry.

[18]  Jessica Royer,et al.  Mapping the impairment in decoding static facial expressions of emotion in prosopagnosia , 2017, Social cognitive and affective neuroscience.

[19]  Computer Vision-Based Aid for the Visually Impaired Persons-A Survey And Proposing New Framework , 2014 .

[20]  Junpeng Lao,et al.  Residual perception of biological motion in cortical blindness , 2016, Neuropsychologia.

[21]  Gretchen A. Stevens,et al.  Global prevalence of vision impairment and blindness: magnitude and temporal trends, 1990-2010. , 2013, Ophthalmology.