Face Recognition Assistant for People with Visual Impairments

Although there are many face recognition systems to help individuals with visual impairments (VIPs) recognize other people, almost all require a database with the pictures and names of the people who should be tracked. These solutions would not be able to help VIPs recognize people they might not know well. In this work, we investigate the requirements and challenges that must be addressed in the design of a face recognition system for helping VIPs recognize people with whom they have weak-ties. We first conducted a formative study with eight visually impaired people. Using insights learned from the formative study, we developed a research prototype that runs on a mobile phone worn around the user's neck. The developed prototype is a wearable face recognition system that opportunistically captures and stores undistorted face images and contextual information about the user's interaction with each person to a database, without the user intervention, as she interacts with new people. We then used this prototype application as a technology probe---asking VIP participants to use the device in a realistic scenario in which they meet and re-encounter several new people. We analyze and report feedback collected from VIPs about the design and use of such a service.

[1]  E. Hall,et al.  The Hidden Dimension , 1970 .

[2]  Mark S. Granovetter.  The Strength of Weak Ties , 1973, American Journal of Sociology.

[3]  Noah E. Friedkin,et al.  A test of structural features of granovetter's strength of weak ties theory , 1980 .

[4]  B Phillips,et al.  Predictors of assistive technology abandonment. , 1993, Assistive technology : the official journal of RESNA.

[5]  T. Seeman.  Social ties and health: the benefits of social integration. , 1996, Annals of epidemiology.

[6]  Regan A. R. Gurung,et al.  Close personal relationships and health outcomes: A key to the role of social support. , 1997 .

[7]  Jesús Chamorro-Martínez,et al.  Diatom autofocusing in brightfield microscopy: a comparative study , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[8]  The stability of impact of assistive devices , 2001 .

[9]  G Strong,et al.  The stability of impact of assistive devices. , 2001, Disability and rehabilitation.

[10]  E. Morrison,et al.  NEWCOMERS' RELATIONSHIPS: THE ROLE OF SOCIAL NETWORK TIES DURING SOCIALIZATION , 2002 .

[11]  Caroline Haythornthwaite,et al.  Strong, Weak, and Latent Ties and the Impact of New Media , 2002, Inf. Soc..

[12]  Thad Starner,et al.  Dual Tasks Attention, Memory, and Wearable Interfaces , 2022 .

[13]  Rogerio DePaula,et al.  A Framework for the Adoption of Assistive Technology , 2002 .

[14]  Laura E. Dreer,et al.  Social Problem-Solving Abilities and Psychological Adjustment of Persons in Low Vision Rehabilitation. , 2005 .

[15]  C. McCreadie,et al.  The acceptability of assistive technology to older people , 2005, Ageing and Society.

[16]  Sethuraman Panchanathan,et al.  iCARE interaction assistant: a wearable face recognition system for individuals with visual impairments , 2005, Assets '05.

[17]  Elizabeth Rochon,et al.  Cell phone software aiding name recall , 2009, CHI Extended Abstracts.

[18]  D.S. Hedin,et al.  Smartphone based face recognition tool for the blind , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[19]  Jacob O. Wobbrock,et al.  In the shadow of misperception: assistive technology use and social interactions , 2011, CHI.

[20]  Martin Kurze,et al.  Smart glasses linking real live and social network's contacts by face recognition , 2011, AH '11.

[21]  Aaron Steinfeld,et al.  Helping visually impaired users properly aim a camera , 2012, ASSETS '12.

[22]  Xi Wang,et al.  Computerized-eyewear based face recognition system for improving social lives of prosopagnosics , 2013, 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops.

[23]  Bruce A. Draper,et al.  The challenge of face recognition from digital point-and-shoot cameras , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[24]  Kai Kunze,et al.  Who are you?: A wearable face recognition system to support human memory , 2013, AH.

[25]  Gillian M. Sandstrom,et al.  Social Interactions and Well-Being , 2014, Personality & social psychology bulletin.

[26]  Kai Kunze,et al.  Haven't we met before?: a realistic memory assistance system to remind you of the person in front of you , 2014, AH.

[27]  Mahmoud Hassaballah,et al.  Face recognition: challenges, achievements and future directions , 2015, IET Comput. Vis..

[28]  Rohitash Chandra,et al.  Design of a Mobile Face Recognition System for Visually Impaired Persons , 2015, ArXiv.

[29]  Anderson Rocha,et al.  A Wearable Face Recognition System Built into a Smartwatch and the Visually Impaired User , 2015, ICEIS.

[30]  Ning Zhang,et al.  Beyond frontal faces: Improving Person Recognition using multiple cues , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Gilly Leshed,et al.  How Blind People Interact with Visual Content on Social Networking Services , 2016, CSCW.

[32]  Eunhyeok Park,et al.  Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications , 2015, ICLR.

[33]  Gregory Gelly,et al.  Improving Speaker Diarization of TV Series using Talking-Face Detection and Clustering , 2016, ACM Multimedia.

[34]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[35]  Jian Cheng,et al.  Quantized Convolutional Neural Networks for Mobile Devices , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Sethuraman Panchanathan,et al.  Social Interaction Assistant: A Person-Centered Approach to Enrich Social Interactions for Individuals With Visual Impairments , 2016, IEEE Journal of Selected Topics in Signal Processing.

[37]  Angelo Cangelosi,et al.  Head pose estimation in the wild using Convolutional Neural Networks and adaptive gradient methods , 2017, Pattern Recognit..

[38]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[39]  Anderson Rocha,et al.  A Kinect-Based Wearable Face Recognition System to Aid Visually Impaired Users , 2017, IEEE Transactions on Human-Machine Systems.

[40]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Yuhang Zhao,et al.  A Face Recognition Application for People with Visual Impairments: Understanding Use Beyond the Lab , 2018, CHI.

[42]  Sheldon Cohen,et al.  Social Integration and Age-Related Decline in Lung Function , 2018, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[43]  Radu Horaud,et al.  Audio-Visual Speaker Diarization Based on Spatiotemporal Bayesian Fusion , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.