Face Recognition in Video Streams for Mobile Assistive Devices Dedicated to Visually Impaired

In this paper, we introduce a novel face detection and recognition system based on deep convolutional networks, designed to improve the visually impaired users' interaction and communication in social encounters. A first feature of the proposed architecture concerns a face detection system able to identify various persons existent in the scene regardless of the subject location or pose. Then, the faces are tracked between successive frames using a CNN (Convolutional Neural Networks) based tracker trained offline with generic motion patterns. The system can handle face occlusion, rotation or pose variation, as well as important illumination changes. Finally, the faces are recognized, in real-time, directly from the video stream. The major contribution of the paper consists in a novel weight adaptation scheme able to determine the relevance of face instances and to create a global, fixed-size representation from all face instances tracked during the video stream. The experimental evaluation performed on a set of 30 video elements validates the approach with average detection and recognition scores superior to 85%.