Visual-based sentiment logging in magic smart mirrors

This paper describes the hardware and software architectures of a smart magic mirror able to acquire and track the user’s face, recognize his identity, analyze and log his facial expressions and emotional states. The magic mirror is basically a see-through mirror made smart by a led display placed behind the mirror that enables to display the User Interface (UI). The mirror is connected to a small single-board computer attached to a set of input sensors (a traditional RGB camera to enable vision-based interaction, a microphone to enable voice interaction, temperature and humidity sensors, and proximity sensors) and to an embedded machine intelligence platform that performs all the neural computations.

[1]  Nassir Navab,et al.  mirracle: An augmented reality magic mirror system for anatomy education , 2012, 2012 IEEE Virtual Reality Workshops (VRW).

[2]  Raimondo Schettini,et al.  Fine-Grained Face Annotation Using Deep Multi-Task CNN , 2018, Sensors.

[3]  Raimondo Schettini,et al.  Robust smile detection using convolutional neural networks , 2016, J. Electronic Imaging.

[4]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[5]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Simone Bianco,et al.  Large Age-Gap face verification by feature injection in deep networks , 2016, Pattern Recognit. Lett..

[7]  Tal Hassner,et al.  Age and Gender Estimation of Unfiltered Faces , 2014, IEEE Transactions on Information Forensics and Security.

[8]  Paolo Napoletano,et al.  Combining heart and breathing rate for car driver stress recognition , 2018, 2018 IEEE 8th International Conference on Consumer Electronics - Berlin (ICCE-Berlin).

[9]  Mohammad H. Mahoor,et al.  AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild , 2017, IEEE Transactions on Affective Computing.

[10]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[13]  Davis E. King.  Max-Margin Object Detection , 2015, ArXiv.