Using Facial Recognition Services as Implicit Feedback for Recommenders

User authentication and feedback gathering are crucial aspects for recommender systems. The most common implementations, a username / password login and star rating systems, require user interaction and a cognitive effort from the user. As a result, users opt to save their password in the interface and optional feedback with a star rating system is often skipped, especially for applica- tions such as video watching in a home environment. In this article, we propose an alternative method for user authentication based on facial recognition and an automatic feedback gathering method by detecting various face characteristics. Using facial recognition with a camera in a tablet, smartphone, or smart TV, the persons in front of the screen can be identified in order to link video watch- ing sessions to their user profile. During video watching, implicit feedback is automatically gathered through emotion recognition, attention measurements, and behavior analysis. An emotion finger- print, which is defined as a unique spectrum of expected emotions for a video scene, is compared to the recognized emotions in order to estimate the experience of a user while watching. An evaluation with a test panel showed that happiness can be most accurately detected and the recognized emotions are correlated with the user’s star rating.

[1]  Marko Tkalcic,et al.  Affective recommender systems: The role of emotions in recommender systems , 2011 .

[2]  Toon De Pessemier,et al.  Enhancing Recommender Systems for TV by Face Recognition , 2016, WEBIST.

[3]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[4]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[5]  Gediminas Adomavicius,et al.  Incorporating contextual information in recommender systems using a multidimensional approach , 2005, TOIS.

[6]  Judith Masthoff,et al.  Group Recommender Systems: Combining Individual Models , 2011, Recommender Systems Handbook.

[7]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Nicu Sebe,et al.  Exploiting facial expressions for affective video summarisation , 2009, CIVR '09.

[9]  Joemon M. Jose,et al.  Integrating facial expressions into user profiling for the improvement of a multimodal recommender system , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[10]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.