Effects of objective feedback of facial expression recognition during video support chat

This paper examines the effects of objective feedback of facial expression recognition on the operators of a video support chat system. We have built a facial expression feedback (FEF) system, which recognizes the users' facial expressions, and displays a visual summary of the recognition results to the user, in order to raise self-awareness of his/her expressions during video communication. We evaluated our system in a supposed simulation of a call center, in which operators provide assistance to customers about how to use their smartphones. Experimental results have verified that our approach is useful to help operators improve their self-awareness of facial expression in the video support chat.

[1]  Jung P. Shim,et al.  Trust in videoconferencing , 2006, CACM.

[2]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[3]  Ansgar E. Depping,et al.  Through the Looking Glass: The Effects of Feedback on Self-Awareness and Conversational Behaviour during Video Chat , 2017, CHI.

[4]  S. Pugh,et al.  Service with a Smile: Emotional Contagion in the Service Encounter , 2001 .

[5]  Yuchi Huang,et al.  Mirroring Facial Expressions: Evidence from Visual Analysis of Dyadic Interactions , 2016, ICMR.

[6]  Randall Sadler,et al.  Comparing six video chat tools: A critical evaluation by language teachers , 2009, Comput. Educ..

[7]  Morgan G. Ames,et al.  Making love in the network closet: the benefits and work of family videochat , 2010, CSCW '10.

[8]  Yuan Tang,et al.  TF.Learn: TensorFlow's High-level Module for Distributed Machine Learning , 2016, ArXiv.

[9]  Xiaodong Cui,et al.  Data Augmentation for Deep Neural Network Acoustic Modeling , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[10]  Gwen Littlewort,et al.  Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction. , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[11]  Yang Song,et al.  Improving the Robustness of Deep Neural Networks via Stability Training , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  John C. Tang,et al.  Focusing on shared experiences: moving beyond the camera in video communication , 2012, DIS '12.

[13]  David D. Cox,et al.  Hyperparameter Optimization and Boosting for Classifying Facial Expressions: How good can a "Null" Model be? , 2013, ICML 2013.

[14]  Yichuan Tang,et al.  Deep Learning using Linear Support Vector Machines , 2013, 1306.0239.

[15]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  Hui Zhang,et al.  “You Should Have Seen the Look on Your Face…”: Self-awareness of Facial Expressions , 2017, Front. Psychol..

[17]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[18]  Jeff B. Pelz,et al.  Eye contact and video-mediated communication: A review , 2013, Displays.