Teaching emotion expressions to a human companion robot using deep neural architectures

Human companion robots need to be sociable and responsive towards emotions to better interact with the human environment they are expected to operate in. This paper is based on the Neuro-Inspired COmpanion robot (NICO) and investigates a hybrid, deep neural network model to teach the NICO to associate perceived emotions with expression representations using its on-board capabilities. The proposed model consists of a Convolutional Neural Network (CNN) and a Self-organising Map (SOM) to perceive the emotions expressed by a human user towards NICO and trains two parallel Multilayer Perceptron (MLP) networks to learn general as well as person-specific associations between perceived emotions and the robot's facial expressions.

[1]  Stefan Wermter,et al.  MIRA: A Learning Multimodal Interactive Robot Agent , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[2]  W. Cannon.  The James-Lange theory of emotions: a critical examination and an alternative theory. By Walter B. Cannon, 1927. , 1927, American Journal of Psychology.

[3]  Andrea Lockerd Thomaz,et al.  Teachable robots: Understanding human teaching behavior to build more effective robot learners , 2008, Artif. Intell..

[4]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[5]  Sang Won Lee,et al.  Neural correlates of text‐based emoticons: a preliminary fMRI study , 2016, Brain and behavior.

[6]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[7]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[8]  J. Russell.  A circumplex model of affect. , 1980 .

[9]  C. F. Kossack,et al.  Rank Correlation Methods , 1949 .

[10]  Eva Hudlicka,et al.  What Are We Modeling When We Model Emotion? , 2008, AAAI Spring Symposium: Emotion, Personality, and Social Behavior.

[11]  James Everett Young,et al.  Animal-inspired human-robot interaction: A robotic tail for communicating state , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[12]  P. Johnson-Laird,et al.  Towards a Cognitive Theory of Emotions , 1987 .

[13]  Cynthia Breazeal,et al.  Emotion and sociable humanoid robots , 2003, Int. J. Hum. Comput. Stud..

[14]  P. Stone,et al.  TAMER: Training an Agent Manually via Evaluative Reinforcement , 2008, 2008 7th IEEE International Conference on Development and Learning.

[15]  Britta Wrede,et al.  The social robot ‘Flobi’: Key concepts of industrial design , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[16]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[17]  Andrea Lockerd Thomaz,et al.  Policy Shaping: Integrating Human Feedback with Reinforcement Learning , 2013, NIPS.

[18]  Stefan Wermter,et al.  Developing crossmodal expression recognition based on a deep neural model , 2016, Adapt. Behav..

[19]  Jean Carletta,et al.  Assessing Agreement on Classification Tasks: The Kappa Statistic , 1996, CL.

[20]  Stefan Kopp,et al.  Towards a Common Framework for Multimodal Generation: The Behavior Markup Language , 2006, IVA.

[21]  James Everett Young,et al.  The Roomba mood ring: An ambient-display robot , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[22]  Stefan Wermter,et al.  NICO — Neuro-inspired companion: A developmental humanoid robot platform for multimodal interaction , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[23]  Daniel Kudenko,et al.  Reinforcement Learning with Reward Shaping and Mixed Resolution Function Approximation , 2009, Int. J. Agent Technol. Syst..

[24]  Hiroshi G. Okuno,et al.  The MEI Robot: Towards Using Motherese to Develop Multimodal Emotional Intelligence , 2014, IEEE Transactions on Autonomous Mental Development.

[25]  P. Ekman.  An argument for basic emotions , 1992 .

[26]  J. Fuster.  Upper processing stages of the perception–action cycle , 2004, Trends in Cognitive Sciences.

[27]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[28]  Giulio Sandini,et al.  The iCub humanoid robot: An open-systems platform for research in cognitive development , 2010, Neural Networks.