Emotion-Age-Gender-Nationality Based Intention Understanding in Human–Robot Interaction Using Two-Layer Fuzzy Support Vector Regression

An intention understanding model based on two-layer fuzzy support vector regression is proposed in human–robot interaction, where fuzzy c-means clustering is used to classify the input data, and intention understanding is mainly obtained by emotion, with identification information such as age, gender, and nationality. It aims to realize the transparent communication by understanding customers’ order intentions at a bar, in such a way that the social relationship between bar staffs and customers becomes smooth. To demonstrate the aptness of intention understanding model, experiments are designed in term of relationship between emotion-age-gender-nationality and order intention. Results show that the proposal obtains an intention understanding accuracy of 70 %/72 %/80 % with clusters number $$C= $$C= 2/3/6 (according to different genders/ages/nationalities), which is 23 %/26 %/33 % and 35.5 %/37.5 %/45.5 % higher than that of support vector regression (SVR) and back propagation neural networks (BPNN), respectively; the computational time of proposal is about 0.976 s/0.935 s/0.67 s with clusters number $$C=$$C= 2/3/6, while 1.889 s for SVR and 3.505 s for BPNN. Additionally, the preliminary application experiment is performed in the developing human–robot interaction system, called mascot robot system, where the experiment is performed in a scenario of “drinking at a bar”, result shows that the bar lady robot obtains an accuracy of 77.8 % for understanding customers’ order intentions and receives a satisfaction evaluation of “satisfied”. According to the preliminary application, the proposal is being extended to an ordering system in the bar for business communication.

[1]  F. S. Wong,et al.  Fuzzy weighted averages and implementation of the extension principle , 1987 .

[2]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[3]  Luc Van Gool,et al.  Real-time 3D hand gesture interaction with a robot for understanding directions from humans , 2011, 2011 RO-MAN.

[4]  Jochen Musch,et al.  Affective priming: Findings and theories. , 2003 .

[5]  Léon Bottou,et al.  Local Learning Algorithms , 1992, Neural Computation.

[6]  Kaoru Hirota,et al.  Emotion Recognition of Violin Music based on Strings Music Theory for Mascot Robot System , 2012, ICINCO.

[7]  Darwin G. Caldwell,et al.  Learning-based control strategy for safe human-robot interaction exploiting task and robot redundancies , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  G Farchi,et al.  Alcohol and survival in the Italian rural cohorts of the Seven Countries Study. , 2000, International journal of epidemiology.

[9]  Majid Moavenian,et al.  A qualitative comparison of Artificial Neural Networks and Support Vector Machines in ECG arrhythmias classification , 2010, Expert Syst. Appl..

[10]  Nikolaos Papanikolopoulos,et al.  Coordination and Longevity in Multi-Robot Teams Involving Miniature Robots , 2013, J. Intell. Robotic Syst..

[11]  Robin R. Murphy,et al.  Human-Robot Interaction , 2012 .

[12]  Monica N. Nicolescu,et al.  Understanding human intentions via Hidden Markov Models in autonomous mobile robots , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[13]  Reid G. Simmons,et al.  Affective social robots , 2010, Robotics Auton. Syst..

[14]  Marc Hanheide,et al.  Expectations, Intentions, and Actions in Human-Robot Interaction , 2012, Int. J. Soc. Robotics.

[15]  Kaoru Hirota,et al.  Multi-robot behavior adaptation to local and global communication atmosphere in humans-robots interaction , 2014, Journal on Multimodal User Interfaces.

[16]  Seung-won Hwang,et al.  An efficient method for learning nonlinear ranking SVM functions , 2012, Inf. Sci..

[17]  Fangyan Dong,et al.  Development of Mascot Robot System in NEDO project , 2008, 2008 4th International IEEE Conference Intelligent Systems.

[18]  Takayuki Kanda,et al.  Adapting Robot Behavior for Human--Robot Interaction , 2008, IEEE Transactions on Robotics.

[19]  Fangyan Dong,et al.  Fuzzy Inference based Mentality Expression for Eye Robot in Affinity Pleasure-Arousal Space , 2008 .

[20]  J. Kelly,et al.  The Evolutionary Significance of Affect in Groups: Communication and Group Bonding , 2004 .

[21]  K. C. Klauer,et al.  The Psychology of Evaluation : Affective Processes in Cognition and Emotion , 2003 .

[22]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[23]  Stephen Burgess,et al.  Understanding citizens' behavioural intention in the adoption of e-government services in the state of Qatar , 2009 .

[24]  Jon Atli Benediktsson,et al.  SVM- and MRF-Based Method for Accurate Classification of Hyperspectral Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[25]  Kaoru Hirota,et al.  Communication atmosphere in humans and robots interaction based on the concept of fuzzy atmosfield generated by emotional states of humans and robots , 2013 .

[26]  Chia-Hui Huang,et al.  A reduced support vector machine approach for interval regression analysis , 2012, Inf. Sci..

[27]  James C. Bezdek,et al.  A Convergence Theorem for the Fuzzy ISODATA Clustering Algorithms , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Illah R. Nourbakhsh,et al.  Planning for Human–Robot Interaction in Socially Situated Tasks , 2013, Int. J. Soc. Robotics.

[29]  Ehud Sharlin,et al.  Evaluating Human-Robot Interaction , 2011, Int. J. Soc. Robotics.

[30]  Kaoru Hirota,et al.  Deep Level Situation Understanding and its Application to Casual Communication between Robots and Humans , 2018, ICINCO.

[31]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[32]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[33]  Martin Lauer,et al.  A generative model for 3D urban scene understanding from movable platforms , 2011, CVPR 2011.

[34]  Danica Kragic,et al.  Enhanced visual scene understanding through human-robot dialog , 2011, IROS 2011.

[35]  Carol Holland,et al.  The effect of age, gender and driver status on pedestrians' intentions to cross the road in risky situations. , 2007, Accident; analysis and prevention.

[36]  D. Keltner,et al.  Social Functions of Emotions at Four Levels of Analysis , 1999 .

[37]  Ruhaidah Samsudin,et al.  A hybrid model of self-organizing maps (SOM) and least square support vector machine (LSSVM) for time-series forecasting , 2011, Expert Syst. Appl..

[38]  Danica Kragic,et al.  Enhanced visual scene understanding through human-robot dialog , 2010, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  Kaoru Hirota,et al.  Adapting Multi-Robot Behavior to Communication Atmosphere in Humans-Robots Interaction Using Fuzzy Production Rule Based Friend-Q Learning , 2013, J. Adv. Comput. Intell. Intell. Informatics.

[40]  Maja J. Mataric,et al.  Using Socially Assistive Human–Robot Interaction to Motivate Physical Exercise for Older Adults , 2012, Proceedings of the IEEE.

[41]  R. Orji,et al.  Impact of Gender and Nationality on Acceptance of a Digital Library: An Empirical Validation of Nationality Based UTAUT Using SEM , 2010 .

[42]  K. Ochsner,et al.  The role of social cognition in emotion , 2008, Trends in Cognitive Sciences.

[43]  Estela Bicho,et al.  The power of prediction: Robots that read intentions , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[44]  B. Keysar,et al.  The Illusory Transparency of Intention: Linguistic Perspective Taking in Text , 1994, Cognitive Psychology.

[45]  A. Paulo Coimbra,et al.  Adaptive PD Controller Modeled via Support Vector Regression for a Biped Robot , 2013, IEEE Transactions on Control Systems Technology.

[46]  Stephanie Bauer,et al.  The role of emotion in bridging the intention–behaviour gap: The case of sports participation , 2009 .

[47]  Hiroshi Ishiguro,et al.  Robovie-IV: A Communication Robot Interacting with People Daily in an Office , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[48]  Johan A. K. Suykens,et al.  Optimal control by least squares support vector machines , 2001, Neural Networks.