Emotion-Aware Connected Healthcare Big Data Towards 5G

The recent development of big data-oriented wireless technologies in terms of emerging 5G, edge computing, interconnected devices of the Internet of Things (IoT), and data analytics, as well as techniques, have enabled connected healthcare services for a happier and healthier life. Although, the quality of the healthcare services can be enhanced through big data-oriented wireless technologies, however, the challenges remain for not considering emotional care, especially for children, elderly, and mentally ill people. In this paper, we propose an emotion-aware connected healthcare system using a powerful emotion detection module. Different IoT devices are used to capture speech and image signals of a patient in a smart home scenario. These signals are used as the input to the emotion detection module. Speech and image signals are processed separately, and classification scores using these signals are fused to produce a final score to take a decision about the emotion. If the emotion is detected as pain, caregivers can visit the patient. Several experiments were performed to validate the proposed system, and good accuracies, up to 99.87%, were achieved for emotion detection. The proposed framework would greatly contribute personalized and seamless emotion-aware healthcare services toward 5G.

[1]  Daniel Díaz Sánchez,et al.  Major requirements for building Smart Homes in Smart Cities based on Internet of Things technologies , 2017, Future Gener. Comput. Syst..

[2]  Benoit Huet,et al.  Toward emotion indexing of multimedia excerpts , 2008, 2008 International Workshop on Content-Based Multimedia Indexing.

[3]  Nasrollah Moghaddam Charkari,et al.  Audiovisual emotion recognition using ANOVA feature selection method and multi-classifier neural networks , 2014, Neural Computing and Applications.

[4]  Victor C. M. Leung,et al.  EMC: Emotion-aware mobile cloud computing in 5G , 2015, IEEE Network.

[5]  Muhammad Ghulam.  Automatic speech recognition using interlaced derivative pattern for cloud based healthcare system , 2015, Cluster Computing.

[6]  Fadel Adib,et al.  Emotion recognition using wireless signals , 2016, MobiCom.

[7]  Nasrollah Moghaddam Charkari,et al.  Multimodal information fusion application to human emotion recognition from face and speech , 2010, Multimedia Tools and Applications.

[8]  Arkady B. Zaslavsky,et al.  Context Aware Computing for The Internet of Things: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[9]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jean Addington,et al.  Mapping structural covariance networks of facial emotion recognition in early psychosis: A pilot study , 2017, Schizophrenia Research.

[11]  Hassan Ghasemzadeh,et al.  Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges , 2017, Inf. Fusion.

[12]  M. Shamim Hossain,et al.  Audio-Visual Emotion Recognition Using Big Data Towards 5G , 2016, Mob. Networks Appl..

[13]  Eric Horvitz,et al.  Addressing bias in machine learning algorithms: A pilot study on emotion recognition for intelligent systems , 2017, 2017 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO).

[14]  M. Shamim Hossain,et al.  Cloud-oriented emotion feedback-based Exergames framework , 2017, Multimedia Tools and Applications.

[15]  M. Shamim Hossain,et al.  Cloud-assisted Industrial Internet of Things (IIoT) - Enabled framework for health monitoring , 2016, Comput. Networks.

[16]  Christopher Joseph Pal,et al.  EmoNets: Multimodal deep learning approaches for emotion recognition in video , 2015, Journal on Multimodal User Interfaces.

[17]  Min Chen,et al.  Emotion Communication System , 2017, IEEE Access.

[18]  M. Shamim Hossain,et al.  Cloud-Supported Cyber–Physical Localization Framework for Patients Monitoring , 2017, IEEE Systems Journal.

[19]  K. YogeshC.,et al.  A new hybrid PSO assisted biogeography-based optimization for emotion and stress recognition from speech signal , 2017, Expert Syst. Appl..

[20]  Min Chen,et al.  A 5G Cognitive System for Healthcare , 2017, Big Data Cogn. Comput..

[21]  Theodoros Iliou,et al.  Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011 , 2012, Artificial Intelligence Review.

[22]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[23]  Muhammad Ghulam,et al.  User emotion recognition from a larger pool of social network data using active learning , 2017, Multimedia Tools and Applications.

[24]  Ahmed Ghoneim,et al.  A Facial-Expression Monitoring System for Improved Healthcare in Smart Cities , 2017, IEEE Access.

[25]  M. Shamim Hossain,et al.  Audio–Visual Emotion-Aware Cloud Gaming Framework , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  M. Shamim Hossain,et al.  An Emotion Recognition System for Mobile Applications , 2017, IEEE Access.

[27]  M. Shamim Hossain,et al.  Audio-visual emotion recognition using multi-directional regression and Ridgelet transform , 2016, Journal on Multimodal User Interfaces.

[28]  Dongmei Jiang,et al.  Audio Visual Emotion Recognition Based on Triple-Stream Dynamic Bayesian Network Models , 2011, ACII.

[29]  Honglak Lee,et al.  Deep learning for robust feature generation in audiovisual emotion recognition , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[30]  Leonardo Tozzi,et al.  Functional magnetic resonance imaging correlates of emotion recognition and voluntary attentional regulation in depression: A generalized psycho-physiological interaction study. , 2017, Journal of affective disorders.