Monitoring System for Patients Using Multimedia for Smart Healthcare

The use of multimodal inputs in a smart healthcare framework is promising due to the increase in accuracy of the systems involved in the framework. In this paper, we propose a user satisfaction detection system using two multimedia contents, namely, speech and image. The three classes of satisfaction are satisfied, not satisfied, and indifferent. In the proposed system, speech and facial image of the user are captured, transmitted to a cloud, and then analyzed. A decision on the satisfaction is then delivered to the appropriate stakeholders. Several features from these two inputs are extracted from the cloud. For speech, directional derivatives of a spectrogram are used as features, whereas for image, a local binary pattern of the image is used to extract features. These features are combined and input to a support vector machine-based classifier. It is shown that the proposed system achieves up to 93% accuracy in detecting satisfaction.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  Musaed Alhussein.  Automatic facial emotion recognition using weber local descriptor for e-Healthcare system , 2016, Cluster Computing.

[3]  Ghulam Muhammad,et al.  Formant analysis in dysphonic patients and automatic Arabic digit speech recognition , 2011, Biomedical engineering online.

[4]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[5]  Vincent Lepetit,et al.  Pose-specific non-linear mappings in feature space towards multiview facial expression recognition , 2017, Image Vis. Comput..

[6]  Ahmed Ghoneim,et al.  A Facial-Expression Monitoring System for Improved Healthcare in Smart Cities , 2017, IEEE Access.

[7]  M. Shamim Hossain,et al.  Audio–Visual Emotion-Aware Cloud Gaming Framework , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Jesús B. Alonso,et al.  New approach in quantification of emotional intensity from the speech signal: emotional temperature , 2015, Expert Syst. Appl..

[10]  M. Shamim Hossain,et al.  Software defined healthcare networks , 2015, IEEE Wireless Communications.

[11]  M. Shamim Hossain,et al.  Cloud-assisted Industrial Internet of Things (IIoT) - Enabled framework for health monitoring , 2016, Comput. Networks.

[12]  Hatem Ben Sta,et al.  Quality and the efficiency of data in "Smart-Cities" , 2017, Future Gener. Comput. Syst..

[13]  R. Balasubramanian,et al.  Integrating Geometric and Textural Features for Facial Emotion Classification Using SVM Frameworks , 2016, CVIP.

[14]  Muhammad Ghulam,et al.  User emotion recognition from a larger pool of social network data using active learning , 2017, Multimedia Tools and Applications.

[15]  Björn W. Schuller,et al.  Deep neural networks for acoustic emotion recognition: Raising the benchmarks , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Samarendra Dandapat,et al.  A novel breathiness feature for analysis and classification of speech under stress , 2015, 2015 Twenty First National Conference on Communications (NCC).

[17]  M. Shamim Hossain,et al.  Healthcare Big Data Voice Pathology Assessment Framework , 2016, IEEE Access.

[18]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[19]  M. Shamim Hossain,et al.  Spectro-temporal directional derivative based automatic speech recognition for a serious game scenario , 2014, Multimedia Tools and Applications.

[20]  Giuseppe De Pietro,et al.  A framework for ECG denoising for mobile devices , 2015, PETRA.

[21]  Sazali Yaacob,et al.  Improved Emotion Recognition Using Gaussian Mixture Model and Extreme Learning Machine in Speech and Glottal Signals , 2015 .

[22]  Elmar Nöth,et al.  Private emotions versus social interaction: a data-driven approach towards analysing emotion in speech , 2008, User Modeling and User-Adapted Interaction.

[23]  M. Shamim Hossain,et al.  Emotion-Aware Connected Healthcare Big Data Towards 5G , 2018, IEEE Internet of Things Journal.

[24]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[25]  Diego López-de-Ipiña,et al.  Citizen-centric data services for smarter cities , 2017, Future Gener. Comput. Syst..

[26]  K. YogeshC.,et al.  A new hybrid PSO assisted biogeography-based optimization for emotion and stress recognition from speech signal , 2017, Expert Syst. Appl..

[27]  Ghulam Muhammad.  Automatic speech recognition using interlaced derivative pattern for cloud based healthcare system , 2015 .

[28]  M. Shamim Hossain,et al.  Patient State Recognition System for Healthcare Using Speech and Facial Expressions , 2016, Journal of Medical Systems.

[29]  M. Shamim Hossain,et al.  Big Data-Driven Service Composition Using Parallel Clustered Particle Swarm Optimization in Mobile Environment , 2016, IEEE Transactions on Services Computing.