Deep interaction: Wearable robot-assisted emotion communication for enhancing perception and expression ability of children with Autism Spectrum Disorders

Abstract Recent changes in both the social economy and people’s living conditions and their habits have had an influence on the incidence of Autism Spectrum Disorder (ASD), which has brought huge economic and mental burden to the society , and has become an urgent public health problem. The main symptom of autism in children is the presence of a social barrier, and one of the main reasons for that is a lack of emotional cognitive ability. However, the existing autism-treatment systems intended for children pay little attention to the emotion cognition disorder. Besides, too much importance has been given to the interaction with children, while limiting the timeliness and movability of these systems. With the aim to address this shortcoming, in this work, we focus on the emotion cognition disorder and design a feasible solution for enhancing perception and expression ability of children with ASD. First, the first-view emotional care system for children with ASD (First-ECS) is developed using a wearable robot as a system carrier and realizing the deep emotional interaction with children with autism from the first-view perspective. Emotion communication is used to meet high timeliness requirements for emotion transmission in the First-ECS. Next, the emotional interaction mechanism that is applicable to the line of sight and non-line of sight communication scenarios is introduced. Also, the emotion perception engine and emotion expression engine are designed. Subsequently, multimodal data collection and processing by a wearable affective robot are discussed. In addition, this paper introduces a multimodal data fusion method from the angle of emotion relevance and emotion computing model based on audio-visual data. Finally, a demo platform is built to verify the feasibility of the proposed system.

[1]  Qingsong Ai,et al.  Feature extraction of four-class motor imagery EEG signals based on functional brain network , 2019, Journal of neural engineering.

[2]  Zdenko Kovacic,et al.  Design and Validation of MOMDP Models for Child–Robot Interaction Within Tasks of Robot-Assisted ASD Diagnostic Protocol , 2020, Int. J. Soc. Robotics.

[3]  Chao Cai,et al.  Smart Home Based on WiFi Sensing: A Survey , 2018, IEEE Access.

[4]  Dong Yu,et al.  Exploring convolutional neural network structures and optimization techniques for speech recognition , 2013, INTERSPEECH.

[5]  Pierluigi Carcagnì,et al.  Automatic Emotion Recognition in Robot-Children Interaction for ASD Treatment , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[6]  Murat Gök,et al.  A novel machine learning model to predict autism spectrum disorders risk gene , 2019, Neural Computing and Applications.

[7]  R. Ponnusamy,et al.  Toward Design and Enhancement of Emotion Recognition System Through Speech Signals of Autism Spectrum Disorder Children for Tamil Language Using Multi-Support Vector Machine , 2018 .

[8]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[9]  Min Chen,et al.  Emotion Communication System , 2017, IEEE Access.

[10]  Ramesh Raskar,et al.  Generative Street Addresses from Satellite Imagery , 2018, ISPRS Int. J. Geo Inf..

[11]  Satoshi Nakamura,et al.  Embodied conversational agents for multimodal automated social skills training in people with autism spectrum disorders , 2017, PloS one.

[12]  Wei Liu,et al.  A Survey on Deep Learning Empowered IoT Applications , 2019, IEEE Access.

[13]  Charles E. Hughes,et al.  Exceptionally Social: Design of an Avatar-Mediated Interactive System for Promoting Social Skills in Children with Autism , 2017, CHI Extended Abstracts.

[14]  Wen Gao,et al.  Speech Emotion Recognition Using Deep Convolutional Neural Network and Discriminant Temporal Pyramid Matching , 2018, IEEE Transactions on Multimedia.

[15]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[16]  Qingsong Ai,et al.  An efficient in-network caching decision algorithm for Internet of things , 2018, Int. J. Commun. Syst..

[17]  Qingsong Ai,et al.  Wireless Body Area Network Mobility-Aware Task Offloading Scheme , 2018, IEEE Access.