An intelligent monitoring system for indoor safety of individuals suffering from Autism Spectrum Disorder (ASD)

Autism Spectrum Disorder (ASD) is distinguished by a variety of behavioral and social deficits. One of the most prominent problems in an autistic child is their aggressive nature that can cause physical injuries. In order to address the following behavioral deficits, this paper is an attempt to present a novel monitoring framework to predict irregularities from the physical activities of an autistic child. The primary motive behind this study is to improve the safety measures for the child in an indoor environment by generating time sensitive-alerts for caretakers and doctors. In this study, a deep 3D CNN and LSTM based activity prediction methodology are proposed to recognize physical irregularities. The 3D CNN model extracts the spatio-temporal features from the video templates for stance prediction with subject position. The LSTM model calculates the temporal relationship in the feature maps to analyze the scale of irregularity. Moreover, in order to deal with such irregularities, a time-sensitive alert-based decision process is proposed in the present work to generate early warnings to the doctor and caretaker. The proficiency of the system is increased by storing the performed activity scores in the local database of the system which can be further utilized to provide medicinal or therapeutic assistance. In addition, the measured experimental results are validated with state-of-the-art methodologies. Hence, the calculated statistical measures justify the utility of the proposed study in the healthcare and assistive-care domain.

[1]  R. Koegel,et al.  Autistic self-stimulation and intertrial interval duration. , 1983, American journal of mental deficiency.

[2]  C Ounsted,et al.  The biological significance of gaze aversion with particular reference to the syndrome of infantile autism. , 1966, Behavioral science.

[3]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[4]  B. J. Myers,et al.  “My greatest joy and my greatest heart ache:” Parents’ own words on how having a child in the autism spectrum has affected their lives and their families’ lives , 2009 .

[5]  Pavlo Molchanov,et al.  Multilayer and Multimodal Fusion of Deep Neural Networks for Video Classification , 2016, ACM Multimedia.

[6]  Wenjun Zeng,et al.  An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data , 2016, AAAI.

[7]  Yufeng Jin,et al.  Mobile Human Airbag System for Fall Protection Using MEMS Sensors and Embedded SVM Classifier , 2009, IEEE Sensors Journal.

[8]  Andrew Zisserman,et al.  Convolutional Two-Stream Network Fusion for Video Action Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  C. Becker,et al.  Evaluation of a fall detector based on accelerometers: A pilot study , 2005, Medical and Biological Engineering and Computing.

[10]  Yi Yang,et al.  A discriminative CNN video representation for event detection , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Pascal Poupart,et al.  Towards a mobility diagnostic tool: Tracking rollator users' leg pose with a monocular vision system , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Shivam Singhal,et al.  Action Recognition Framework Based on Normalized Local Binary Pattern , 2018, Advances in Intelligent Systems and Computing.

[13]  C Hutt,et al.  CARDIAC ARRHYTHMIA AND BEHAVIOUR IN AUTISTIC CHILDREN , 1975, Acta psychiatrica Scandinavica.

[14]  Jong-Hoon Youn,et al.  Survey and evaluation of real-time fall detection approaches , 2009, 2009 6th International Symposium on High Capacity Optical Networks and Enabling Technologies (HONET).

[15]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[16]  Ling Shao,et al.  A survey on fall detection: Principles and approaches , 2013, Neurocomputing.

[17]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[18]  Catherine Lord,et al.  Genetics of childhood disorders: XLII. Autism, part 1: Diagnosis and assessment in autistic spectrum disorders. , 2002, Journal of the American Academy of Child and Adolescent Psychiatry.

[19]  Lin Sun,et al.  Human Action Recognition Using Factorized Spatio-Temporal Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[20]  Patrick M. Pilarski,et al.  First steps towards an intelligent laser welding architecture using deep neural networks and reinforcement learning , 2014 .

[21]  Gang Wang,et al.  NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Aggelos K. Katsaggelos,et al.  Wireless Video Surveillance: A Survey , 2013, IEEE Access.

[23]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[25]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[26]  M. Ganz,et al.  The lifetime distribution of the incremental societal costs of autism. , 2007, Archives of pediatrics & adolescent medicine.

[27]  Chalavadi Krishna Mohan,et al.  Human action recognition in RGB-D videos using motion sequence information and deep learning , 2017, Pattern Recognit..

[28]  Hassan Ghasemzadeh,et al.  Structural Action Recognition in Body Sensor Networks: Distributed Classification Based on String Matching , 2010, IEEE Transactions on Information Technology in Biomedicine.

[29]  Yale Song,et al.  Video co-summarization: Video summarization by visual co-occurrence , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Gang Wang,et al.  Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition , 2016, ECCV.

[31]  Hassan Ghasemzadeh,et al.  A Body Sensor Network With Electromyogram and Inertial Sensors: Multimodal Interpretation of Muscular Activities , 2010, IEEE Transactions on Information Technology in Biomedicine.

[32]  Matti Linnavuo,et al.  Detection of falls among the elderly by a floor sensor using the electric near field , 2010, IEEE Transactions on Information Technology in Biomedicine.

[33]  Can Zhang,et al.  STMP: Spatial Temporal Multi-level Proposal Network for Activity Detection , 2019, MMM.

[34]  Ripul Ghosh,et al.  Deep learning approach for human action recognition in infrared images , 2018, Cognitive Systems Research.

[35]  P. Barbara,et al.  Effect of sample preparation and excitation conditions on the single molecule spectroscopy of conjugated polymers. , 2006, The journal of physical chemistry. B.

[36]  Xiaochun Luo,et al.  Towards efficient and objective work sampling: Recognizing workers' activities in site surveillance videos with two-stream convolutional networks , 2018, Automation in Construction.

[37]  Rachid Oulad Haj Thami,et al.  Conception and development of a video surveillance system for detecting, tracking and profile analysis of a person , 2013, 2013 3rd International Symposium ISKO-Maghreb.

[38]  Simon A. Dobson,et al.  KCAR: A knowledge-driven approach for concurrent activity recognition , 2015, Pervasive Mob. Comput..

[39]  M N Nyan,et al.  Distinguishing fall activities from normal activities by angular rate characteristics and high-speed camera characterization. , 2006, Medical engineering & physics.

[40]  G. ÓLaighin,et al.  A proposal for the classification and evaluation of fall detectors Une proposition pour la classification et l'évaluation des détecteurs de chutes , 2008 .

[41]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[42]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[43]  Cordelia Schmid,et al.  P-CNN: Pose-Based CNN Features for Action Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[44]  Philip S. Yu,et al.  Spatiotemporal Pyramid Network for Video Action Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Richard P. Wildes,et al.  Spatiotemporal Multiplier Networks for Video Action Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Martin J. Wainwright,et al.  Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions , 2011, ICML.

[47]  G. Arbanas.  Diagnostic and Statistical Manual of Mental Disorders (DSM-5) , 2015 .

[48]  R. Hastings,et al.  Using Matched Groups to Explore Child Behavior Problems and Maternal Well-Being in Children with Down Syndrome and Autism , 2010, Journal of autism and developmental disorders.

[49]  N. Noury.  A smart sensor for the remote follow up of activity and fall detection of the elderly , 2002, 2nd Annual International IEEE-EMBS Special Topic Conference on Microtechnologies in Medicine and Biology. Proceedings (Cat. No.02EX578).