Multi-Stream Deep Convolutional Network Using High-Level Features Applied to Fall Detection in Video Sequences

Sporadic falls, due to the lack of balance and other factors, are some of the complications that elderly people might experience more frequently than others. Accordingly, as there is a high probability of these events causing major health casualties, such as bone breaking or head clots, studies have been monitoring these falls to rapidly assist the victim. In this work, we propose and evaluate a multi-stream learning model based on convolutional neural networks using high-level handcrafted features as input in order to cope with this situation. Therefore, our approach consists of extracting high-level handcrafted features, for instance, human pose estimation and optical flow, and using each one as an input for a distinct VGG-16 classifier. In addition, these experiments are able to showcase what features can be used in fall detection. The results have shown that by assembling our directed input learners, our approach outperforms, in terms of accuracy and sensitivity rates, to other similar tested methods found in literature.

[1]  Diwakar Gautam,et al.  Abnormal behaviour detection at traffic junctions using Lucas Kanade and Harris Corner detector , 2018, 2018 4th International Conference on Recent Advances in Information Technology (RAIT).

[2]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[3]  Lesya Anishchenko,et al.  Machine learning in video surveillance for fall detection , 2018, 2018 Ural Symposium on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT).

[4]  Mubarak Shah,et al.  UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild , 2012, ArXiv.

[5]  Stefanos D. Kollias,et al.  Computer vision based fall detection by a convolutional neural network , 2017, ICMI.

[6]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Irene Y. H. Gu,et al.  Fall detection in RGB-D videos by combining shape and motion features , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Jun Zhou,et al.  Recognizing human actions from silhouettes described with weighted distance metric and kinematics , 2017, Multimedia Tools and Applications.

[9]  Marcelo Bernardes Vieira,et al.  Multi-stream Convolutional Neural Networks for Action Recognition in Video Sequences Based on Adaptive Visual Rhythms , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[10]  Bogdan Kwolek,et al.  Improving fall detection by the use of depth sensor and accelerometer , 2015, Neurocomputing.

[11]  Zhe Wang,et al.  Towards Good Practices for Very Deep Two-Stream ConvNets , 2015, ArXiv.

[12]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Nasser Kehtarnavaz,et al.  A Convolutional Neural Network-Based Sensor Fusion System for Monitoring Transition Movements in Healthcare Applications , 2018, 2018 IEEE 14th International Conference on Control and Automation (ICCA).

[14]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[15]  Zhihai He,et al.  Recognizing Falls from Silhouettes , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  Hao Chen,et al.  A Fall Detection System Based on Human Body Silhouette , 2013, 2013 Ninth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[17]  Chia-Wen Lin,et al.  Compressed-domain fall incident detection for intelligent home surveillance , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[18]  Nabil Zerrouki,et al.  Fall detection using supervised machine learning algorithms: A comparative study , 2016, 2016 8th International Conference on Modelling, Identification and Control (ICMIC).

[19]  Miao Yu,et al.  Fall detection in the elderly by head tracking , 2009, 2009 IEEE/SP 15th Workshop on Statistical Signal Processing.

[20]  Miao Yu,et al.  A robust fall detection system for the elderly in a Smart Room , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Ignacio Arganda-Carreras,et al.  Vision-Based Fall Detection with Convolutional Neural Networks , 2017, Wirel. Commun. Mob. Comput..

[22]  Xiaogang Li,et al.  Fall detection for elderly person care using convolutional neural networks , 2017, 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[23]  Frank Vahid,et al.  Automated fall detection on privacy-enhanced video , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[24]  Bogdan Kwolek,et al.  Human fall detection on embedded platform using depth maps and wireless accelerometer , 2014, Comput. Methods Programs Biomed..

[25]  Mooi Choo Chuah,et al.  Dense Optical Flow Based Emotion Recognition Classifier , 2015, 2015 IEEE 12th International Conference on Mobile Ad Hoc and Sensor Systems.

[26]  Li Feng,et al.  Deep Learning for Fall Detection: Three-Dimensional CNN Combined With LSTM on Video Kinematic Data , 2019, IEEE Journal of Biomedical and Health Informatics.

[27]  Nabil Zerrouki,et al.  Combined curvelets and hidden Markov models for human fall detection , 2018, Multimedia Tools and Applications.

[28]  Haibo Wang,et al.  Silhouette Orientation Volumes for Efficient Fall Detection in Depth Videos , 2017, IEEE Journal of Biomedical and Health Informatics.

[29]  Jean Meunier,et al.  Robust Video Surveillance for Fall Detection Based on Human Shape Deformation , 2011, IEEE Transactions on Circuits and Systems for Video Technology.