An Edge-device Based Fast Fall Detection Using Spatio-temporal Optical Flow Model

The elderly fall detection is one critical function in health of the elderly. A real-time fall detection for the elderly has been a significant healthcare issue. The traditional video analysis on cloud has large communication overhead. In this paper, a fast fall detection system based on the spatio-temporal optical flow model is proposed, which is further deeply compressed by a structured tensorization towards an implementation on edge devices. Firstly, an object extractor is built to extract motion objects from video clips. The spatio-temporal optical flow model is formed to estimate optical flow fields of motion objects. It can extract features from objects and their corresponding optical flow fields. Then these two features are fused to form new spatio-temporal features. Finally, the tensor-compressed model processes the fused features to determine fall detection, where the strongest optical field would indicate the fall. We conduct experiments with Multicam and URFD datasets.Clinical relevance— It demonstrates that the proposed model achieves the accuracy of 96.23% and 99.37%, respectively. Besides, it attains the inference speed of 83.3 FPS and storage reduction of 210.9×. Our work is further implemented on an AI acceleration core based edge device, and the runtime is reduced by 9.21×.This high performance system can be applied to the field of clinical monitoring in the future.

[1]  Nabil Zerrouki,et al.  Combined curvelets and hidden Markov models for human fall detection , 2018, Multimedia Tools and Applications.

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Jinbo Song,et al.  Visual guided deep learning scheme for fall detection , 2017, 2017 13th IEEE Conference on Automation Science and Engineering (CASE).

[4]  Matjaz Gams,et al.  Detecting Falls with Location Sensors and Accelerometers , 2011, IAAI.

[5]  M. Hawley,et al.  Automatic fall detectors and the fear of falling , 2004, Journal of telemedicine and telecare.

[6]  Wen-Hsien Fang,et al.  A hybrid human fall detection scheme , 2010, 2010 IEEE International Conference on Image Processing.

[7]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[8]  P. Phukpattaranont,et al.  Evaluation of fall detection for the elderly on a variety of subject groups , 2009, i-CREATe.

[9]  Cordelia Schmid,et al.  Action Recognition with Improved Trajectories , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  Suyuan Li,et al.  Vision-Based Fall Detection With Multi-Task Hourglass Convolutional Auto-Encoder , 2020, IEEE Access.

[11]  Haibo Wang,et al.  Depth-Based Human Fall Detection via Shape Features and Improved Extreme Learning Machine , 2014, IEEE Journal of Biomedical and Health Informatics.

[12]  Pranav Gupta,et al.  A novel approach for fall detection in home environment , 2017, 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE).

[13]  Nicolas Thome,et al.  A Real-Time, Multiview Fall Detection System: A LHMM-Based Approach , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Alexander Novikov,et al.  Tensorizing Neural Networks , 2015, NIPS.

[15]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[16]  Richard P. Wildes,et al.  Spatiotemporal Residual Networks for Video Action Recognition , 2016, NIPS.

[17]  Hiram Ponce,et al.  Application of Convolutional Neural Networks for Fall Detection Using Multiple Cameras , 2020 .

[18]  Elisson Rocha,et al.  Accelerometer-Based Human Fall Detection Using Convolutional Neural Networks , 2019, Sensors.

[19]  Shuicheng Yan,et al.  Multi-Fiber Networks for Video Recognition , 2018, ECCV.

[20]  Jean Meunier,et al.  Robust Video Surveillance for Fall Detection Based on Human Shape Deformation , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Nabil Zerrouki,et al.  Vision-based fall detection system for improving safety of elderly people , 2017, IEEE Instrumentation & Measurement Magazine.

[22]  Arif Mahmood,et al.  Using Temporal Covariance of Motion and Geometric Features via Boosting for Human Fall Detection , 2018, Sensors.