A Framework for Activity Recognition Through Deep Learning and Abnormality Detection in Daily Activities

Activity recognition plays a key role in providing activity assistance and care for users in intelligent homes. This paper presents a two layer of convolutional neural networks to perform human action recognition using images provided by multiple cameras. We consider one PTZ camera and multiple Kinects in order to offer continuity over the users movement. The drawbacks of using only one type of sensor is minimized. For example, field of view provided by Kinect sensor is not wide enough to cover the entire room. Also, the PTZ camera is not able to detect and track a person in case of different situations, such as the person is sitting or it is under the camera. Also the system will identify abnormalities that can appear in sequences of performed daily activities. The system is tested in Ambient Intelligence Laboratory (AmI-Lab) at the University Politehnica of Bucharest.

[1]  Larry S. Davis,et al.  A unified tree-based framework for joint action localization, recognition and segmentation , 2013, Comput. Vis. Image Underst..

[2]  Miguel A. Labrador,et al.  A mobile platform for real-time human activity recognition , 2012, 2012 IEEE Consumer Communications and Networking Conference (CCNC).

[3]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[4]  Diane J. Cook,et al.  Detecting Abnormality in Activities Performed by People with Dementia in a Smart Environment , 2014 .

[5]  Weihua Sheng,et al.  Multi-sensor fusion for human daily activity recognition in robot-assisted living , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[6]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[7]  Ying Wu,et al.  Mining actionlet ensemble for action recognition with depth cameras , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Lin Sun,et al.  DL-SFA: Deeply-Learned Slow Feature Analysis for Action Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Thomas Serre,et al.  HMDB: A large video database for human motion recognition , 2011, 2011 International Conference on Computer Vision.

[12]  Fei-Fei Li,et al.  Learning latent temporal structure for complex event detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Hakil Kim,et al.  Real-Time Human Action Recognition Using CNN Over Temporal Images for Static Video Surveillance Cameras , 2015, PCM.

[14]  Mubarak Shah,et al.  UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild , 2012, ArXiv.

[15]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[16]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  R. Dobrescu,et al.  Smart sensor network for continuous monitoring at home of elderly population with chronic diseases , 2012, 2012 20th Telecommunications Forum (TELFOR).

[18]  Young-Koo Lee,et al.  Semi-Markov conditional random fields for accelerometer-based activity recognition , 2010, Applied Intelligence.

[19]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[20]  Adina Magda Florea,et al.  Multimodal Indoor Tracking of a Single Elder in an AAL Environment , 2013, ISAmI.

[21]  Yi Zeng,et al.  HCNN: A Neural Network Model for Combining Local and Global Features Towards Human-Like Classification , 2016, Int. J. Pattern Recognit. Artif. Intell..

[22]  Irina Mocanu,et al.  Human tracking using multiple views , 2017, 2017 40th International Conference on Telecommunications and Signal Processing (TSP).

[23]  Cordelia Schmid,et al.  Action Recognition with Improved Trajectories , 2013, 2013 IEEE International Conference on Computer Vision.