Vision and Sensor-Based Human Activity Recognition

Human action or activity or behavior analysis, recognition, and understanding are very important research areas in the field of computer vison, internet of things (IoT) sensor-based analysis, human-computer interaction (HCI), affective computing, intelligent system, healthcare facilities, and so on. There is much importance in human action recognition. This chapter introduces the core aspects of human action or activity recognition (HAR). These are split into two different domains: computer vision-based action/activity recognition, action localization, etc.; and wearable IoT sensor-based HAR. Though cameras are also sensors that provide vision-based information, the author puts camera-based or vision-based methods in another category. The chapter introduces core challenges and strategies to move forward.

[1]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Tanushyam Chattopadhyay,et al.  Action recognition using joint coordinates of 3D skeleton data , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[3]  Naonori Ueda,et al.  Mobile Activity Recognition through Training Labels with Inaccurate Activity Segments , 2016, MobiQuitous.

[4]  Yuting Su,et al.  Multiple/Single-View Human Action Recognition via Part-Induced Multitask Structural Learning , 2015, IEEE Transactions on Cybernetics.

[5]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Marco Morana,et al.  Human Activity Recognition Process Using 3-D Posture Data , 2015, IEEE Transactions on Human-Machine Systems.

[7]  Ennio Gambi,et al.  A Human Activity Recognition System Using Skeleton Data from RGBD Sensors , 2016, Comput. Intell. Neurosci..

[8]  David Nistér,et al.  Preemptive RANSAC for live structure and motion estimation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[9]  Sungyoung Lee,et al.  Describing body-pose feature - poselet - activity relationship using Pachinko Allocation Model , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[10]  Zhi Liu,et al.  3D-based Deep Convolutional Neural Network for action recognition with depth sequences , 2016, Image Vis. Comput..

[11]  John Sell,et al.  The Xbox One System on a Chip and Kinect Sensor , 2014, IEEE Micro.

[12]  Radha Poovendran,et al.  Human activity recognition for video surveillance , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[13]  Mubarak Shah,et al.  UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild , 2012, ArXiv.

[14]  Rama Chellappa,et al.  Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Hakil Kim,et al.  Real-Time Action Detection in Video Surveillance using Sub-Action Descriptor with Multi-CNN , 2017, ArXiv.

[16]  Xiaodong Yang,et al.  Super Normal Vector for Activity Recognition Using Depth Sequences , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Chen Li,et al.  3D Human Activity Recognition Using Skeletal Data from RGBD Sensors , 2016, ISVC.

[18]  Dario Maio,et al.  Joint Orientations from Skeleton Data for Human Activity Recognition , 2017, ICIAP.

[19]  Guillermo Sapiro,et al.  Sparse Modeling of Human Actions from Motion Imagery , 2012, International Journal of Computer Vision.

[20]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Yi Zhang,et al.  Mining Key Skeleton Poses with Latent SVM for Action Recognition , 2017, Appl. Comput. Intell. Soft Comput..

[22]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[23]  Ivan Laptev,et al.  On Space-Time Interest Points , 2005, International Journal of Computer Vision.

[24]  Alberto Del Bimbo,et al.  Submitted to Ieee Transactions on Cybernetics 1 3d Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold , 2022 .

[25]  Subhasis Bhaumik,et al.  Modeling and Control of Electric Linear Actuator for Driving Knee Joint of a Lower Body Exoskeleton , 2020 .

[26]  Abdelhamid Bouchachia,et al.  A review of smart homes in healthcare , 2015, J. Ambient Intell. Humaniz. Comput..

[27]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Md. Atiqur Rahman Ahad,et al.  Motion history image: its variants and applications , 2012, Machine Vision and Applications.

[29]  Yun Fu,et al.  Bilinear heterogeneous information machine for RGB-D action recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  D. Brann,et al.  Neurotrophic and neuroprotective actions of estrogen: Basic mechanisms and clinical implications , 2007, Steroids.

[31]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[32]  Carl E. Rasmussen,et al.  Learning Depth from Stereo , 2004, DAGM-Symposium.

[33]  Zhengming Ding,et al.  Latent Tensor Transfer Learning for RGB-D Action Recognition , 2014, ACM Multimedia.

[34]  Mohamed E. Hussein,et al.  CovP3DJ: Skeleton-parts-based-covariance Descriptor for Human Action Recognition , 2018, VISIGRAPP.

[35]  Yi Zhang,et al.  Improved Key Poses Model for Skeleton-Based Action Recognition , 2017, PCM.

[36]  Claudio Fanti.  Towards Automatic Discovery of Human Movemes , 2008 .

[37]  Wei Liu,et al.  Latent Max-Margin Multitask Learning With Skelets for 3-D Action Recognition , 2017, IEEE Transactions on Cybernetics.

[38]  J. Weickert,et al.  Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods , 2005 .

[39]  Xiaodong Yang,et al.  Recognizing actions using depth motion maps-based histograms of oriented gradients , 2012, ACM Multimedia.

[40]  Alberto Del Bimbo,et al.  Recognizing Actions from Depth Cameras as Weakly Aligned Multi-part Bag-of-Poses , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.