An enhanced multi-view human action recognition system for virtual training simulator

Virtual military training systems have received considerable attention as a possible substitute for conventional real military training. In our previous work, human action recognition system using multiple Kinects (HARS-MK) has been implemented as a prototype of virtual military training simulator. However, the classification accuracy of HARS-MK is not enough to be utilized for virtual military training simulator. In addition, the experiments are carried out under just two simple action types; walking and crouching walking. In order to overcome these limitations, in this paper, we propose an enhanced multi-view human action recognition system (EM-HARS). Compared to HARS-MK, in EM-HARS, feature extractor is enhanced by employing covariance descriptor. In addition, the feasibility test of EM-HARS is conducted under various human actions including military training actions which are newly captured. The experiment results show that EM-HARS achieves higher classification accuracy than that of HARS-MK.

[1]  Dara Meldrum,et al.  Virtual reality rehabilitation of balance: assessment of the usability of the Nintendo Wii® Fit Plus , 2012, Disability and rehabilitation. Assistive technology.

[2]  Ramakant Nevatia,et al.  Single View Human Action Recognition using Key Pose Matching and Viterbi Path Searching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Alan C. Bovik,et al.  3D Visual Activity Assessment Based on Natural Scene Statistics , 2014, IEEE Transactions on Image Processing.

[4]  Junghwan Kim,et al.  Implementation of Human Action Recognition System Using Multiple Kinect Sensors , 2015, PCM.

[5]  Zahira Merchant,et al.  Effectiveness of virtual reality-based instruction on students' learning outcomes in K-12 and higher education: A meta-analysis , 2014, Comput. Educ..

[6]  Kwanghyun Lee,et al.  3D Perception Based Quality Pooling: Stereopsis, Binocular Rivalry, and Binocular Suppression , 2015, IEEE Journal of Selected Topics in Signal Processing.

[7]  Yu-Ting Su,et al.  Single/multi-view human action recognition via regularized multi-task learning , 2015, Neurocomputing.

[8]  Alan C. Bovik,et al.  3D Visual Discomfort Prediction: Vergence, Foveation, and the Physiological Optics of Accommodation , 2014, IEEE Journal of Selected Topics in Signal Processing.

[9]  Jonathan W. Decker,et al.  Performance measurements for the Microsoft Kinect skeleton , 2012, 2012 IEEE Virtual Reality Workshops (VRW).

[10]  Junghwan Kim,et al.  Implementation of an Omnidirectional Human Motion Capture System Using Multiple Kinect Sensors , 2015, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[11]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Alan C. Bovik,et al.  Stereoscopic 3D Visual Discomfort Prediction: A Dynamic Accommodation and Vergence Interaction Model , 2016, IEEE Transactions on Image Processing.

[13]  Ajey Lele,et al.  Virtual reality and its military utility , 2011, Journal of Ambient Intelligence and Humanized Computing.

[14]  Qi Tian,et al.  Human Daily Action Analysis with Multi-view and Color-Depth Data , 2012, ECCV Workshops.

[15]  Alan C. Bovik,et al.  Transfer Function Model of Physiological Mechanisms Underlying Temporal Visual Discomfort Experienced When Viewing Stereoscopic 3D Images , 2015, IEEE Transactions on Image Processing.

[16]  Alan C. Bovik,et al.  Multimodal Interactive Continuous Scoring of Subjective 3D Video Quality of Experience , 2014, IEEE Transactions on Multimedia.

[17]  Marwan Torki,et al.  Human Action Recognition Using a Temporal Hierarchy of Covariance Descriptors on 3D Joint Locations , 2013, IJCAI.

[18]  Petros Daras,et al.  Real-Time Skeleton-Tracking-Based Human Action Recognition Using Kinect Data , 2014, MMM.

[19]  Bingbing Ni,et al.  RGBD-HuDaAct: A color-depth video database for human daily activity recognition , 2011, ICCV Workshops.

[20]  Bruce W Knerr.  Immersive Simulation Training for the Dismounted Soldier , 2007 .

[21]  Alan C. Bovik,et al.  Saliency Prediction on Stereoscopic Videos , 2014, IEEE Transactions on Image Processing.

[22]  Lan Li,et al.  Human Action Recognition Using Maximum Temporal Inter-Class Dissimilarity , 2014 .

[23]  Alan C. Bovik,et al.  3D Visual Discomfort Predictor: Analysis of Disparity and Neural Activity Statistics , 2015, IEEE Transactions on Image Processing.