Choreographic Pose Identification using Convolutional Neural Networks

In this paper we present a deep learning scheme for classification of dance postures using Kinect II RGB data and Convolutional Neural Networks (CNN). This is achieved through the analysis of a data-set that includes three traditional Greek dances, where each dance was performed by 3 different dancers. The obtained data were processed and analyzed using a deep convolutional neural network, in order to identify the primitive postures that comprise the choreography. To enhance the classification performance, a background subtraction framework was utilized, while the CNN architecture was adapted to simulate a moving average behavior. The overall system can be used as an AI module for assessing the performance of users in a serious game for learning traditional dance choreographies

[1]  Bin Sheng,et al.  Deep Convolutional Neural Networks for Human Action Recognition Using Depth Maps and Postures , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[2]  Feng-ping He.  Research on Protection and Inheritance of Intangible Cultural Heritage in Dazhou –Taking Fanshanjiaozi as an Example , 2019, Proceedings of the 2018 6th International Education, Economics, Social Science, Arts, Sports and Management Engineering Conference (IEESASM 2018).

[3]  Javed Imran,et al.  Combining CNN streams of RGB-D and skeletal data for human activity recognition , 2018, Pattern Recognit. Lett..

[4]  J. Blake.  From Traditional Culture and Folklore to Intangible Cultural Heritage: Evolution of a Treaty , 2018 .

[5]  Wei Liu,et al.  Deep Background Subtraction with Guided Learning , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[6]  Qingquan Song,et al.  Auto-Keras: Efficient Neural Architecture Search with Network Morphism , 2018 .

[7]  Nikolaos Doulamis,et al.  Spatio-temporal summarization of dance choreographies , 2018, Comput. Graph..

[8]  Gerhard Rigoll,et al.  A deep convolutional neural network for video sequence background subtraction , 2018, Pattern Recognit..

[9]  Nikolaos Doulamis,et al.  Deep Learning for Computer Vision: A Brief Review , 2018, Comput. Intell. Neurosci..

[10]  Björn Ommer,et al.  Unsupervised Video Understanding by Reconciliation of Posture Similarities , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[11]  Lianzheng Ge,et al.  Three-stream CNNs for action recognition , 2017, Pattern Recognit. Lett..

[12]  Deba Prasad Dash,et al.  Hidden Markov Model based human activity recognition using shape and optical flow based features , 2016, 2016 IEEE Region 10 Conference (TENCON).

[13]  Yuanliu Liu,et al.  Video-based emotion recognition using CNN-RNN and C3D hybrid networks , 2016, ICMI.

[14]  Nikolaos Doulamis,et al.  Deep learning based human behavior recognition in industrial workflows , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[15]  Huiyu Zhou,et al.  Region-based Mixture of Gaussians modelling for foreground detection in dynamic scenes , 2015, Pattern Recognit..

[16]  Cordelia Schmid,et al.  P-CNN: Pose-Based CNN Features for Action Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Yi Yang,et al.  DevNet: A Deep Event Network for multimedia event detection and evidence recounting , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[19]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[20]  Nikolaos D. Doulamis,et al.  Vision-based maritime surveillance system using fused visual attention maps and online adaptable tracker , 2013, 2013 14th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS).

[21]  Richard Bowden,et al.  Hollywood 3D: Recognizing Actions in 3D Natural Scenes , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Cordelia Schmid,et al.  Action recognition by dense trajectories , 2011, CVPR 2011.

[23]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[24]  R. Kurin.  Safeguarding Intangible Cultural Heritage in the 2003 UNESCO Convention: a critical appraisal , 2004 .

[25]  Shyamsundar Rajaram,et al.  Human Activity Recognition Using Multidimensional Indexing , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.