CGA: a new feature selection model for visual human action recognition

Recognition of human actions from visual contents is a budding field of computer vision and image understanding. The problem with such a recognition system is the huge dimensions of the feature vectors. Many of these features are irrelevant to the classification mechanism. For this reason, in this paper, we propose a novel feature selection (FS) model called cooperative genetic algorithm (CGA) to select some of the most important and discriminating features from the entire feature set to improve the classification accuracy as well as the time requirement of the activity recognition mechanism. In CGA, we have made an effort to embed the concepts of cooperative game theory in GA to create a both-way reinforcement mechanism to improve the solution of the FS model. The proposed FS model is tested on four benchmark video datasets named Weizmann, KTH, UCF11, HMDB51, and two sensor-based UCI HAR datasets. The experiments are conducted using four state-of-the-art feature descriptors, namely HOG, GLCM, SURF, and GIST. It is found that there is a significant improvement in the overall classification accuracy while considering very small fraction of the original feature vector.

[1]  Limin Wang,et al.  Motionlets: Mid-level 3D Parts for Human Motion Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Limin Wang,et al.  Multi-view Super Vector for Action Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Ayoub Al-Hamadi,et al.  Affine-Invariant Feature Extraction for Activity Recognition , 2013 .

[4]  C. A. Murthy,et al.  Unsupervised Feature Selection Using Feature Similarity , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  A. G. Amitha Perera,et al.  Human Action Recognition in Large-Scale Datasets Using Histogram of Spatiotemporal Gradients , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[6]  J. K. Mandal,et al.  Activity recognition system using inbuilt sensors of smart mobile phone and minimizing feature vectors , 2016 .

[7]  Ram Sarkar,et al.  Hybrid of Harmony Search Algorithm and Ring Theory-Based Evolutionary Algorithm for Feature Selection , 2020, IEEE Access.

[8]  Ram Sarkar,et al.  Late Acceptance Hill Climbing Based Social Ski Driver Algorithm for Feature Selection , 2020, IEEE Access.

[9]  David Picard,et al.  Learning features combination for human action recognition from skeleton sequences , 2017, Pattern Recognit. Lett..

[10]  Hossein Nezamabadi-pour,et al.  BGSA: binary gravitational search algorithm , 2010, Natural Computing.

[11]  M. Student,et al.  Sobel Edge Detection Algorithm , 2013 .

[12]  Davide Anguita,et al.  A Public Domain Dataset for Human Activity Recognition using Smartphones , 2013, ESANN.

[13]  Thomas Serre,et al.  HMDB: A large video database for human motion recognition , 2011, 2011 International Conference on Computer Vision.

[14]  Ram Sarkar,et al.  Feature Selection for Facial Emotion Recognition Using Cosine Similarity-Based Harmony Search Algorithm , 2020, Applied Sciences.

[15]  Cordelia Schmid,et al.  Dense Trajectories and Motion Boundary Descriptors for Action Recognition , 2013, International Journal of Computer Vision.

[16]  Yupin Luo,et al.  Recognizing human actions using a new descriptor based on spatial-temporal interest points and weighted-output classifier , 2012, Neurocomputing.

[17]  Muhammad Younus Javed,et al.  A framework of human detection and action recognition based on uniform segmentation and combination of Euclidean distance and joint entropy-based features selection , 2017, EURASIP J. Image Video Process..

[18]  Morton D. Davis,et al.  The kernel of a cooperative game , 1965 .

[19]  Ram Sarkar,et al.  Introducing clustering based population in Binary Gravitational Search Algorithm for Feature Selection , 2020, Appl. Soft Comput..

[20]  Daijin Kim,et al.  Hidden Markov Model Ensemble for Activity Recognition Using Tri-Axis Accelerometer , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[21]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[22]  Niloy Sikder,et al.  Human Activity Recognition Using Multichannel Convolutional Neural Network , 2019, 2019 5th International Conference on Advances in Electrical Engineering (ICAEE).

[23]  Zong Woo Geem,et al.  Improved Binary Sailfish Optimizer Based on Adaptive β-Hill Climbing for Feature Selection , 2020, IEEE Access.

[24]  Rajeev Srivastava,et al.  Multi-view recognition system for human activity based on multiple features for video surveillance system , 2018, Multimedia Tools and Applications.

[25]  Jiebo Luo,et al.  Recognizing realistic actions from videos , 2009, CVPR.

[26]  Norwati Mustapha,et al.  Two-stage feature selection using ranking self-adaptive differential evolution algorithm for recognition of acceleration activity , 2018 .

[27]  Ronen Basri,et al.  Actions as space-time shapes , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[28]  Wenyu Zhang,et al.  Multivariate Time Series Classification Using Dynamic Time Warping Template Selection for Human Activity Recognition , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[29]  Kristen Grauman,et al.  Efficient Activity Detection in Untrimmed Video with Max-Subgraph Search , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Mita Nasipuri,et al.  M-HMOGA: A New Multi-Objective Feature Selection Algorithm for Handwritten Numeral Classification , 2019, J. Intell. Syst..

[31]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, ICPR 2004.

[32]  Wei Huang,et al.  Human action recognition based on Self Organizing Map , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[33]  Mubarak Shah,et al.  A 3-dimensional sift descriptor and its application to action recognition , 2007, ACM Multimedia.

[34]  Debotosh Bhattacharjee,et al.  A Comparative Study of Different Feature Descriptors for Video-Based Human Action Recognition , 2020 .

[35]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[36]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[37]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .

[38]  Jacek M. Zurada,et al.  Normalized Mutual Information Feature Selection , 2009, IEEE Transactions on Neural Networks.

[39]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[40]  Ram Sarkar,et al.  A wrapper-filter feature selection technique based on ant colony optimization , 2019, Neural Computing and Applications.

[41]  Ram Rup Sarkar,et al.  Binary Genetic Swarm Optimization: A Combination of GA and PSO for Feature Selection , 2019, J. Intell. Syst..

[42]  Akif Durdu,et al.  Human action recognition with bag of visual words using different machine learning methods and hyperparameter optimization , 2019, Neural Computing and Applications.

[43]  Ujjwal Maulik,et al.  Recursive Memetic Algorithm for gene selection in microarray data , 2019, Expert Syst. Appl..

[44]  Samit Ari,et al.  Fusion of histogram based features for Human Action Recognition , 2019, 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS).

[45]  Juan Carlos Niebles,et al.  A Hierarchical Model of Shape and Appearance for Human Action Classification , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Nicolai Petkov,et al.  Learning skeleton representations for human action recognition , 2019, Pattern Recognition Letters.

[47]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[48]  Wen Zhou,et al.  Human Action Recognition With Multiple-Instance Markov Model , 2014, IEEE Transactions on Information Forensics and Security.

[49]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[50]  Jack Sklansky,et al.  A note on genetic algorithms for large-scale feature selection , 1989, Pattern Recognition Letters.

[51]  Vikrant Bhateja,et al.  Deluge based Genetic Algorithm for feature selection , 2019, Evolutionary Intelligence.

[52]  Ram Sarkar,et al.  Binary Social Mimic Optimization Algorithm With X-Shaped Transfer Function for Feature Selection , 2020, IEEE Access.

[53]  Jiquan Ngiam,et al.  Sparse Filtering , 2011, NIPS.

[54]  D. Suryanarayana,et al.  A Comparative Study of Random Forest & K – Nearest Neighbors on HAR dataset Using Caret , 2017 .

[55]  Davide Anguita,et al.  Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine , 2012, IWAAL.

[56]  Thomas B. Moeslund,et al.  A selective spatio-temporal interest point detector for human action recognition in complex scenes , 2011, 2011 International Conference on Computer Vision.

[57]  Rajeev Srivastava,et al.  A Framework for Human Activity Recognition using Pose Feature for Video Surveillance System , 2016 .

[58]  Adam Lipowski,et al.  Roulette-wheel selection via stochastic acceptance , 2011, ArXiv.

[59]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[60]  M. Mukaka,et al.  Statistics corner: A guide to appropriate use of correlation coefficient in medical research. , 2012, Malawi medical journal : the journal of Medical Association of Malawi.

[61]  Haibin Ling,et al.  3D R Transform on Spatio-temporal Interest Points for Action Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[62]  Hossein Nezamabadi-pour,et al.  A quantum-inspired gravitational search algorithm for binary encoded optimization problems , 2015, Eng. Appl. Artif. Intell..

[63]  L. Lin,et al.  A concordance correlation coefficient to evaluate reproducibility. , 1989, Biometrics.

[64]  Mita Nasipuri,et al.  Feature Selection for Handwritten Word Recognition Using Memetic Algorithm , 2019 .

[65]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words , 2008, International Journal of Computer Vision.

[66]  L. Davis,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002, Proc. IEEE.

[67]  Mita Nasipuri,et al.  Feature Selection Using Histogram-Based Multi-objective GA for Handwritten Devanagari Numeral Recognition , 2018 .

[68]  Mubarak Shah,et al.  Classifying web videos using a global video descriptor , 2013, Machine Vision and Applications.

[69]  Khalil El-Khatib,et al.  A Comparative Analysis of the Impact of Features on Human Activity Recognition with Smartphone Sensors , 2017, WebMedia.

[70]  Nasser Yazdani,et al.  Mutual information-based feature selection for intrusion detection systems , 2011, J. Netw. Comput. Appl..

[71]  Mita Nasipuri,et al.  A GA based hierarchical feature selection approach for handwritten word recognition , 2019, Neural Computing and Applications.

[72]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[73]  Ram Sarkar,et al.  Genetic algorithm based cancerous gene identification from microarray data using ensemble of filter methods , 2018, Medical & Biological Engineering & Computing.

[74]  Subhadip Basu,et al.  Benchmark databases of handwritten Bangla-Roman and Devanagari-Roman mixed-script document images , 2018, Multimedia Tools and Applications.

[75]  David E. Goldberg,et al.  Genetic Algorithms, Tournament Selection, and the Effects of Noise , 1995, Complex Syst..

[76]  Gabriella Balestra,et al.  Comparison of Different Sets of Features for Human Activity Recognition by Wearable Sensors , 2018, Sensors.

[77]  Showmik Bhowmik,et al.  Mutually Informed Correlation Coefficient (MICC) - a New Filter Based Feature Selection Method , 2020, 2020 IEEE Calcutta Conference (CALCON).

[78]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[79]  Manuel López-Ibáñez,et al.  Ant colony optimization , 2010, GECCO '10.

[80]  Nanning Zheng,et al.  View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  Claudia Eckert,et al.  Neural Network-Based User-Independent Physical Activity Recognition for Mobile Devices , 2015, IDEAL.

[82]  Haibin Zhu,et al.  Fuzzy kNN Text Classifier Based on Gini Index , 2006 .

[83]  Vikrant Bhateja,et al.  A histogram based fuzzy ensemble technique for feature selection , 2019, Evolutionary Intelligence.

[84]  Ram Sarkar,et al.  Feature selection for facial emotion recognition using late hill-climbing based memetic algorithm , 2019, Multimedia Tools and Applications.

[85]  Fanglin Chen,et al.  Action recognition by hidden temporal models , 2013, The Visual Computer.

[86]  J. M. Bilbao,et al.  Cooperative Games on Combinatorial Structures , 2000 .

[87]  Nazli Ikizler-Cinbis,et al.  Object, Scene and Actions: Combining Multiple Features for Human Action Recognition , 2010, ECCV.

[88]  Laith Mohammad Abualigah,et al.  APPLYING GENETIC ALGORITHMS TO INFORMATION RETRIEVAL USING VECTOR SPACE MODEL , 2015 .

[89]  Samit Ari,et al.  On an algorithm for human action recognition , 2019, Expert Syst. Appl..