Investigation of Channel Selection for Gesture Classification for Prosthesis Control Using Force Myography: A Case Study

Background: Various human machine interfaces (HMIs) are used to control prostheses, such as robotic hands. One of the promising HMIs is Force Myography (FMG). Previous research has shown the potential for the use of high density FMG (HD-FMG) that can lead to higher accuracy of prosthesis control. Motivation: The more sensors used in an FMG controlled system, the more complicated and costlier the system becomes. This study proposes a design method that can produce powered prostheses with performance comparable to that of HD-FMG controlled systems using a fewer number of sensors. An HD-FMG apparatus would be used to collect information from the user only in the design phase. Channel selection would then be applied to the collected data to determine the number and location of sensors that are vital to performance of the device. This study assessed the use of multiple channel selection (CS) methods for this purpose. Methods: In this case study, three datasets were used. These datasets were collected from force sensitive resistors embedded in the inner socket of a subject with transradial amputation. Sensor data were collected as the subject carried out five repetitions of six gestures. Collected data were then used to asses five CS methods: Sequential forward selection (SFS) with two different stopping criteria, minimum redundancy-maximum relevance (mRMR), genetic algorithm (GA), and Boruta. Results: Three out of the five methods (mRMR, GA, and Boruta) were able to decrease channel numbers significantly while maintaining classification accuracy in all datasets. Neither of them outperformed the other two in all datasets. However, GA resulted in the smallest channel subset in all three of the datasets. The three selected methods were also compared in terms of stability [i.e., consistency of the channel subset chosen by the method as new training data were introduced or some training data were removed (Chandrashekar and Sahin, 2014)]. Boruta and mRMR resulted in less instability compared to GA when applied to the datasets of this study. Conclusion: This study shows feasibility of using the proposed design method that can produce prosthetic systems that are simpler than HD-FMG systems but have performance comparable to theirs.

[1]  J. Kautz,et al.  Hand Gesture Recognition with 3 D Convolutional Neural Networks , 2015 .

[2]  G. Naik,et al.  Transradial Amputee Gesture Classification Using an Optimal Number of sEMG Sensors: An Approach Using ICA Clustering , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  Oluwarotimi Williams Samuel,et al.  A motion-classification strategy based on sEMG-EEG signal combination for upper-limb amputees , 2017, Journal of NeuroEngineering and Rehabilitation.

[4]  V. K. Jayaraman,et al.  An SVM classifier incorporating simultaneous noise reduction and feature selection: illustrative case examples , 2005, Pattern Recognit..

[5]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[6]  Stefan Wermter,et al.  Gesture Recognition with a Convolutional Long Short-Term Memory Recurrent Neural Network , 2016, ESANN.

[7]  Carlo Menon,et al.  A Case Study of a Force-myography Controlled Bionic Hand Mitigating Limb Position Effect , 2017 .

[8]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[9]  Sergio Escalera,et al.  A Survey on Deep Learning Based Approaches for Action and Gesture Recognition in Image Sequences , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[10]  Erik J Scheme,et al.  A proportional control scheme for high density force myography , 2018, Journal of neural engineering.

[11]  Richard Bowden,et al.  Sign Language Recognition , 2011, Visual Analysis of Humans.

[12]  Oliver Chiu-sing Choy,et al.  Integrating channel selection and feature selection in a real time epileptic seizure detection system , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[13]  Nitish V. Thakor,et al.  Stable force-myographic control of a prosthetic hand using incremental learning , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[14]  Carlo Menon,et al.  Exploration of Force Myography and surface Electromyography in hand gesture classification. , 2017, Medical engineering & physics.

[15]  Zenglin Xu,et al.  Discriminative Semi-Supervised Feature Selection Via Manifold Regularization , 2009, IEEE Transactions on Neural Networks.

[16]  C. Menon,et al.  Counting Grasping Action Using Force Myography: An Exploratory Study With Healthy Individuals , 2017, JMIR rehabilitation and assistive technologies.

[17]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Claudio Castellini,et al.  Using a high spatial resolution tactile sensor for intention detection , 2013, 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR).

[20]  Erik Scheme,et al.  High-density force myography: A possible alternative for upper-limb prosthetic control. , 2016, Journal of rehabilitation research and development.

[21]  Constantinos Gavriel,et al.  Dynamic forward prediction for prosthetic hand control by integration of EMG, MMG and kinematic signals , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[22]  Huiqing Liu,et al.  A comparative study on feature selection and classification methods using gene expression profiles and proteomic patterns. , 2002, Genome informatics. International Conference on Genome Informatics.

[23]  Alexey P. Porfirev,et al.  Polarization conversion when focusing cylindrically polarized vortex beams , 2016, Scientific Reports.

[24]  Weidong Geng,et al.  Gesture recognition by instantaneous surface EMG images , 2016, Scientific Reports.

[25]  David Zhang,et al.  Feature selection and analysis on correlated gas sensor data with recursive feature elimination , 2015 .

[26]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Héctor M. Pérez Meana,et al.  Cross View Gait Recognition Using Joint-Direct Linear Discriminant Analysis , 2017, Sensors.

[28]  Claudio Castellini,et al.  A wearable low-cost device based upon Force-Sensing Resistors to detect single-finger forces , 2014, 5th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics.

[29]  E. Biddiss,et al.  Upper limb prosthesis use and abandonment: A survey of the last 25 years , 2007, Prosthetics and orthotics international.

[30]  P. Gong,et al.  Object-based Detailed Vegetation Classification with Airborne High Spatial Resolution Remote Sensing Imagery , 2006 .

[31]  Steven E. Franklin,et al.  Multi-scale object-based image analysis and feature selection of multi-sensor earth observation imagery using random forests , 2012 .

[32]  Hideki Nakayama,et al.  Multimodal Gesture Recognition Using Multi-stream Recurrent Neural Network , 2015, PSIVT.

[33]  James Yang,et al.  Design, control, and sensory feedback of externally powered hand prostheses: a literature review. , 2013, Critical reviews in biomedical engineering.

[34]  Petros Maragos,et al.  Multimodal gesture recognition , 2017, The Handbook of Multimodal-Multisensor Interfaces, Volume 1.

[35]  Hongxiang Wang,et al.  Online convolution network tracking via spatio-temporal context , 2017, Multimedia Tools and Applications.

[36]  Jerald Siby,et al.  Hand Gesture Recognition , 2022, SSRN Electronic Journal.

[37]  Tanja Schultz,et al.  Recognizing Hand and Finger Gestures with IMU based Motion and EMG based Muscle Activity Sensing , 2015, BIOSIGNALS.

[38]  Giulio Sandini,et al.  Fine detection of grasp force and posture by amputees via surface electromyography , 2009, Journal of Physiology-Paris.

[39]  Carlo Menon,et al.  Force Myography to Control Robotic Upper Extremity Prostheses: A Feasibility Study , 2016, Front. Bioeng. Biotechnol..

[40]  Carlo Menon,et al.  Performance of forearm FMG and sEMG for estimating elbow, forearm and wrist positions , 2017 .

[41]  Stefan Wermter,et al.  An analysis of Convolutional Long Short-Term Memory Recurrent Neural Networks for gesture recognition , 2017, Neurocomputing.

[42]  Carlo Menon,et al.  Toward Intuitive Prosthetic Control: Solving Common Issues Using Force Myography, Surface Electromyography, and Pattern Recognition in a Pilot Case Study , 2017, IEEE Robotics & Automation Magazine.

[43]  James W. Davis,et al.  GESTURE RECOGNITION , 2023, International Research Journal of Modernization in Engineering Technology and Science.

[44]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[45]  Emery N. Brown,et al.  Human Leg Model Predicts Ankle Muscle-Tendon Morphology, State, Roles and Energetics in Walking , 2011, PLoS Comput. Biol..

[46]  J. Gerring.  A case study , 2011, Technology and Society.

[47]  John-John Cabibihan,et al.  Towards enhanced control of upper prosthetic limbs: A force-myographic approach , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[48]  Giles M. Foody,et al.  Feature Selection for Classification of Hyperspectral Data by SVM , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[49]  Stefano Stramigioli,et al.  Myoelectric forearm prostheses: state of the art from a user-centered perspective. , 2011, Journal of rehabilitation research and development.

[50]  Kristof Van Laerhoven,et al.  Feature Selection for Wireless Sensor Networks , 2004 .

[51]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[52]  Jian Weng,et al.  Feature selection for text classification: A review , 2018, Multimedia Tools and Applications.

[53]  Erik Scheme,et al.  EMG Pattern Recognition in the Era of Big Data and Deep Learning , 2018, Big Data Cogn. Comput..

[54]  Yu Hu,et al.  Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation , 2017, Sensors.

[55]  Chris H. Q. Ding,et al.  Minimum redundancy feature selection from microarray gene expression data , 2003, Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003.

[56]  Yinfeng Fang,et al.  Facilitate sEMG-Based Human-Machine Interaction Through Channel Optimization , 2019, Int. J. Humanoid Robotics.