Application of wavelet scattering networks in classification of ultrasound image sequences

Recently, ultrasound imaging of muscle contractions has been used by several research groups to infer volitional motor intent of the user, and has shown promise as a novel muscle computer interface. Learning spatiotemporal features from ultrasound image sequences is challenging because of deformations introduced by probe repositioning. The image features are sensitive to probe placement and even small displacements during cross-session donning and doffing of the probe could compromise the classification accuracy when using a model trained on a previous session. This requires frequent recalibration. Deep learning based feature extractors have been shown to be invariant to translation, rotation and slight deformations. In this study, we investigated the feasibility of wavelet-based deep scattering features to preserve motion classification accuracy across multiple donning and doffing sessions. It was demonstrated that scattering features, which do not require learning filter weights, can significantly improve the cross-session classification accuracy by 20–30%. It can be concluded that these features are robust to minor probe displacements. They need to be further investigated with a larger data set to investigate their robustness in accurately classifying different muscle movements.

[1]  Chu-Song Chen,et al.  A Learning Framework for Age Rank Estimation Based on Face Images With Scattering Transform , 2015, IEEE Transactions on Image Processing.

[2]  Xinjun Sheng,et al.  Toward an Enhanced Human–Machine Interface for Upper-Limb Prosthesis Control With Combined EMG and NIRS Signals , 2017, IEEE Transactions on Human-Machine Systems.

[3]  Mike Fraser,et al.  EchoFlex: Hand Gesture Recognition using Ultrasound Imaging , 2017, CHI.

[4]  Patrick van der Smagt,et al.  Surface EMG in advanced hand prosthetics , 2008, Biological Cybernetics.

[5]  Stéphane Mallat,et al.  Invariant Scattering Convolution Networks , 2012, IEEE transactions on pattern analysis and machine intelligence.

[6]  Stéphane Mallat,et al.  Classification with scattering operators , 2010, CVPR 2011.

[7]  Michael Egmont-Petersen,et al.  Image processing with neural networks - a review , 2002, Pattern Recognit..

[8]  C. Castellini,et al.  Using Ultrasound Images of the Forearm to Predict Finger Positions , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Desney S. Tan,et al.  Enabling always-available input with muscle-computer interfaces , 2009, UIST '09.

[10]  Sajid Saleem,et al.  Mallat’s Scattering Transform Based Anomaly Sensing for Detection of Seizures in Scalp EEG , 2017, IEEE Access.

[11]  Zhi-Hong Mao,et al.  Limitations of Surface EMG Signals of Extrinsic Muscles in Predicting Postures of Human Hand , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Jana Kosecka,et al.  Real-Time Classification of Hand Motions Using Ultrasound Imaging of Forearm Muscles , 2016, IEEE Transactions on Biomedical Engineering.

[13]  Huzefa Rangwala,et al.  Novel Method for Predicting Dexterous Individual Finger Movements by Imaging Muscle Activity Using a Wearable Ultrasonic System , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.