Arm Motion Classification Using Time-Series Analysis of the Spectrogram Frequency Envelopes

Hand and arm gesture recognition using radio frequency (RF) sensing modality proves valuable in man–machine interfaces and smart environments. In this paper, we use the time-series analysis method to accurately measure the similarity of the micro-Doppler (MD) signatures between the training and test data, thus providing improved gesture classification. We characterize the MD signatures by the maximum instantaneous Doppler frequencies depicted in the spectrograms. In particular, we apply two machine learning (ML) techniques, namely, the dynamic time warping (DTW) method and the long short-term memory (LSTM) network. Both methods take into account the values as well as the temporal evolution and characteristics of the time-series data. It is shown that the DTW method achieves high gesture classification rates and is robust to time misalignment.

[1]  Bijan G. Mobasseri,et al.  A time-frequency classifier for human gait recognition , 2009, Defense + Commercial Sensing.

[2]  Moeness G. Amin,et al.  Radar-Based Human-Motion Recognition With Deep Learning: Promising applications for indoor monitoring , 2019, IEEE Signal Processing Magazine.

[3]  Attaphongse Taparugssanagorn,et al.  Human activity classification using long short-term memory network , 2018, Signal, Image and Video Processing.

[4]  L. Cifola,et al.  Multi-target human gait classification using LSTM recurrent neural networks applied to micro-Doppler , 2017, 2017 European Radar Conference (EURAD).

[5]  Frans C. A. Groen,et al.  Feature-based human motion parameter estimation with radar , 2008 .

[6]  Moeness Amin,et al.  Radar for Indoor Monitoring: Detection, Classification, and Assessment , 2017 .

[7]  Zhengxin Zeng,et al.  Automatic Arm Motion Recognition Using Radar for Smart Home Technologies , 2019, 2019 International Radar Conference (RADAR).

[8]  Youngwook Kim,et al.  Human Detection Using Doppler Radar Based on Physical Characteristics of Targets , 2015, IEEE Geoscience and Remote Sensing Letters.

[9]  Tanya Makkar,et al.  Analogizing time complexity of KNN and CNN in recognizing handwritten digits , 2017, 2017 Fourth International Conference on Image Information Processing (ICIIP).

[10]  Helmut Alt,et al.  Computing the Fréchet distance between two polygonal curves , 1995, Int. J. Comput. Geom. Appl..

[11]  Moeness G. Amin,et al.  Analysis of micro-Doppler signals using linear FM basis decomposition , 2006, SPIE Defense + Commercial Sensing.

[12]  Moeness G. Amin,et al.  RF Sensing for Continuous Monitoring of Human Activities for Home Consumer Applications , 2020, ArXiv.

[13]  Abdelhak M. Zoubir,et al.  Subspace Classification of Human Gait Using Radar Micro-Doppler Signatures , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[14]  Junyou Shi,et al.  Complex Rotation Quantum Dynamic Neural Networks (CRQDNN) using Complex Quantum Neuron (CQN): Applications to time series prediction , 2015, Neural Networks.

[15]  Vit Niennattrakul,et al.  On Clustering Multimedia Time Series Data Using K-Means and Dynamic Time Warping , 2007, 2007 International Conference on Multimedia and Ubiquitous Engineering (MUE'07).

[16]  Wenyi Zeng,et al.  Clustering Time Series with Granular Dynamic Time Warping Method , 2007, 2007 IEEE International Conference on Granular Computing (GRC 2007).

[17]  Andrew W. Senior,et al.  Long short-term memory recurrent neural network architectures for large scale acoustic modeling , 2014, INTERSPEECH.

[18]  CuiYiqian,et al.  Complex Rotation Quantum Dynamic Neural Networks (CRQDNN) using Complex Quantum Neuron (CQN) , 2015 .

[19]  M. Amin.  Time-Frequency Spectrum Analysis and Estimation for Nonstationary Random-Processes , 1992 .

[20]  Michael Inggs,et al.  A CNN and LSTM-based approach to classifying transient radio frequency interference , 2018, Astron. Comput..

[21]  Boualem Boashash,et al.  Range-Doppler radar sensor fusion for fall detection , 2017, 2017 IEEE Radar Conference (RadarConf).

[22]  Yimin Zhang,et al.  Radar Signal Processing for Elderly Fall Detection: The future for in-home monitoring , 2016, IEEE Signal Processing Magazine.

[23]  Pietro Perona,et al.  Continuous dynamic time warping for translation-invariant curve alignment with applications to signature verification , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[24]  Branka Jokanovic,et al.  Radar fall motion detection using deep learning , 2016, 2016 IEEE Radar Conference (RadarConf).

[25]  Kevin Buchin,et al.  Computing the Fréchet distance between simple polygons in polynomial time , 2006, SCG '06.

[26]  Robin J. Evans,et al.  Hand-Gesture Recognition Using Two-Antenna Doppler Radar With Deep Convolutional Neural Networks , 2019, IEEE Sensors Journal.

[27]  Gang Li,et al.  Sparsity-Driven Micro-Doppler Feature Extraction for Dynamic Hand Gesture Recognition , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[28]  Zhengxin Zeng,et al.  Automatic Arm Motion Recognition Based on Radar Micro-Doppler Signature Envelopes , 2019, IEEE Sensors Journal.

[29]  Thomas Philip Runarsson,et al.  Support vector machines and dynamic time warping for time series , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[30]  Youngwook Kim,et al.  Hand Gesture Recognition Using Micro-Doppler Signatures With Convolutional Neural Network , 2016, IEEE Access.

[31]  G. Priya,et al.  EFFICIENT KNN CLASSIFICATION ALGORITHM FOR BIG DATA , 2017 .

[32]  Zhengxin Zeng,et al.  Hand Gesture Recognition based on Radar Micro-Doppler Signature Envelopes , 2018, 2019 IEEE Radar Conference (RadarConf).

[33]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[34]  Mohammed Waleed Kadous,et al.  Temporal classification: extending the classification paradigm to multivariate time series , 2002 .

[35]  Christos Faloutsos,et al.  FTW: fast similarity search under the time warping distance , 2005, PODS.

[36]  Yee Hui Lee,et al.  Long Short-Term Memory for Radio Frequency Spectral Prediction and its Real-Time FPGA Implementation , 2018, MILCOM 2018 - 2018 IEEE Military Communications Conference (MILCOM).

[37]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[38]  Carmine Clemente,et al.  Micro-doppler-based in-home aided and unaided walking recognition with multiple radar and sonar systems , 2017 .

[39]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[40]  Abdelhak M. Zoubir,et al.  Detection of Gait Asymmetry Using Indoor Doppler Radar , 2019, 2019 IEEE Radar Conference (RadarConf).

[41]  Gang Li,et al.  Dynamic hand gesture classification based on radar micro-Doppler signatures , 2016, 2016 CIE International Conference on Radar (RADAR).

[42]  Houshang Darabi,et al.  Efficient Classification of Long Time Series by 3-D Dynamic Time Warping , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[43]  Philip Chan,et al.  Toward accurate dynamic time warping in linear time and space , 2007, Intell. Data Anal..

[44]  Frank Nielsen,et al.  K-nearest neighbor search: Fast GPU-based implementations and application to high-dimensional feature matching , 2010, 2010 IEEE International Conference on Image Processing.

[45]  Moeness Amin,et al.  Fall Detection Using Deep Learning in Range-Doppler Radars , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[46]  Peter C. Y. Chen,et al.  LSTM network: a deep learning approach for short-term traffic forecast , 2017 .

[47]  Suresh Venkatasubramanian,et al.  Curve Matching, Time Warping, and Light Fields: New Algorithms for Computing Similarity between Curves , 2007, Journal of Mathematical Imaging and Vision.

[48]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[49]  Karl Woodbridge,et al.  A real-time high resolution passive WiFi Doppler-radar and its applications , 2014, 2014 International Radar Conference.

[50]  Ivan Poupyrev,et al.  Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum , 2016, UIST.

[51]  Antonello Rizzi,et al.  Cancer Diagnosis Using Deep Learning: A Bibliographic Review , 2019, Cancers.

[52]  Abdelhak M. Zoubir,et al.  Parameter Estimation for Locally Linear FM Signals Using a Time-Frequency Hough Transform , 2008, IEEE Transactions on Signal Processing.

[53]  Francesco Fioranelli,et al.  Activity Classification Using Raw Range and I & Q Radar Data with Long Short Term Memory Layers , 2018, 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech).

[54]  Alan Bundy,et al.  Dynamic Time Warping , 1984 .

[55]  Pavel Senin,et al.  Dynamic Time Warping Algorithm Review , 2008 .

[56]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.