Attention-Based Dual-Stream Vision Transformer for Radar Gait Recognition

Radar gait recognition is robust to light variations and less infringement on privacy. Previous studies often utilize either spectrograms or cadence velocity diagrams. While the former shows the time-frequency patterns, the latter encodes the repetitive frequency patterns. In this work, a dual-stream neural network with attention-based fusion is proposed to fully aggregate the discriminant information from these two representations. The both streams are designed based on the Vision Transformer, which well captures the gait characteristics embedded in these representations. The proposed method is validated on a large benchmark dataset for radar gait recognition, which shows that it significantly outperforms state-of-the-art solutions.

[1]  M. Bernardi,et al.  Temporal Convolutional Neural Networks for Radar Micro-Doppler Based Gait Recognition † , 2021, Sensors.

[2]  Francesco Fioranelli,et al.  Centroid features for classification of armed/unarmed multiple personnel using multistatic human micro-Doppler , 2016 .

[3]  Marimuthu Palaniswami,et al.  Support vector machines for automated gait classification , 2005, IEEE Transactions on Biomedical Engineering.

[4]  Abdesselam Bouzerdoum,et al.  Human Motion Classification with Micro-Doppler Radar and Bayesian-Optimized Convolutional Neural Networks , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  Ming Ye,et al.  Radar‐ID: human identification based on radar micro‐Doppler signatures using deep convolutional neural networks , 2018, IET Radar, Sonar & Navigation.

[6]  Mu Zhou,et al.  Latern: Dynamic Continuous Hand Gesture Recognition Using FMCW Radar Sensor , 2018, IEEE Sensors Journal.

[7]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[8]  Xudong Jiang,et al.  Regularized 2-D complex-log spectral analysis and subspace reliability analysis of micro-Doppler signature for UAV detection , 2017, Pattern Recognit..

[9]  Georg Heigold,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2021, ICLR.

[10]  Shiqi Yu,et al.  A model-based gait recognition method with body pose and human prior knowledge , 2020, Pattern Recognit..

[11]  Youngwook Kim,et al.  Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[12]  Xudong Jiang,et al.  A three-step classification framework to handle complex data distribution for radar UAV detection , 2021, Pattern Recognit..

[13]  S. Bjorklund,et al.  Evaluation of a micro-Doppler classification method on mm-wave data , 2012, 2012 IEEE Radar Conference.

[14]  G. P. Cabic,et al.  Radar micro-Doppler feature extraction using the spectrogram and the cepstrogram , 2014, 2014 11th European Radar Conference.

[15]  Wu Liu,et al.  Siamese neural network based gait recognition for human identification , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Yaowu Chen,et al.  Attention-Based Two-Stream Convolutional Networks for Face Spoofing Detection , 2020, IEEE Transactions on Information Forensics and Security.

[18]  Abdelhak M. Zoubir,et al.  New analysis of radar micro-Doppler gait signatures for rehabilitation and assisted living , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  W.L. van Rossum,et al.  Classification of human gaits using interrupted radar measurements , 2017, 2017 IEEE Radar Conference (RadarConf).