Multi-frequency and multi-domain human activity recognition based on SFCW radar using deep learning

Abstract Deep learning and radar make it feasible to automatically recognize human activities in various lighting conditions, even occlusion case, which significantly promotes the application of activity recognition in the fields of security surveillance, health care, and so on. In this paper, an approach for human activity recognition (HAR) using deep learning is proposed based on stepped frequency continues wave (SFCW) radar. Specifically, SFCW radar is utilized to generate two types of characteristic representation domains, namely multiple frequencies of spectrograms in time-frequency domain and range maps in range domain. On the one hand, spectrograms and range maps provide different types of features. On the other hand, multi-frequency spectrograms furnish same type of features while with different scattering properties and frequency resolutions. Then a specific deep learning network including multiple parallel deep convolutional neural networks (DCNNs) and a sparse autoencoder is designed to extract and fuse these features associated with human activities from the multi-frequency spectrograms and rang map. In particular, each DCNN is aimed at extracting the detailed micro-Doppler features from a spectrogram, while sparse autoencoder learns prime range distribution features by compressing each range map to reduce complexity and improve robustness. Experimental results verify that the proposed deep learning scheme achieves 96.42% recognition accuracy about six types of activities by incorporating three frequencies of spectrograms and range map, and surpasses two existed methods depending on single-frequency spectrogram and combination of single-frequency spectrogram and range map.