DopNet: A Deep Convolutional Neural Network to Recognize Armed and Unarmed Human Targets

The work presented in this paper aims to distinguish between armed or unarmed personnel using multi-static radar data and advanced Doppler processing. We propose two modified deep convolutional neural networks (DCNNs) termed single channel DopNet (SC-DopNet) and multiple channel DopNet (MC-DopNet) for mono-static and multi-static micro-Doppler signature (<inline-formula> <tex-math notation="LaTeX">$\mu $ </tex-math></inline-formula>-DS) classification. Differentiating armed and unarmed walking personnel is challenging due to the effect of the aspect angle and the channel diversity in real-world scenarios. In addition, the DCNN easily overfits the relatively small-scale <inline-formula> <tex-math notation="LaTeX">$\mu $ </tex-math></inline-formula>-DS dataset. To address these problems, the work carried out in this paper makes three key contributions. First, two effective schemes including a data augmentation operation and a regularization term are proposed to train the SC-DopNet from scratch. Next, a factor analysis of the SC-DopNet is conducted based on various operating parameters in both the processing and radar operations. Third, to solve the problem of aspect angle diversity for the <inline-formula> <tex-math notation="LaTeX">$\mu $ </tex-math></inline-formula>-DS classification, we design the MC-DopNet for multi-static <inline-formula> <tex-math notation="LaTeX">$\mu $ </tex-math></inline-formula>-DS which is embedded with two new fusion schemes termed as greedy importance reweighting (GIR) and <inline-formula> <tex-math notation="LaTeX">$\ell _{21}$ </tex-math></inline-formula>-Norm. These two schemes are based on two different strategies and have been evaluated experimentally. The GIR uses a win by sacrificing worst case approach, whereas <inline-formula> <tex-math notation="LaTeX">$\ell _{21}$ </tex-math></inline-formula>-Norm adopts a win by sacrificing best case approach. The SC-DopNet outperforms the non-deep methods by 12.5% in average, and the proposed MC-DopNet with two fusion methods outperforms the conventional binary voting by 1.2% in average. Note that we also argue and discuss how to utilize the statistics of SC-DopNet results to infer the selection of fusion strategies for the MC-DopNet under different experimental scenarios.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  S. Z. Gürbüz,et al.  Deep Neural Network Initialization Methods for Micro-Doppler Classification With Low Training Sample Support , 2017, IEEE Geoscience and Remote Sensing Letters.

[3]  Youngwook Kim,et al.  Human Detection Using Doppler Radar Based on Physical Characteristics of Targets , 2015, IEEE Geoscience and Remote Sensing Letters.

[4]  Gang Li,et al.  Sparsity-Driven Micro-Doppler Feature Extraction for Dynamic Hand Gesture Recognition , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[5]  Carmine Clemente,et al.  'The Micro-Doppler Effect in Radar' by V.C. Chen , 2012 .

[6]  Francesco Fioranelli,et al.  Aspect angle dependence and multistatic data fusion for micro-Doppler classification of armed/unarmed personnel , 2015 .

[7]  Youngwook Kim,et al.  Classification of micro-Doppler signatures of human aquatic activity through simulation and measurement using transferred learning , 2017, Defense + Security.

[8]  Francesco Fioranelli,et al.  Multistatic human micro-Doppler classification of armed/unarmed personnel , 2015 .

[9]  H. Wechsler,et al.  Micro-Doppler effect in radar: phenomenon, model, and simulation study , 2006, IEEE Transactions on Aerospace and Electronic Systems.

[10]  Karl Woodbridge,et al.  Activity recognition based on micro-Doppler signature with in-home Wi-Fi , 2016, 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom).

[11]  Branka Jokanovic,et al.  Radar fall motion detection using deep learning , 2016, 2016 IEEE Radar Conference (RadarConf).

[12]  Ram M. Narayanan,et al.  Classification of human motions using empirical mode decomposition of human micro-Doppler signatures , 2014 .

[13]  Branka Jokanovic,et al.  Multiple joint-variable domains recognition of human motion , 2017, 2017 IEEE Radar Conference (RadarConf).

[14]  Gang Li,et al.  Personnel Recognition and Gait Classification Based on Multistatic Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[15]  Alessio Balleri,et al.  Extraction and analysis of micro-Doppler signatures by the Empirical Mode Decomposition , 2015, 2015 IEEE Radar Conference (RadarCon).

[16]  Qingchao Chen,et al.  Through Wall Radar Classification of Human Micro-Doppler Using Singular Value Decomposition Analysis , 2016, Sensors.

[17]  Youngwook Kim,et al.  Hand Gesture Recognition Using Micro-Doppler Signatures With Convolutional Neural Network , 2016, IEEE Access.

[18]  Branka Jokanovic,et al.  Effect of data representations on deep learning in fall detection , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[19]  Youngwook Kim,et al.  Human Activity Classification Based on Micro-Doppler Signatures Using a Support Vector Machine , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Youngwook Kim,et al.  Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[21]  Francesco Fioranelli,et al.  Bistatic human micro-Doppler signatures for classification of indoor activities , 2017, 2017 IEEE Radar Conference (RadarConf).

[22]  Yang Liu,et al.  Joint fall and aspect angle recognition using fine-grained micro-Doppler classification , 2017, 2017 IEEE Radar Conference (RadarConf).

[23]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[24]  Francesco Fioranelli,et al.  Multistatic radar classification of armed vs unarmed personnel using neural networks , 2018, Evol. Syst..

[25]  Alessio Balleri,et al.  Practical investigation of multiband mono- and bistatic radar signatures of wind turbines , 2017 .

[26]  Francesco Fioranelli,et al.  Performance Analysis of Centroid and SVD Features for Personnel Recognition Using Multistatic Micro-Doppler , 2016, IEEE Geoscience and Remote Sensing Letters.

[27]  Ram M. Narayanan,et al.  Radar micro-Doppler signatures of various human activities , 2015 .

[28]  Sevgi Zubeyde Gurbuz,et al.  Operational assessment and adaptive selection of micro-Doppler features , 2015 .

[29]  Moeness Amin,et al.  Fall Detection Using Deep Learning in Range-Doppler Radars , 2018, IEEE Transactions on Aerospace and Electronic Systems.