DNN Transfer Learning From Diversified Micro-Doppler for Motion Classification

Recently, deep neural networks (DNNs) have been the subject of intense research for the classification of radio frequency signals, such as synthetic aperture radar imagery or micro-Doppler signatures. However, a fundamental challenge is the typically small amount of data available due to the high costs and resources required for measurements. Small datasets limit the depth of DNNs implementable, and limit performance. In this work, a novel method for generating diversified radar micro-Doppler signatures using Kinect-based motion capture simulations is proposed as a training database for transfer learning with DNNs. In particular, it is shown that together with residual learning, the proposed DivNet approach allows for the construction of DNNs and offers improved performance in comparison to transfer learning from optical imagery. Furthermore, it is shown that initializing the network using diversified synthetic micro-Doppler signatures enables not only robust performance for previously unseen target profiles, but also class generalization. Results are presented for 7-class and 11-class human activity recognition scenarios using a 4-GHz continuous wave software-defined radar.

[1]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[2]  Matus Telgarsky,et al.  Representation Benefits of Deep Feedforward Networks , 2015, ArXiv.

[3]  Chu He,et al.  SAR image classification based on the multi-layer network and transfer learning of mid-level representations , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[4]  L. Cifola,et al.  Multi-target human gait classification using deep convolutional neural networks on micro-doppler spectrograms , 2016, 2016 European Radar Conference (EuRAD).

[5]  Daniel Thalmann,et al.  A global human walking model with real-time kinematic personification , 1990, The Visual Computer.

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[8]  Ohad Shamir,et al.  The Power of Depth for Feedforward Neural Networks , 2015, COLT.

[9]  Youngwook Kim,et al.  Hand Gesture Recognition Using Micro-Doppler Signatures With Convolutional Neural Network , 2016, IEEE Access.

[10]  Seong-Ook Park,et al.  Drone Classification Using Convolutional Neural Networks With Merged Doppler Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[11]  Jin Wei,et al.  Deep learning based doppler radar for micro UAS detection and classification , 2016, MILCOM 2016 - 2016 IEEE Military Communications Conference.

[12]  S. Z. Gürbüz,et al.  Deep convolutional autoencoder for radar-based classification of similar aided and unaided human activities , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[13]  Sevgi Zubeyde Gurbuz,et al.  Operational assessment and adaptive selection of micro-Doppler features , 2015 .

[14]  Asli Çelikyilmaz,et al.  Associative Adversarial Networks , 2016, ArXiv.

[15]  Maoguo Gong,et al.  Change Detection in Synthetic Aperture Radar Images Based on Deep Neural Networks , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Erik Blasch,et al.  Radar signals dismount tracking for urban operations , 2006, SPIE Defense + Commercial Sensing.

[17]  Masao Masugi,et al.  Gait Classification of Young Adults, Elderly Non-Fallers, and Elderly Fallers Using Micro-Doppler Radar Signals: Simulation Study , 2017, IEEE Sensors Journal.

[18]  Christian Cochin,et al.  Classification of ships using real and simulated data in a convolutional neural network , 2016, 2016 IEEE Radar Conference (RadarConf).

[19]  Pavlo Molchanov,et al.  Multi-sensor system for driver's hand-gesture recognition , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[20]  Arumugam Nallanathan,et al.  Moving Target Recognition Based on Transfer Learning and Three-Dimensional Over-Complete Dictionary , 2016, IEEE Sensors Journal.

[21]  Tomoaki Ohtsuki,et al.  A new human model for Doppler radar simulation , 2012, 2012 IEEE 23rd International Symposium on Personal, Indoor and Mobile Radio Communications - (PIMRC).

[22]  André Bourdoux,et al.  Micro-Doppler feature extraction using convolutional auto-encoders for low latency target classification , 2017, 2017 IEEE Radar Conference (RadarConf).

[23]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[24]  Sevgi Zubeyde Gurbuz,et al.  A kinect-based human micro-doppler simulator , 2015, IEEE Aerospace and Electronic Systems Magazine.

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[27]  B. P. Salmon,et al.  The development of deep learning in synthetic aperture radar imagery , 2017, 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).

[28]  Hongwei Liu,et al.  Convolutional Neural Network With Data Augmentation for SAR Target Recognition , 2016, IEEE Geoscience and Remote Sensing Letters.

[29]  Andrea Vedaldi,et al.  Visualizing Deep Convolutional Neural Networks Using Natural Pre-images , 2015, International Journal of Computer Vision.

[30]  Youngwook Kim,et al.  Micro-Doppler Based Classification of Human Aquatic Activities via Transfer Learning of Convolutional Neural Networks , 2016, Sensors.

[31]  Chao Wang,et al.  Automatic radar waveform recognition based on time-frequency analysis and convolutional neural network , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[32]  Jianchao Fan,et al.  Change detection of SAR images based on supervised contractive autoencoders and fuzzy clustering , 2017, 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).

[33]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Huanxin Zou,et al.  Deep Convolutional Highway Unit Network for SAR Target Classification With Limited Labeled Training Data , 2017, IEEE Geoscience and Remote Sensing Letters.

[35]  Simon Wagner,et al.  A deep learning SAR ATR system using regularization and prioritized classes , 2017, 2017 IEEE Radar Conference (RadarConf).

[36]  W. Marsden.  I and J , 2012 .

[37]  Hao Ling,et al.  Microdoppler signature simulation of computer animated human and animal motions , 2008, 2008 IEEE Antennas and Propagation Society International Symposium.

[38]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[39]  Melda Yuksel,et al.  Information-Theoretic Feature Selection for Human Micro-Doppler Signature Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[40]  S. Z. Gürbüz,et al.  Deep Neural Network Initialization Methods for Micro-Doppler Classification With Low Training Sample Support , 2017, IEEE Geoscience and Remote Sensing Letters.

[41]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[42]  Shuang Wang,et al.  Using deep neural networks for synthetic aperture radar image registration , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[43]  Markus Hahn,et al.  Potential of radar for static object classification using deep learning methods , 2016, 2016 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM).

[44]  K. Trott,et al.  Stationary Phase Derivation for RCS of an Ellipsoid , 2007, IEEE Antennas and Wireless Propagation Letters.

[45]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  James H. Williams.  Fundamentals of Applied Dynamics , 1995 .

[47]  Cagatay Candan,et al.  Classification of human motion using radar micro-Doppler signatures with hidden Markov models , 2016, 2016 IEEE Radar Conference (RadarConf).

[48]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[49]  E. Blasch,et al.  A comparative cepstral based analysis of simulated and measured S-band and X-band radar Doppler spectra of human motion , 2015, 2015 IEEE Radar Conference.

[50]  Visa Koivunen,et al.  Deep learning for HRRP-based target recognition in multistatic radar systems , 2016, 2016 IEEE Radar Conference (RadarConf).

[51]  Ali Cafer Gürbüz,et al.  Knowledge Exploitation for Human Micro-Doppler Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[52]  Liu Dong,et al.  Work mode identification of airborne phased array radar based on the combination of multi-level modeling and deep learning , 2016, 2016 35th Chinese Control Conference (CCC).

[53]  Hao Ling,et al.  Simulation and Analysis of Human Micro-Dopplers in Through-Wall Environments , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[54]  S. Z. Gürbüz,et al.  Deep learning of micro-Doppler features for aided and unaided gait recognition , 2017, 2017 IEEE Radar Conference (RadarConf).

[55]  Tomas Pfister,et al.  Learning from Simulated and Unsupervised Images through Adversarial Training , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Shobha Sundar Ram,et al.  Modeling and Simulation of Human Motions for Micro-Doppler Signatures , 2017 .

[57]  F. Groen,et al.  Human walking estimation with radar , 2003 .

[58]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[59]  Branka Jokanovic,et al.  Radar fall motion detection using deep learning , 2016, 2016 IEEE Radar Conference (RadarConf).

[60]  Haipeng Wang,et al.  Target Classification Using the Deep Convolutional Networks for SAR Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[61]  Bo Zhang,et al.  Classification of very high resolution SAR image based on convolutional neural network , 2017, 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).

[62]  Igal Bilik,et al.  Pedestrian motion direction estimation using simulated automotive MIMO radar , 2016, IEEE Transactions on Aerospace and Electronic Systems.

[63]  Youngwook Kim,et al.  Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[64]  Nojun Kwak,et al.  Human detection by Neural Networks using a low-cost short-range Doppler radar sensor , 2017, 2017 IEEE Radar Conference (RadarConf).

[65]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.