View-Semantic Transformer With Enhancing Diversity for Sparse-View SAR Target Recognition

With the rapid development of supervised learning-based synthetic aperture radar (SAR) target recognition technology, it is easy to find that the recognition performance is proportional to the number of training samples. However, the biased data distribution and under-representation of the model caused by incomplete data within categories exacerbate the challenge of SAR interpretation. In this article, we propose a new view-semantic transformer network (VSTNet) that generates synthesized samples to complete the statistical distribution of training data and improve the discriminative representation of the model. First, SAR images from different views are encoded into a disentangled latent space, which allows us to synthesize data with more diverse views by manipulating view-semantic features. Second, the synthesized data as a complement effectively expands the training set and alleviates the overfitting problem of limited data in sparse views. Third, the proposed method unifies SAR image synthesis and SAR target recognition into an end-to-end framework to boost their performance against each other. Experiments conducted on moving and stationary target acquisition and recognition (MSTAR) data demonstrate the robustness and effectiveness of the proposed method.

[1]  Yilong Yin,et al.  Adaptive Feature Aggregation in Deep Multi-Task Convolutional Neural Networks , 2022, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Quan Pan,et al.  Rotation Awareness Based Self-Supervised Learning for SAR Target Recognition With Limited Training Samples , 2021, IEEE Transactions on Image Processing.

[3]  Feng Xu,et al.  Learning to Generate SAR Images With Adversarial Autoencoder , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Yudong Chen,et al.  A Survey on Curriculum Learning , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Hongming Shan,et al.  When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Xiaorui Ma,et al.  Transfer Learning for SAR Image Classification Via Deep Joint Distribution Adaptation Networks , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Ekaterina Iakovleva,et al.  Meta-Learning with Shared Amortized Variational Inference , 2020, ICML.

[8]  Ching Y. Suen,et al.  Towards Robust Pattern Recognition: A Review , 2020, Proceedings of the IEEE.

[9]  Zongjie Cao,et al.  LDGAN: A Synthetic Aperture Radar Image Generation Method for Automatic Target Recognition , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Aaron Hertzmann,et al.  GANSpace: Discovering Interpretable GAN Controls , 2020, NeurIPS.

[11]  Xianzhi Li,et al.  PointAugment: An Auto-Augmentation Framework for Point Cloud Classification , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Michael I. Jordan,et al.  AUTO-ENCODING VARIATIONAL BAYES , 2020 .

[13]  Artem Babenko,et al.  Unsupervised Discovery of Interpretable Directions in the GAN Latent Space , 2020, ICML.

[14]  Bolei Zhou,et al.  Interpreting the Latent Space of GANs for Semantic Face Editing , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Phillip Isola,et al.  On the "steerability" of generative adversarial networks , 2019, ICLR.

[16]  Eric Eaton,et al.  Deep Transfer Learning for Few-Shot SAR Image Classification , 2019, Remote. Sens..

[17]  Peter Wonka,et al.  Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[18]  Zongjie Cao,et al.  Image Data Augmentation for SAR Sensor via Generative Adversarial Nets , 2019, IEEE Access.

[19]  Siwei Ma,et al.  Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Feng Xu,et al.  Reciprocal translation between SAR and optical remote sensing images with cascaded-residual adversarial networks , 2019, Science China Information Sciences.

[21]  Yoshua Bengio,et al.  MetaGAN: An Adversarial Approach to Few-Shot Learning , 2018, NeurIPS.

[22]  Vladlen Koltun,et al.  Multi-Task Learning as Multi-Objective Optimization , 2018, NeurIPS.

[23]  Shih-Fu Chang,et al.  Low-shot Learning via Covariance-Preserving Adversarial Augmentation Networks , 2018, NeurIPS.

[24]  Lei Liu,et al.  SAR Target Classification with CycleGAN Transferred Simulated Samples , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[25]  Jianyu Yang,et al.  SAR Automatic Target Recognition Based on Multiview Deep Learning Framework , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Wang Ping,et al.  Research on data augmentation for image classification based on convolution neural networks , 2017, 2017 Chinese Automation Congress (CAC).

[27]  Andrew Zisserman,et al.  Multi-task Self-Supervised Visual Learning , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[28]  Allan Aasbjerg Nielsen,et al.  Improving SAR Automatic Target Recognition Models With Transfer Learning From Simulated Data , 2017, IEEE Geoscience and Remote Sensing Letters.

[29]  Zhi Chen,et al.  Adversarial Feature Matching for Text Generation , 2017, ICML.

[30]  Yueting Zhang,et al.  Synthetic Aperture Radar Image Synthesis by Using Generative Adversarial Nets , 2017, IEEE Geoscience and Remote Sensing Letters.

[31]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[32]  Hong Yu,et al.  Meta Networks , 2017, ICML.

[33]  Arumugam Nallanathan,et al.  Moving Target Recognition Based on Transfer Learning and Three-Dimensional Over-Complete Dictionary , 2016, IEEE Sensors Journal.

[34]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[35]  Andrea Vedaldi,et al.  Integrated perception with recurrent multi-task neural networks , 2016, NIPS.

[36]  Haipeng Wang,et al.  Target Classification Using the Deep Convolutional Networks for SAR Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Hongwei Liu,et al.  Convolutional Neural Network With Data Augmentation for SAR Target Recognition , 2016, IEEE Geoscience and Remote Sensing Letters.

[38]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[40]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[41]  Jing Wang,et al.  Robust Face Recognition via Adaptive Sparse Representation , 2014, IEEE Transactions on Cybernetics.

[42]  Hugh Griffiths,et al.  Radar Automatic Target Recognition (ATR) and Non-Cooperative Target Recognition (NCTR) , 2013 .

[43]  I. Hajnsek,et al.  A tutorial on synthetic aperture radar , 2013, IEEE Geoscience and Remote Sensing Magazine.

[44]  Yoshua Bengio,et al.  Better Mixing via Deep Representations , 2012, ICML.

[45]  Thomas S. Huang,et al.  Multi-View Automatic Target Recognition using Joint Sparse Representation , 2012, IEEE Transactions on Aerospace and Electronic Systems.

[46]  Karsten Schulz,et al.  Coherent simulation of SAR images , 2009, Remote Sensing.

[47]  Bir Bhanu,et al.  Exploiting azimuthal variance of scatterers for multiple-look SAR recognition , 2002, SPIE Defense + Commercial Sensing.

[48]  Michael Lee Bryant,et al.  Standard SAR ATR evaluation experiments using the MSTAR public release data set , 1998, Defense, Security, and Sensing.

[49]  Armin W. Doerry,et al.  Synthetic Aperture Radar , 1992, Inverse Synthetic Aperture Radar Imaging with MATLAB® Algorithms.

[50]  S. Lee,et al.  Shooting and bouncing rays: calculating the RCS of an arbitrarily shaped cavity , 1989 .

[51]  Chenwei Wang,et al.  Multiview Attention CNN-LSTM Network for SAR Automatic Target Recognition , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[52]  Jinsung Yoon,et al.  GENERATIVE ADVERSARIAL NETS , 2018 .

[53]  Stefan Auer,et al.  3D Synthetic Aperture Radar Simulation for Interpreting Complex Urban Reflection Scenarios , 2011 .