A Deformation Robust ISAR Image Satellite Target Recognition Method Based on PT-CCNN

To tackle the inherent unknown deformation (e.g., translation, rotation and scaling) of the inverse synthetic aperture radar (ISAR) images, a deep polar transformer-circular convolutional neural network, i.e., PT-CCNN, is proposed to achieve deformation robust ISAR image automatic target recognition (ATR) in this article. Inspired by human visual system and canonical coordinate of Lie-groups, we adopt a polar transformer module to transform the deformation ISAR images to the log-polar representations, before which a conventional convolutional neural network (CNN) is utilized to predict the origin of log-polar transformation. The above techniques make the proposed network invariant to translation, and equivariant to rotation and scaling. On this basis, for the log-polar representations with wrap-around structure, a circular convolutional neural network (CCNN) is further applied to extract more effective and highly discriminative features and improve recognition accuracy. The proposed network is end-to-end trainable with a classification loss, and could extract deformation-robust and essential features automatically. For multiple practical ISAR image datasets of six satellites, the performance testing and comparison experiments demonstrate that the techniques utilized in PT-CCNN are effective, and our proposed network achieve higher recognition accuracy than those previous common methods based on deep learning. For instance, our proposed PT-CCNN beats traditional CNN on rotation, scaling and practical deformation datasets by 24.5-49.3%, 9.0-40.8% and 22.3-26.7%. And it also outperforms the polar CNN without using the above techniques on rotation, scaling and practical deformation datasets by 9.2-53.7%, 5.2-54.6% and 9.0-49.9%. Additionally, the presented visualization results show the abilities and advantages of our method in terms of handling image deformation and extracting effective features.

[1]  Kyung-Tae Kim,et al.  ISAR Cross-Range Scaling Using Iterative Processing via Principal Component Analysis and Bisection Algorithm , 2016, IEEE Transactions on Signal Processing.

[2]  Peer Neubert,et al.  Circular Convolutional Neural Networks for Panoramic Images and Laser Data , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[3]  Daniel Serrano,et al.  Computer Vision and Deep Learning Techniques for the Analysis of Drone-Acquired Forest Images, a Transfer Learning Study , 2020, Remote. Sens..

[4]  Xiang Li,et al.  Target classification of ISAR images based on feature space optimisation of local non-negative matrix factorisation , 2012, IET Signal Process..

[5]  Abdelmalek Toumi,et al.  Radar Target Recognition Using Salient Keypoint Descriptors and Multitask Sparse Representation , 2018, Remote. Sens..

[6]  Tsorng-Lin Chia,et al.  Using invariants to recognize airplanes in inverse synthetic aperture radar images , 2003 .

[7]  Marco Martorella,et al.  Point Target Classification via Fast Lossless and Sufficient $\Omega$–$\Psi$ –$\Phi$ Invariant Decomposition of High-Resolution and Fully Polarimetric SAR/ISAR Data , 2013, Proceedings of the IEEE.

[8]  Zhu Yan Automatic target recognition of ISAR images based on SIFT features , 2009 .

[9]  Jonathan Tompson,et al.  Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation , 2014, NIPS.

[10]  Yehoshua Y. Zeevi,et al.  The Canonical Coordinates Method for Pattern Deformation: Theoretical and Computational Considerations , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Alexandre Bernardino,et al.  A review of log-polar imaging for visual perception in robotics , 2010, Robotics and Autonomous Systems.

[12]  Pengfei Duan,et al.  Real-Time 3D Face Alignment Using an Encoder-Decoder Network With an Efficient Deconvolution Layer , 2020, IEEE Signal Processing Letters.

[13]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  Simon A. Wagner,et al.  SAR ATR by a combination of convolutional neural network and support vector machines , 2016, IEEE Transactions on Aerospace and Electronic Systems.

[15]  Antonino Laudani,et al.  Automatic Aircraft Target Recognition by ISAR Image Processing based on Neural Classifier , 2012 .

[16]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[17]  LiChenliang,et al.  Adversarial Attacks on Deep-learning Models in Natural Language Processing , 2020 .

[18]  Xiaofeng Ma,et al.  ISAR Image Recognition with Fusion of Gabor Magnitude and Phase Feature: ISAR Image Recognition with Fusion of Gabor Magnitude and Phase Feature , 2014 .

[19]  Weiwei Cai,et al.  Remote Sensing Image Classification Based on a Cross-Attention Mechanism and Graph Convolution , 2020, IEEE Geoscience and Remote Sensing Letters.

[20]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[21]  Julie Ann Jackson,et al.  Canonical Scattering Feature Models for 3D and Bistatic SAR , 2010, IEEE Transactions on Aerospace and Electronic Systems.

[22]  Abdelmalek Toumi,et al.  Aircrafts Recognition using Convolutional Neurons Network , 2017 .

[23]  Zheng Bao,et al.  Joint ISAR Imaging and Cross-Range Scaling Method Based on Compressive Sensing With Adaptive Dictionary , 2015, IEEE Transactions on Antennas and Propagation.

[24]  Zhongyuan Wang,et al.  Saliency-Aware Convolution Neural Network for Ship Detection in Surveillance Video , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Marco Martorella,et al.  Classification of Man-Made Targets via Invariant Coherency-Matrix Eigenvector Decomposition of Polarimetric SAR/ISAR Images , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Andrew R. Webb,et al.  Nonlinear feature extraction applied to ISAR images of targets for classification , 2001, SPIE Defense + Commercial Sensing.

[27]  Max Welling,et al.  Group Equivariant Convolutional Networks , 2016, ICML.

[28]  Kyung-Tae Kim,et al.  Efficient classification of ISAR images , 2005, IEEE Transactions on Antennas and Propagation.

[29]  Ningning Tong,et al.  Real-World ISAR Object Recognition Using Deep Multimodal Relation Learning , 2020, IEEE Transactions on Cybernetics.

[30]  Wang Lu,et al.  A Deep Learning-Based Satellite Target Recognition Method Using Radar Data , 2019, Sensors.

[31]  S. Kesaraju,et al.  Range-Doppler Mapping of Space-Based Targets Using the JRO 50 MHz Radar , 2017 .

[32]  Zhenfeng Shao,et al.  IHS-GTF: A Fusion Method for Optical and Synthetic Aperture Radar Data , 2020, Remote. Sens..

[33]  Lei Wang,et al.  Stacked Sparse Autoencoder Modeling Using the Synergy of Airborne LiDAR and Satellite Optical and SAR Data to Map Forest Above-Ground Biomass , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Rafael Pires de Lima,et al.  Convolutional Neural Network for Remote-Sensing Scene Classification: Transfer Learning Analysis , 2019, Remote. Sens..

[36]  Xiaowei Hu,et al.  Automatic recognition of ISAR images based on deep learning , 2016, 2016 CIE International Conference on Radar (RADAR).

[37]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[38]  Lei Guo,et al.  The Generalized Gamma-DBN for High-Resolution SAR Image Classification , 2018, Remote. Sens..

[39]  Andrea Vedaldi,et al.  Understanding Image Representations by Measuring Their Equivariance and Equivalence , 2014, International Journal of Computer Vision.

[40]  Xiaowei Zhou,et al.  Polar Transformer Networks , 2017, ICLR.

[41]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[42]  Sang-Hong Park,et al.  Efficient classification of ISAR images using 2d fourier transform and polar mapping , 2015, IEEE Transactions on Aerospace and Electronic Systems.

[43]  Deren Li,et al.  Object Detection in UAV Images via Global Density Fused Convolutional Network , 2020, Remote. Sens..

[44]  Xueru Bai,et al.  Robust Pol-ISAR Target Recognition Based on ST-MC-DCNN , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[46]  Li Xiang Automatic target recognition of ISAR images based on geometric hash , 2012 .

[47]  Li Yin,et al.  Mapping Urban Impervious Surface by Fusing Optical and SAR Data at the Decision Level , 2016, Remote. Sens..

[48]  Driss Aboutajdine,et al.  Automatic target recognition of aircraft models based on ISAR images , 2009, 2009 IEEE International Geoscience and Remote Sensing Symposium.

[49]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Anil K. Jain,et al.  Object Matching Using Deformable Templates , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Andrea Vedaldi,et al.  Warped Convolutions: Efficient Invariance to Spatial Transformations , 2016, ICML.

[52]  Jiebo Luo,et al.  Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-Grained Image Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).