Real-World ISAR Object Recognition Using Deep Multimodal Relation Learning

Real-world inverse synthetic aperture radar (ISAR) object recognition is a critical and challenging problem in computer vision tasks. In this article, an efficient real-world ISAR object recognition method is proposed, namely, real-world ISAR object recognition (RIOR), based on deep multimodal relation learning (DMRL). It cannot only handle the complex multimodal recognition problem efficiently but also exploit the relations among the features, attributes, labels, and classes with semantic knowledge: 1) an adaptive multimodal mechanism (AMM) is proposed in convolutional neural network (CNN) to substantially promote the CNN sampling and transformation capability and significantly raise the output feature map resolutions by keeping almost all of the information; 2) deep attribute relation graph learning (DARGL) is proposed to jointly estimate the large numbers of heterogeneous attributes and collaboratively explore the relations among the features, attributes, labels, and classes with common knowledge graphs; and 3) relational-regularized convolutional sparse learning (RCSL) is proposed to further achieve good translation invariance and improve the accuracy and speed of the entire system. Extensive qualitative and quantitative experiments are performed on two real-world ISAR datasets, demonstrating that RIOR outperforms the state-of-the-art methods while running quickly.

[1]  Ningning Tong,et al.  DIOD: Fast and Efficient Weakly Semi-Supervised Deep Complex ISAR Object Detection , 2019, IEEE Transactions on Cybernetics.

[2]  Sang Uk Lee,et al.  Attributed relational graph matching based on the nested assignment structure , 2010, Pattern Recognit..

[3]  Xiang Li,et al.  Fast ISAR Cross-Range Scaling Using Modified Newton Method , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[4]  Xiang Li,et al.  Joint Sparse Aperture ISAR Autofocusing and Scaling via Modified Newton Method-Based Variational Bayesian Inference , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Baojun Zhao,et al.  Compressed-Domain Ship Detection on Spaceborne Optical Image Using Deep Neural Network and Extreme Learning Machine , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Salwani Abdullah,et al.  ANTSC: An Intelligent Naïve Bayesian Probabilistic Estimation Practice for Traffic Flow to Form Stable Clustering in VANET , 2018, IEEE Access.

[7]  Xuelong Li,et al.  Hierarchical Recurrent Neural Hashing for Image Retrieval With Hierarchical Convolutional Features , 2018, IEEE Transactions on Image Processing.

[8]  Barnabás Póczos,et al.  Minimax Reconstruction Risk of Convolutional Sparse Dictionary Learning , 2018, AISTATS.

[9]  Seong-Ook Jung,et al.  An Offset-Tolerant Dual-Reference-Voltage Sensing Scheme for Deep Submicrometer STT-RAM , 2016, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[10]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[11]  Lihi Zelnik-Manor,et al.  Template Matching with Deformable Diversity Similarity , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Xuelong Li,et al.  A Hybrid Sparsity and Distance-Based Discrimination Detector for Hyperspectral Images , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Jitendra Malik,et al.  Region-Based Convolutional Networks for Accurate Object Detection and Segmentation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Tao Mei,et al.  Deep Collaborative Embedding for Social Image Understanding , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Xiaogang Wang,et al.  DeepID-Net: Object Detection with Deformable Part Based Convolutional Neural Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Francesco Visin,et al.  A guide to convolution arithmetic for deep learning , 2016, ArXiv.

[18]  Xuan Song,et al.  Object Discovery: Soft Attributed Graph Mining , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Sethuraman Panchanathan,et al.  Deep-Learning Systems for Domain Adaptation in Computer Vision: Learning Transferable Feature Representations , 2017, IEEE Signal Processing Magazine.

[20]  Cordelia Schmid,et al.  DeepMatching: Hierarchical Deformable Dense Matching , 2015, International Journal of Computer Vision.

[21]  Barnabás Póczos,et al.  On the Reconstruction Risk of Convolutional Sparse Dictionary Learning , 2017, ArXiv.

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[23]  Lisa Tang,et al.  Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation , 2016, IEEE Transactions on Medical Imaging.

[24]  Xiang Li,et al.  Bayesian Bistatic ISAR Imaging for Targets With Complex Motion Under Low SNR Condition , 2018, IEEE Transactions on Image Processing.

[25]  Jiao Jiao,et al.  A Densely Connected End-to-End Neural Network for Multiscale and Multiscene SAR Ship Detection , 2018, IEEE Access.

[26]  Hao Guo,et al.  Object Detection with Mask-based Feature Encoding , 2018, ArXiv.

[27]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[28]  Xiaowei Hu,et al.  Automatic recognition of ISAR images based on deep learning , 2016, 2016 CIE International Conference on Radar (RADAR).

[29]  Kyung-Tae Kim,et al.  Efficient Autofocus Chain for ISAR Imaging of Non-Uniformly Rotating Target , 2017, IEEE Sensors Journal.

[30]  Bin Xue,et al.  Real-World ISAR Object Recognition and Relation Discovery Using Deep Relation Graph Learning , 2019, IEEE Access.

[31]  Xin Xu,et al.  DIOD: Fast, Semi-Supervised Deep ISAR Object Detection , 2019, IEEE Sensors Journal.

[32]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[33]  Erwan Scornet,et al.  Random Forests and Kernel Methods , 2015, IEEE Transactions on Information Theory.

[34]  Xiangtao Zheng,et al.  Exploring Models and Data for Remote Sensing Image Caption Generation , 2017, IEEE Transactions on Geoscience and Remote Sensing.