H2RBox-v2: Incorporating Symmetry for Boosting Horizontal Box Supervised Oriented Object Detection

With the rapidly increasing demand for oriented object detection e.g. in autonomous driving and remote sensing, the recently proposed paradigm involving weakly-supervised detector H2RBox for learning rotated box (RBox) from the (currently) more readily-available horizontal box (HBox) has shown promise. This paper presents H2RBox-v2, to further bridge the gap between HBox-supervised and RBox-supervised oriented object detection. Specifically, we propose to leverage the reflection symmetry via flip and rotate consistencies, using a weakly-supervised network branch similar to H2RBox, together with a novel self-supervised branch that learns orientations from the symmetry inherent in visual objects. The detector is further stabilized and enhanced by practical techniques to cope with peripheral issues e.g. angular periodicity. To our best knowledge, H2RBox-v2 is the first symmetry-aware self-supervised paradigm for oriented object detection. In particular, our method shows less susceptibility to low-quality annotation and insufficient training data compared to H2RBox. Specifically, H2RBox-v2 achieves very close performance to a rotation annotation trained counterpart -- Rotated FCOS: 1) DOTA-v1.0/1.5/2.0: 72.31%/64.76%/50.33% vs. 72.44%/64.53%/51.77%; 2) HRSC: 89.66% vs. 88.99%; 3) FAIR1M: 42.27% vs. 41.25%.

[1]  T. Drummond,et al.  Knowledge Combination to Learn Rotated Detection without Rotated Annotation , 2023, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Long Wen,et al.  A comprehensive survey of oriented object detection in remote sensing images , 2023, Expert Syst. Appl..

[3]  Junchi Yan,et al.  Detecting Rotated Objects as Gaussian Distributions and its 3-D Generalization , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Kai Xu,et al.  Learning to Detect 3D Symmetry From Single-View RGB-D Images With Weak Supervision , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Xue Yang,et al.  G-Rep: Gaussian Representation for Arbitrary-Oriented Object Detection , 2022, Remote. Sens..

[6]  Zhi-guo Jiang,et al.  WSODet: A Weakly Supervised Oriented Detector for Aerial Object Detection , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Yi Yu,et al.  Phase-Shifting Coder: Predicting Accurate Orientation in Oriented Object Detection , 2022, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Junchi Yan,et al.  H2RBox: Horizontal Box Annotation is All You Need for Oriented Object Detection , 2022, ICLR.

[9]  Jianke Zhu,et al.  Box-supervised Instance Segmentation with Level Set Evolution , 2022, ECCV.

[10]  Jian Xue,et al.  Shape-Adaptive Selection and Measurement for Oriented Object Detection , 2022, AAAI Conference on Artificial Intelligence.

[11]  Junchi Yan,et al.  MMRotate: A Rotated Object Detection Benchmark using PyTorch , 2022, ACM Multimedia.

[12]  Suha Kwak,et al.  Reflection and Rotation Symmetry Detection via Equivariant Learning , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Junchi Yan,et al.  The KFIoU Loss for Rotated Object Detection , 2022, ICLR.

[14]  Junwei Han,et al.  Anchor-Free Oriented Proposal Generator for Object Detection , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Jianke Zhu,et al.  Oriented RepPoints for Aerial Object Detection , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Stefan Hinz,et al.  FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagery , 2021, ISPRS Journal of Photogrammetry and Remote Sensing.

[17]  Gui-Song Xia,et al.  Align Deep Features for Oriented Object Detection , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Junchi Yan,et al.  On the Arbitrary-Oriented Object Detection: Classification Based Approaches Revisited , 2020, International Journal of Computer Vision.

[19]  Hongjin Wu,et al.  PCBNet: A Lightweight Convolutional Neural Network for Defect Inspection in Surface Mount Technology , 2022, IEEE Transactions on Instrumentation and Measurement.

[20]  Junwei Han,et al.  Oriented R-CNN for Object Detection , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[21]  Junchi Yan,et al.  Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence , 2021, NeurIPS.

[22]  Jianbin Jiao,et al.  Beyond Bounding-Box: Convex-hull Feature Adaptation for Oriented and Densely Packed Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Arif Mahmood,et al.  Leveraging orientation for weakly supervised object detection with application to firearm localization , 2021, Neurocomputing.

[24]  Gui-Song Xia,et al.  ReDet: A Rotation-equivariant Detector for Aerial Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Junchi Yan,et al.  Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss , 2021, ICML.

[26]  Zhi Tian,et al.  BoxInst: High-Performance Instance Segmentation with Box Annotations , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Junchi Yan,et al.  Dense Label Encoding for Boundary Discontinuity Free Rotation Detection , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[29]  Xue Yang,et al.  Learning Modulated Loss for Rotated Object Detection , 2019, AAAI.

[30]  Junchi Yan,et al.  R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object , 2019, AAAI.

[31]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[32]  Shi-Min Hu,et al.  Jittor: a novel deep learning framework with meta-operators and unified graph execution , 2020, Science China Information Sciences.

[33]  Hongli Gao,et al.  A Data-Flow Oriented Deep Ensemble Learning Method for Real-Time Surface Defect Inspection , 2020, IEEE Transactions on Instrumentation and Measurement.

[34]  Shanmuganathan Raman,et al.  3DSymm: Robust and Accurate 3D Reflection Symmetry Detection , 2020, Pattern Recognit..

[35]  Weiming Dong,et al.  Dynamic Refinement Network for Oriented and Densely Packed Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Junchi Yan,et al.  Arbitrary-Oriented Object Detection with Circular Smooth Label , 2020, ECCV.

[37]  Hao Chen,et al.  Conditional Convolutions for Instance Segmentation , 2020, ECCV.

[38]  Yue Zhang,et al.  Rotation-aware and multi-scale convolutional neural network for object detection in remote sensing images , 2020 .

[39]  Junwei Han,et al.  Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[40]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[42]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[43]  Yang Long,et al.  Learning RoI Transformer for Oriented Object Detection in Aerial Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Stephen Lin,et al.  RepPoints: Point Set Representation for Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[45]  Hao Chen,et al.  FCOS: Fully Convolutional One-Stage Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[46]  Tal Hassner,et al.  Precise Detection in Densely Packed Scenes , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Yue Zhang,et al.  SCRDet: Towards More Robust Detection for Small, Cluttered and Rotated Objects , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[48]  Matti Pietikäinen,et al.  Deep Learning for Generic Object Detection: A Survey , 2018, International Journal of Computer Vision.

[49]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[50]  Frank Hutter,et al.  Decoupled Weight Decay Regularization , 2017, ICLR.

[51]  Shanmuganathan Raman,et al.  Detecting Approximate Reflection Symmetry in a Point Set Using Optimization on Manifold , 2017, IEEE Transactions on Signal Processing.

[52]  Yung-Yu Chuang,et al.  Weakly Supervised Instance Segmentation using the Bounding Box Tightness Prior , 2019, NeurIPS.

[53]  Gui-Song Xia,et al.  Rotation-Sensitive Regression for Oriented Scene Text Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[54]  Menglong Yan,et al.  Automatic Ship Detection in Remote Sensing Images from Google Earth of Complex Scenes Based on Multiscale Rotation Dense Feature Pyramid Networks , 2018, Remote. Sens..

[55]  Junjie Yan,et al.  FOTS: Fast Oriented Text Spotting with a Unified Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[56]  Jiebo Luo,et al.  DOTA: A Large-Scale Dataset for Object Detection in Aerial Images , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[57]  Baoxin Li,et al.  Joint Cuts and Matching of Partitions in One Graph , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[58]  Xiangyang Xue,et al.  Arbitrary-Oriented Scene Text Detection via Rotation Proposals , 2017, IEEE Transactions on Multimedia.

[59]  Wafa Khlif,et al.  ICDAR2017 Robust Reading Challenge on Multi-Lingual Scene Text Detection and Script Identification - RRC-MLT , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[60]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[61]  Sven J. Dickinson,et al.  2017 ICCV Challenge: Detecting Symmetry in the Wild , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[62]  Shuchang Zhou,et al.  EAST: An Efficient and Accurate Scene Text Detector , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Yiping Yang,et al.  A High Resolution Optical Satellite Image Dataset for Ship Recognition and Some New Baselines , 2017, ICPRAM.

[64]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Bernt Schiele,et al.  Simple Does It: Weakly Supervised Instance and Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[66]  Yuning Jiang,et al.  UnitBox: An Advanced Object Detection Network , 2016, ACM Multimedia.

[67]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[68]  池内 克史,et al.  Computer Vision: A Reference Guide , 2014 .

[69]  Yanxi Liu,et al.  Computational Symmetry , 2014, Computer Vision, A Reference Guide.

[70]  Yanxi Liu,et al.  Curved Glide-Reflection Symmetry Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.