AI-based object detection latest trends in remote sensing, multimedia and agriculture applications

Object detection is a vital research direction in machine vision and deep learning. The object detection technique based on deep understanding has achieved tremendous progress in feature extraction, image representation, classification, and recognition in recent years, due to this rapid growth of deep learning theory and technology. Scholars have proposed a series of methods for the object detection algorithm as well as improvements in data processing, network structure, loss function, and so on. In this paper, we introduce the characteristics of standard datasets and critical parameters of performance index evaluation, as well as the network structure and implementation methods of two-stage, single-stage, and other improved algorithms that are compared and analyzed. The latest improvement ideas of typical object detection algorithms based on deep learning are discussed and reached, from data enhancement, a priori box selection, network model construction, prediction box selection, and loss calculation. Finally, combined with the existing challenges, the future research direction of typical object detection algorithms is surveyed.

[1]  A. Bimbo,et al.  A full data augmentation pipeline for small object detection based on generative adversarial networks , 2022, Pattern Recognit..

[2]  Huchuan Lu,et al.  PANet: Patch-Aware Network for Light Field Salient Object Detection , 2021, IEEE Transactions on Cybernetics.

[3]  Mesmin J. Mbyamm Kiki,et al.  An Improved Approach to Detection of Rice Leaf Disease with GAN-Based Data Augmentation Pipeline , 2023, SSRN Electronic Journal.

[4]  Ling Jian,et al.  SS R-CNN: Self-Supervised Learning Improving Mask R-CNN for Ship Detection in Remote Sensing Images , 2022, Remote. Sens..

[5]  Hayat Ullah,et al.  A Spatial AI-Based Agricultural Robotic Platform for Wheat Detection and Collision Avoidance , 2022, AI.

[6]  Qichao Zhao,et al.  YOLOPv2: Better, Faster, Stronger for Panoptic Driving Perception , 2022, ArXiv.

[7]  Hoanh Nguyen.  An Efficient License Plate Detection Approach Using Lightweight Deep Convolutional Neural Networks , 2022, Advances in Multimedia.

[8]  Jitendra V. Tembhurne,et al.  Object detection using YOLO: challenges, architectural successors, datasets and applications , 2022, Multimedia tools and applications.

[9]  B. Liu,et al.  An Improved EfficientNet for Rice Germ Integrity Classification and Recognition , 2022, Agriculture.

[10]  Deepak Kumar,et al.  Comparative analysis of validating parameters in the deep learning models for remotely sensed images , 2022, Journal of Discrete Mathematical Sciences and Cryptography.

[11]  Xu Chang.  Application of Computer Vision Technology in Post-Harvest Processing of Fruits and Vegetables: Starting from Shape Recognition Algorithm , 2022, 2022 International Conference on Applied Artificial Intelligence and Computing (ICAAIC).

[12]  SK Hafizul Islam,et al.  An object detection-based few-shot learning approach for multimedia quality assessment , 2022, Multimedia Systems.

[13]  Qinggang Meng,et al.  Leaf Disease Segmentation and Detection in Apple Orchards for Precise Smart Spraying in Sustainable Agriculture , 2022, Sustainability.

[14]  Lu Zheng,et al.  A Survey of Research on Crowd Abnormal Behavior Detection Algorithm Based on YOLO Network , 2022, 2022 2nd International Conference on Consumer Electronics and Computer Engineering (ICCECE).

[15]  Jun Liu,et al.  Synthetic Data Augmentation Using Multiscale Attention CycleGAN for Aircraft Detection in Remote Sensing Images , 2021, IEEE Geoscience and Remote Sensing Letters.

[16]  Turgay Celik,et al.  Gated Ladder-Shaped Feature Pyramid Network for Object Detection in Optical Remote Sensing Images , 2021, IEEE Geoscience and Remote Sensing Letters.

[17]  Zhongxiang Wang,et al.  Real-time Detection of Tiny Objects Based on a Weighted Bi-directional FPN , 2022, MMM.

[18]  Deepak Hanike Basavegowda,et al.  Indicator plant species detection in grassland using EfficientDet object detector , 2022, GIL Jahrestagung.

[19]  N. Savarimuthu,et al.  Investigation on Object Detection Models for Plant Disease Detection Framework , 2021, 2021 IEEE 6th International Conference on Computing, Communication and Automation (ICCCA).

[20]  M. Kumar,et al.  Emerging non-destructive imaging techniques for fruit damage detection: Image processing and analysis , 2021, Trends in Food Science & Technology.

[21]  Zhenhong Du,et al.  Identifying Damaged Buildings in Aerial Images Using the Object Detection Method , 2021, Remote. Sens..

[22]  Dewei Yi,et al.  Probabilistic faster R-CNN with stochastic region proposing: Towards object detection and recognition in remote sensing imagery , 2021, Neurocomputing.

[23]  Muhammad Usman Shoukat,et al.  A hybrid approach to forecast the COVID-19 epidemic trend , 2021, PloS one.

[24]  Daxiang Xiang,et al.  Object detection with noisy annotations in high-resolution remote sensing images using robust EfficientDet , 2021, Remote Sensing.

[25]  E. Sert.  A deep learning based approach for the detection of diseases in pepper and potato leaves , 2021, ANADOLU JOURNAL OF AGRICULTURAL SCIENCES.

[26]  Shuo Wang.  Research Towards Yolo-Series Algorithms: Comparison and Analysis of Object Detection Models for Real-Time UAV Applications , 2021, Journal of Physics: Conference Series.

[27]  Xiaodong Zeng,et al.  Improved Mask R-CNN for Aircraft Detection in Remote Sensing Images , 2021, Sensors.

[28]  Linwang Yuan,et al.  Advanced Color Edge Detection Using Clifford Algebra in Satellite Images , 2021, IEEE Photonics Journal.

[29]  Xiwen Yao,et al.  Cross-Scale Feature Fusion for Object Detection in Optical Remote Sensing Images , 2021, IEEE Geoscience and Remote Sensing Letters.

[30]  Che-Lun Hung,et al.  A Smartphone-Based Application for Scale Pest Detection Using Multiple-Object Detection Methods , 2021, Electronics.

[31]  Harin Sellahewa,et al.  An Efficient Pest Classification In Smart Agriculture Using Transfer Learning , 2021, EAI Endorsed Trans. Ind. Networks Intell. Syst..

[32]  R. Namitha,et al.  Effective fault detection approach for cloud computing , 2021 .

[33]  Ehsan Dehghan-Niri,et al.  Non-destructive thermal imaging for object detection via advanced deep learning for robotic inspection and harvesting of chili peppers , 2021 .

[34]  Thanseeha Kassim,et al.  Modified ML-kNN and Rank SVM for Multi-label Pattern Classification , 2021 .

[35]  Zhenhong Jia,et al.  Multi-Feature Fusion Target Re-Location Tracking Based on Correlation Filters , 2021, IEEE Access.

[36]  Yakoub Bazi,et al.  Classification of Remote Sensing Images Using EfficientNet-B3 CNN Model With Attention , 2021, IEEE Access.

[37]  Huang-Chu Huang,et al.  Multi-scale ResNet for real-time underwater object detection , 2020, Signal, Image and Video Processing.

[38]  Jingbing Li,et al.  A Novel Hybrid Discrete Cosine Transform Speeded Up Robust Feature-Based Secure Medical Image Watermarking Algorithm , 2020, J. Medical Imaging Health Informatics.

[39]  Ian Goodfellow,et al.  Generative adversarial networks , 2020, Commun. ACM.

[40]  Hanzi Wang,et al.  Dual Semantic Fusion Network for Video Object Detection , 2020, ACM Multimedia.

[41]  Min Li,et al.  Agricultural Greenhouses Detection in High-Resolution Satellite Images Based on Convolutional Neural Networks: Comparison of Faster R-CNN, YOLO v3 and SSD , 2020, Sensors.

[42]  Ubaid M. Al-Saggaf,et al.  Real-Time Stress Assessment Using Sliding Window Based Convolutional Neural Network , 2020, Sensors.

[43]  Chengjun Xie,et al.  AF-RCNN: An anchor-free convolutional neural network for multi-categories agricultural pest detection , 2020, Comput. Electron. Agric..

[44]  Xinlu Zhang,et al.  Target detection algorithm based on improved multi-scale SSD , 2020, Journal of Physics: Conference Series.

[45]  Jun Guo,et al.  Attention-guided Context Feature Pyramid Network for Object Detection , 2020, ArXiv.

[46]  Yiquan Wu,et al.  Recent advances in small object detection based on deep learning: A review , 2020, Image Vis. Comput..

[47]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[48]  Huihui Yu,et al.  Applications of Deep Learning for Dense Scenes Analysis in Agriculture: A Review , 2020, Sensors.

[49]  Peng Zhang,et al.  Tiny-RetinaNet: a one-stage detector for real-time object detection , 2020, International Conference on Graphic and Image Processing.

[50]  Alexandros G. Dimakis,et al.  Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Quoc V. Le,et al.  EfficientDet: Scalable and Efficient Object Detection , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Junwei Han,et al.  Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[53]  J. Uijlings,et al.  The Open Images Dataset V4 , 2018, International Journal of Computer Vision.

[54]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Ross B. Girshick,et al.  Mask R-CNN , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Shoulin Yin,et al.  A brief review and challenges of object detection in optical remote sensing imagery , 2020, Multiagent Grid Syst..

[57]  Amit Kumar Das,et al.  Automated Detection of Plant Diseases Using Image Processing and Faster R-CNN Algorithm , 2019, 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI).

[58]  Hyeong-Ju Kang,et al.  Real-Time Object Detection on 640x480 Image With VGG16+SSD , 2019, 2019 International Conference on Field-Programmable Technology (ICFPT).

[59]  Dino Ienco,et al.  Combining Sentinel-1 and Sentinel-2 Satellite Image Time Series for land cover mapping via a multi-source deep learning architecture , 2019 .

[60]  Rongrong Ji,et al.  Holistic CNN Compression via Low-Rank Decomposition with Knowledge Transfer , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Licheng Jiao,et al.  A multiscale object detection approach for remote sensing images based on MSE-DenseNet and the dynamic anchor assignment , 2019, Remote Sensing Letters.

[62]  S. Sofana Reka,et al.  Real Time Multi Object Detection for Blind Using Single Shot Multibox Detector , 2019, Wirel. Pers. Commun..

[63]  Qi Hu,et al.  RGB-D Image Multi-Target Detection Method Based on 3D DSF R-CNN , 2019, Int. J. Pattern Recognit. Artif. Intell..

[64]  Mingyu Gao,et al.  Adaptive anchor box mechanism to improve the accuracy in the object detection system , 2019, Multimedia Tools and Applications.

[65]  Yiming Yan,et al.  A Data Augmentation Strategy Based on Simulated Samples for Ship Detection in RGB Remote Sensing Images , 2019, ISPRS Int. J. Geo Inf..

[66]  Yuan Yan Tang,et al.  Maximum Likelihood Estimation-Based Joint Sparse Representation for the Classification of Hyperspectral Remote Sensing Images , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[67]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[68]  Yuanzhi Li,et al.  What Can ResNet Learn Efficiently, Going Beyond Kernels? , 2019, NeurIPS.

[69]  Xun Wang,et al.  Eleventh International Conference on Graphics and Image Processing (ICGIP 2019) , 2019 .

[70]  Quoc V. Le,et al.  NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[71]  Di Wu,et al.  Recommendation system using feature extraction and pattern recognition in clinical care systems , 2018, Enterp. Inf. Syst..

[72]  Oliver Wang,et al.  MSG-GAN: Multi-Scale Gradient GAN for Stable Image Synthesis , 2019, ArXiv.

[73]  Ying Chen,et al.  M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network , 2018, AAAI.

[74]  Han Zhang,et al.  Self-Attention Generative Adversarial Networks , 2018, ICML.

[75]  Dong Xu,et al.  Learning Rotation-Invariant and Fisher Discriminative Convolutional Neural Networks for Object Detection , 2019, IEEE Transactions on Image Processing.

[76]  Yongqiang Zhang,et al.  SOD-MTGAN: Small Object Detection via Multi-Task Generative Adversarial Network , 2018, ECCV.

[77]  Ye Zhang,et al.  A light and faster regional convolutional neural network for object detection in optical remote sensing images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[78]  Ye Zhang,et al.  A Comparison: Different DCNN Models for Intelligent Object Detection in Remote Sensing Images , 2018, Neural Processing Letters.

[79]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[80]  Saifuddin Hitawala,et al.  Evaluating ResNeXt Model Architecture for Image Classification , 2018, ArXiv.

[81]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[82]  Yanfei Zhong,et al.  Multi-class geospatial object detection based on a position-sensitive balancing framework for high spatial resolution remote sensing imagery , 2018 .

[83]  Shu Liu,et al.  Path Aggregation Network for Instance Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[84]  Shifeng Zhang,et al.  Single-Shot Refinement Neural Network for Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[85]  Wenqian Huang,et al.  Detection of early bruises on peaches (Amygdalus persica L.) using hyperspectral imaging coupled with improved watershed segmentation algorithm , 2018 .

[86]  Fuqiang Zhou,et al.  FSSD: Feature Fusion Single Shot Multibox Detector , 2017, ArXiv.

[87]  Zhiqiang Shen,et al.  DSOD: Learning Deeply Supervised Object Detectors from Scratch , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[88]  Wuzhen Shi,et al.  Single image super-resolution with dilated convolution based multi-scale information learning inception module , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[89]  Fuchun Sun,et al.  RON: Reverse Connection with Objectness Prior Networks for Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[90]  Nojun Kwak,et al.  Enhancement of SSD by concatenating feature maps for object detection , 2017, BMVC.

[91]  Abhinav Gupta,et al.  A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[92]  Zulin Wang,et al.  Road Structure Refined CNN for Road Extraction in Aerial Image , 2017, IEEE Geoscience and Remote Sensing Letters.

[93]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[94]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[95]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[96]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[97]  Jian Dong,et al.  Attentive Contexts for Object Detection , 2016, IEEE Transactions on Multimedia.

[98]  Yang Wang,et al.  Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation , 2016, ISVC.

[99]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[100]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[101]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[102]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[103]  De-Tian Huang,et al.  An novel random forests and its application to the classification of mangroves remote sensing image , 2016, Multimedia Tools and Applications.

[104]  Tong Zhang,et al.  Deep Learning Based Feature Selection for Remote Sensing Scene Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[105]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[106]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[107]  Wang Xiaorui,et al.  High-accuracy infrared simulation model based on establishing the linear relationship between the outputs of different infrared imaging systems , 2015 .

[108]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[109]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[110]  Haipeng Wang,et al.  SAR target recognition based on deep learning , 2014, 2014 International Conference on Data Science and Advanced Analytics (DSAA).

[111]  C. Lawrence Zitnick,et al.  Edge Boxes: Locating Object Proposals from Edges , 2014, ECCV.

[112]  Krista A. Ehinger,et al.  SUN Database: Exploring a Large Collection of Scene Categories , 2014, International Journal of Computer Vision.

[113]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[114]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[115]  Dumitru Erhan,et al.  Scalable Object Detection Using Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[116]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[117]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[118]  Raúl Siche,et al.  Review: computer vision applied to the inspection and quality control of fruits and vegetables , 2013 .

[119]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[120]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[121]  Hervé Delbarre,et al.  Fast changes in chemical composition and size distribution of fine particles during the near-field transport of industrial plumes. , 2012, The Science of the total environment.

[122]  Kun Tan,et al.  A novel binary tree support vector machine for hyperspectral remote sensing image classification , 2012 .

[123]  M. A. Khan,et al.  Machine vision system: a tool for quality inspection of food and agricultural products , 2012, Journal of Food Science and Technology.

[124]  J. Gómez-Sanchís,et al.  Advances in Machine Vision Applications for Automatic Inspection and Quality Evaluation of Fruits and Vegetables , 2011 .

[125]  John DeNero,et al.  L1 and L2 regularization for multiclass hinge loss models , 2011, MLSLP.

[126]  Geoffrey E. Hinton,et al.  Learning to Detect Roads in High-Resolution Aerial Images , 2010, ECCV.

[127]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[128]  Andrew Zisserman,et al.  Multiple kernels for object detection , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[129]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[130]  B. Schiele,et al.  Pedestrian detection: A benchmark , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[131]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[132]  Mohammad Khaja Nazeeruddin,et al.  Fabrication of screen‐printing pastes from TiO2 powders for dye‐sensitised solar cells , 2007 .

[133]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[134]  F. Ma,et al.  Exposure of the Shaded Side of Apple Fruit to Full Sun Leads to Up-regulation of Both the Xanthophyll Cycle and the Ascorbate-glutathione Cycle , 2004 .

[135]  F. Ma,et al.  Exposure of the shaded side of apple fruit to full sun leads to up-regulation of both the xanthophyll cycle and the ascorbate–glutathione cycle , 2004 .

[136]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[137]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[138]  Yang Tao,et al.  Application of machine vision in inspecting stem and shape of fruits , 2000, SPIE Optics East.

[139]  David C. Slaughter,et al.  Discriminating Fruit for Robotic Harvest Using Color in Natural Outdoor Scenes , 1989 .