Satellite and Scene Image Classification Based on Transfer Learning and Fine Tuning of ResNet50

Image classification has gained lot of attention due to its application in different computer vision tasks such as remote sensing, scene analysis, surveillance, object detection, and image retrieval. The primary goal of image classification is to assign the class labels to images according to the image contents. The applications of image classification and image analysis in remote sensing are important as they are used in various applied domains such as military and civil fields. Earlier approaches for remote sensing images and scene analysis are based on low-level feature representations such as color- and texture-based features. Vector of Locally Aggregated Descriptors (VLAD) and orderless Bag-of-Features (BoF) representations are the examples of mid-level approaches for remote sensing image classification. Recent trends for remote sensing and scene classification are focused on the use of Convolutional Neural Network (CNN). Keeping in view the success of CNN models, in this research, we aim to fine-tune ResNet50 by using network surgery and creation of network head along with the fine-tuning of hyperparameters. The learning of hyperparameters is tuned by using a linear decay learning rate scheduler known as piecewise scheduler. To tune the optimizer hyperparameter, Stochastic Gradient Descent with Momentum (SGDM) is used with the usage of weight learn and bias learn rate factor. Experiments and analysis are conducted on five different datasets, that is, UC Merced Land Use Dataset (UCM), RSSCN (the remote sensing scene classification image dataset), SIRI-WHU, Corel-1K, and Corel-1.5K. The analysis and competitive results exemplify that our proposed image classification-based model can classify the images in a more effective and efficient manner as compared to the state-of-the-art research.

[1]  Leslie N. Smith,et al.  Cyclical Learning Rates for Training Neural Networks , 2015, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[2]  Padmashree Desai,et al.  Impact of Multi-Feature Extraction on Image Retrieval and classification Using Machine Learning Technique , 2021, SN Computer Science.

[3]  Stefano Ermon,et al.  Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping , 2015, AAAI.

[4]  Jing Wang,et al.  Multiple kernel-based multi-instance learning algorithm for image classification , 2014, J. Vis. Commun. Image Represent..

[5]  Bushra Zafar,et al.  Short search space and synthesized-reference re-ranking for face image retrieval , 2020 .

[6]  Xueliang Zhang,et al.  Deep learning in remote sensing applications: A meta-analysis and review , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[7]  Muhammad Tariq Mahmood,et al.  Fabric Defect Detection Using Computer Vision Techniques: A Comprehensive Review , 2020 .

[8]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Muhammad Tariq Mahmood,et al.  Modeling global geometric spatial information for rotation invariant classification of satellite images , 2019, PloS one.

[10]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[11]  Subarna Shakya,et al.  Analysis of Artificial Intelligence based Image Classification Techniques , 2020, Journal of Innovative Image Processing.

[12]  Nouman Ali,et al.  Image Classification Based on Mid-Level Feature Fusion , 2019, 2019 15th International Conference on Emerging Technologies (ICET).

[13]  Gui-Song Xia,et al.  Dirichlet-Derived Multiple Topic Scene Classification Model for High Spatial Resolution Remote Sensing Imagery , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[16]  Shanjun Mao,et al.  Spectral–spatial classification of hyperspectral images using deep convolutional neural networks , 2015 .

[17]  Gong Cheng,et al.  Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[18]  Doreswamy,et al.  Feature selection approach using ensemble learning for network anomaly detection , 2020, CAAI Trans. Intell. Technol..

[19]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[20]  Biswajeet Pradhan,et al.  Deep Learning Approaches Applied to Remote Sensing Datasets for Road Extraction: A State-Of-The-Art Review , 2020, Remote. Sens..

[21]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[22]  Tatjana Atanasova-Pacemska,et al.  Aerial Scene Classification through Fine-Tuning with Adaptive Learning Rates and Label Smoothing , 2020, Applied Sciences.

[23]  Muhammad Usman,et al.  Deep Learning in Age-invariant Face Recognition: A Comparative Study , 2020, Comput. J..

[24]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[25]  Bushra Zafar,et al.  Data Augmentation-Assisted Makeup-Invariant Face Recognition , 2018, Mathematical Problems in Engineering.

[26]  Nouman Ali,et al.  Home Security and Automation Based on Internet of Things: A Comprehensive Review , 2020, IOP Conference Series: Materials Science and Engineering.

[27]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[28]  Jun Li,et al.  Recent Advances on Spectral–Spatial Hyperspectral Image Classification: An Overview and New Guidelines , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Yang Yang,et al.  Multi-robot path planning based on a deep reinforcement learning DQN algorithm , 2020, CAAI Trans. Intell. Technol..

[30]  Pawan Kumar,et al.  An Extreme Learning Machine-Relevance Feedback Framework for Enhancing the Accuracy of a Hybrid Image Retrieval System , 2020, Int. J. Interact. Multim. Artif. Intell..

[31]  Hui Xiong,et al.  A Comprehensive Survey on Transfer Learning , 2019, Proceedings of the IEEE.

[32]  Saadat Hanif Dar,et al.  Detection of glaucoma using retinal fundus images: A comprehensive review. , 2021, Mathematical biosciences and engineering : MBE.

[33]  Tong Zhang,et al.  Deep Learning Based Feature Selection for Remote Sensing Scene Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[34]  Haokui Zhang,et al.  Deep learning for remote sensing image classification: A survey , 2018, WIREs Data Mining Knowl. Discov..

[35]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Li Zhuo,et al.  Hyperspectral remote sensing image retrieval system using spectral and texture features. , 2017, Applied optics.

[37]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Jun Li,et al.  Advanced Spectral Classifiers for Hyperspectral Images: A review , 2017, IEEE Geoscience and Remote Sensing Magazine.

[39]  Zhen Xu,et al.  Learning an Adaptive Learning Rate Schedule , 2019, ArXiv.

[40]  Xinyue Zhang,et al.  Scene Classification of High-Resolution Remotely Sensed Image Based on ResNet , 2019, Journal of Geovisualization and Spatial Analysis.

[41]  Neha Sharma,et al.  An Analysis Of Convolutional Neural Networks For Image Classification , 2018 .

[42]  Jieun Park,et al.  A Novel Learning Rate Schedule in Optimization for Neural Networks and It's Convergence , 2020, Symmetry.

[43]  Awais Ahmad,et al.  Intelligent image classification-based on spatial weighted histograms of concentric circles , 2018, Comput. Sci. Inf. Syst..

[44]  Josef Strobl,et al.  What’s wrong with pixels? Some recent developments interfacing remote sensing and GIS , 2001 .

[45]  Shan Liu,et al.  Neural saliency algorithm guide bi-directional visual perception style transfer , 2020, CAAI Trans. Intell. Technol..

[46]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[47]  James Ze Wang,et al.  Real-time computerized annotation of pictures. , 2008, IEEE transactions on pattern analysis and machine intelligence.

[48]  Bushra Zafar,et al.  A Hybrid Geometric Spatial Image Representation for scene classification , 2018, PloS one.

[49]  Bidyut Baran Chaudhuri,et al.  diffGrad: An Optimization Method for Convolutional Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[50]  Licheng Jiao,et al.  Feature integration of EODH and Color-SIFT: Application to image retrieval based on codebook , 2014, Signal Process. Image Commun..

[51]  Mohamed F. Tolba,et al.  Comparison of CNNs for Remote Sensing Scene Classification , 2018, 2018 13th International Conference on Computer Engineering and Systems (ICCES).

[52]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[53]  Gang Liu,et al.  A Hierarchical Scheme of Multiple Feature Fusion for High-Resolution Satellite Scene Categorization , 2013, ICVS.

[54]  Wenzhong Guo,et al.  Land-Use Classification via Extreme Learning Classifier Based on Deep Convolutional Features , 2017, IEEE Geoscience and Remote Sensing Letters.

[55]  Yuan Yan Tang,et al.  High-Order Distance-Based Multiview Stochastic Learning in Image Classification , 2014, IEEE Transactions on Cybernetics.

[56]  Ying Liu,et al.  Criminal Investigation Image Classification Based on Spatial CNN Features and ELM , 2018, 2018 10th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC).

[57]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Adam Glowacz,et al.  Integration of Discrete Wavelet Transform, DBSCAN, and Classifiers for Efficient Content Based Image Retrieval , 2020 .

[59]  Md. Rashid Mahmood,et al.  Color Object Detection Based Image Retrieval Using ROI Segmentation with Multi-Feature Method , 2020, Wireless Personal Communications.

[60]  Rehan Ashraf,et al.  A Novel Discriminating and Relative Global Spatial Image Representation with Applications in CBIR , 2018, Applied Sciences.

[61]  Shiv Ram Dubey,et al.  Rotation and scale invariant hybrid image descriptor and retrieval , 2015, Comput. Electr. Eng..

[62]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[63]  Xiaolei Zhao,et al.  Residual Dense Network Based on Channel-Spatial Attention for the Scene Classification of a High-Resolution Remote Sensing Image , 2020, Remote. Sens..

[64]  Jameel Ahmed,et al.  Content-Based Image Retrieval and Feature Extraction: A Comprehensive Review , 2019, Mathematical Problems in Engineering.

[65]  Bushra Zafar,et al.  Facial asymmetry-based feature extraction for different applications: a review complemented by new advances , 2021, Artif. Intell. Rev..