Weight Initialization Techniques for Deep Learning Algorithms in Remote Sensing: Recent Trends and Future Perspectives

During the last decade, several research works have focused on providing novel deep learning methods in many application fields. However, few of them have investigated the weight initialization process for deep learning, although its importance is revealed in improving deep learning performance. This can be justified by the technical difficulties in proposing new techniques for this promising research field. In this paper, a survey related to weight initialization techniques for deep algorithms in remote sensing is conducted. This survey will help practitioners to drive further research in this promising field. To the best of our knowledge, this paper constitutes the first survey focusing on weight initialization for deep learning models.

[1]  Zhenfeng Shao,et al.  Remote Sensing Image Fusion With Deep Convolutional Neural Network , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Xiao Chen,et al.  Infrastructure Quality Assessment in Africa using Satellite Imagery and Deep Learning , 2018, KDD.

[3]  Amir Hussain,et al.  A novel decision support system for the interpretation of remote sensing big data , 2018, Earth Science Informatics.

[4]  Emrullah Acar,et al.  Detection of unregistered electric distribution transformers in agricultural fields with the aid of Sentinel-1 SAR images by machine learning approaches , 2020, Comput. Electron. Agric..

[5]  Imed Romdhani,et al.  Big data and IoT-based applications in smart environments: A systematic review , 2021, Comput. Sci. Rev..

[6]  Junjie Liu,et al.  Single-Image Super-Resolution for Remote Sensing Data Using Deep Residual-Learning Neural Network , 2017, ICONIP.

[7]  Wadii Boulila,et al.  Leveraging Deep Learning and IoT big data analytics to support the smart cities development: Review and future directions , 2020, Comput. Sci. Rev..

[8]  Celso André R. de Sousa,et al.  An overview on weight initialization methods for feedforward neural networks , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[9]  Wadii Boulila,et al.  A top-down approach for semantic segmentation of big remote sensing images , 2019, Earth Science Informatics.

[10]  Chris Kreucher,et al.  Modern approaches in deep learning for SAR ATR , 2016, SPIE Defense + Security.

[11]  Farhan Ullah,et al.  Clone detection in 5G-enabled social IoT system using graph semantics and deep learning model , 2021, International Journal of Machine Learning and Cybernetics.

[12]  Zhiqiang Li,et al.  Road Extraction from Remote Sensing Images Using Parallel Softplus Networks , 2020, Journal of the Indian Society of Remote Sensing.

[13]  Ying Wang,et al.  Deep Learning Ensemble for Hyperspectral Image Classification , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[14]  Simon Fong,et al.  How meta-heuristic algorithms contribute to deep learning in the hype of big data analytics , 2018 .

[15]  Guilherme Perin,et al.  A Comparison of Weight Initializers in Deep Learning-based Side-channel Analysis , 2020, IACR Cryptol. ePrint Arch..

[16]  Haokui Zhang,et al.  Deep learning for remote sensing image classification: A survey , 2018, WIREs Data Mining Knowl. Discov..

[17]  Yun Sing Koh,et al.  Using Supervised Pretraining to Improve Generalization of Neural Networks on Binary Classification Problems , 2018, ECML/PKDD.

[18]  Peng Zhang,et al.  Pixel-Wise PolSAR Image Classification via a Novel Complex-Valued Deep Fully Convolutional Network , 2019, Remote. Sens..

[19]  Alan C. Bovik,et al.  Surface Water Mapping by Deep Learning , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[20]  Pingbo Tang,et al.  Augmenting a deep-learning algorithm with canal inspection knowledge for reliable water leak detection from multispectral satellite images , 2020, Adv. Eng. Informatics.

[21]  Xiao Xiang Zhu,et al.  Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional LSTM network for multi-label aerial image classification , 2018, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[22]  Liping Di,et al.  Using long short-term memory recurrent neural network in land cover classification on Landsat and Cropland data layer time series , 2018, International Journal of Remote Sensing.

[23]  Wadii Boulila,et al.  A comparison of big remote sensing data processing with Hadoop MapReduce and Spark , 2018, 2018 4th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP).

[24]  Fenghua Huang,et al.  Automatic building change image quality assessment in high resolution remote sensing based on deep learning , 2019, J. Vis. Commun. Image Represent..

[25]  Javier Lacasta,et al.  End-to-end sequence labeling via deep learning for automatic extraction of agricultural regulations , 2019, Comput. Electron. Agric..

[26]  Ali Kashif Bashir,et al.  Chaos‐based privacy preserving vehicle safety protocol for 5G Connected Autonomous Vehicle networks , 2020, Trans. Emerg. Telecommun. Technol..

[27]  Rui Camacho,et al.  Using autoencoders as a weight initialization method on deep neural networks for disease detection , 2020, BMC Medical Informatics and Decision Making.

[28]  Redouane Lguensat,et al.  EddyNet: A Deep Neural Network For Pixel-Wise Classification of Oceanic Eddies , 2017, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[29]  Michael Kampffmeyer,et al.  Semantic Segmentation of Small Objects and Modeling of Uncertainty in Urban Remote Sensing Images Using Deep Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Wadii Boulila,et al.  Big Data: Concepts, Challenges and Applications , 2015, ICCCI.

[31]  Yugen Yi,et al.  Toward Efficient Image Recognition in Sensor-Based IoT: A Weight Initialization Optimizing Method for CNN Based on RGB Influence Proportion , 2020, Sensors.

[32]  Xin Zhao,et al.  Use of Unmanned Aerial Vehicle Imagery and Deep Learning UNet to Extract Rice Lodging , 2019, Sensors.

[33]  Ryan M. Gibson,et al.  WiFreeze: Multiresolution Scalograms for Freezing of Gait Detection in Parkinson’s Leveraging 5G Spectrum with Deep Learning , 2019, Electronics.

[34]  Xiaorui Ma,et al.  Semisupervised classification for hyperspectral image based on multi-decision labeling and deep feature learning , 2016 .

[35]  Fuad A. Ghaleb,et al.  RS-DCNN: A novel distributed convolutional-neural-networks based-approach for big remote-sensing image classification , 2021, Comput. Electron. Agric..

[36]  Sina Keller,et al.  Deep Learning for Land Cover Change Detection , 2020, Remote. Sens..