Metaheuristic pansharpening based on symbiotic organisms search optimization

Abstract This study proposed a metaheuristic pansharpening (MP) method, which performs in a Synthetic Variable Ratio (SVR)-like manner. The proposed method introduced the Symbiotic Organisms Search (SOS) algorithm, an advanced nature-based optimization algorithm, to estimate a weight for each multispectral (MS) band to achieve the optimum intensity. The SVR pansharpening formula was used as the objective function and the Root Mean Square Error (RMSE) metric was used as the fitness function of the SOS algorithm to optimize the intensity. The spectral and spatial quality of the results of the MP method were qualitatively and quantitatively compared against those of 15 widely-used pansharpening methods in 5 test sites in Turkey with different land cover features. The experiments aimed to spatially enhance WorldView-2 MS images by using WorldView-2 panchromatic (PAN) bands and a UAV-derived PAN orthophoto. It was also aimed to sharpen IKONOS MS images by using a QuickBird pansharpened image and an IKONOS PAN band. The MATLAB software was used to implement the proposed method and to compute the spatial and spectral quality metrics. The spatial quality of each pansharpened image was evaluated at full-scale, whereas the spectral quality of each pansharpened image was evaluated at both full-scale and reduced-scale. A scoring strategy based on giving a performance score with respect to the spatial and spectral quality metrics was used to ensure a fair comparison among the pansharpening methods used. The results demonstrated that, out of 16 pansharpening methods, the MP method achieved the highest overall spectral and spatial quality scores of 15.5 and 15.5, respectively. The proposed method was found to perform successfully with both singlesensor and multisensor input images. It was also concluded that the proposed method is able to deal with high spatial resolution ratio between the input images.

[1]  Liangpei Zhang,et al.  A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[3]  Chandi Witharana,et al.  Benchmarking of data fusion algorithms in support of earth observation based Antarctic wildlife monitoring , 2016 .

[4]  Damian Wierzbicki,et al.  Image data fusion for flood plain mapping , 2014 .

[5]  Min-Yuan Cheng,et al.  A novel Multiple Objective Symbiotic Organisms Search (MOSOS) for time-cost-labor utilization tradeoff problem , 2016, Knowl. Based Syst..

[6]  Hassan Ghassemian,et al.  A review of remote sensing image fusion methods , 2016, Inf. Fusion.

[7]  Jocelyn Chanussot,et al.  Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[9]  Fabrice Muller,et al.  Adaptive intensity matching filters: a new tool for multi-resolution data fusion , 1998 .

[10]  Luciano Alparone,et al.  Full scale assessment of pansharpening methods and data products , 2014, Remote Sensing.

[11]  Shutao Li,et al.  Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[12]  A. Kallel,et al.  Iterative scheme for MS image pansharpening based on the combination of multi-resolution decompositions , 2016 .

[13]  Davide Cozzolino,et al.  Pansharpening by Convolutional Neural Networks , 2016, Remote. Sens..

[14]  W. A. Hallada,et al.  Image sharpening for mixed spatial and spectral resolution satellite systems , 1983 .

[15]  Xavier Otazu,et al.  Multiresolution-based image fusion with additive wavelet decomposition , 1999, IEEE Trans. Geosci. Remote. Sens..

[16]  Xin-She Yang,et al.  Firefly Algorithms for Multimodal Optimization , 2009, SAGA.

[17]  Hongyi Liu,et al.  A New Pan-Sharpening Method With Deep Neural Networks , 2015, IEEE Geoscience and Remote Sensing Letters.

[18]  Jocelyn Chanussot,et al.  Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening , 2014, IEEE Geoscience and Remote Sensing Letters.

[19]  Shutao Li,et al.  A New Pan-Sharpening Method Using a Compressed Sensing Technique , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Andrea Garzelli,et al.  Hypercomplex Quality Assessment of Multi/Hyperspectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[21]  D. Pham,et al.  THE BEES ALGORITHM, A NOVEL TOOL FOR COMPLEX OPTIMISATION PROBLEMS , 2006 .

[22]  Xavier Otazu,et al.  Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Bin Chen,et al.  Multi-source remotely sensed data fusion for improving land cover classification , 2017 .

[24]  Yaonan Wang,et al.  Using the discrete wavelet frame transform to merge Landsat TM and SPOT panchromatic images , 2002, Inf. Fusion.

[25]  Aggelos K. Katsaggelos,et al.  A survey of classical methods and new trends in pansharpening of multispectral images , 2011, EURASIP J. Adv. Signal Process..

[26]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network , 2018, Remote. Sens..

[27]  Hairong Qi,et al.  DOES multispectral / hyperspectral pansharpening improve the performance of anomaly detection? , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[28]  Vivekananda Mukherjee,et al.  A novel symbiotic organisms search algorithm for optimal power flow of power system with FACTS devices , 2016 .

[29]  Johannes R. Sveinsson,et al.  Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[30]  Jocelyn Chanussot,et al.  Pansharpening Based on Deconvolution for Multiband Filter Estimation , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[31]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation: A spectral preserve image fusion technique for improving spatial details , 2000 .

[32]  Zhi Li,et al.  The PAN and MS Image Pansharpening Algorithm Based on Adaptive Neural Network and Sparse Representation in the NSST Domain , 2019, IEEE Access.

[33]  Qingshan Liu,et al.  Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[34]  Mohamed Elhoseny,et al.  Multi-spectral and panchromatic image fusion approach using stationary wavelet transform and swarm flower pollination optimization for remote sensing applications , 2018, Future Gener. Comput. Syst..

[35]  Robert A. Schowengerdt,et al.  Reconstruction of multispatial, multispectral image data using spatial frequency content , 1980 .

[36]  Jie Shan,et al.  A genetic algorithm solution to the gram-schmidt image fusion , 2019, International Journal of Remote Sensing.

[37]  Marguerite Madden,et al.  Assessing Optimal Image Fusion Methods for Very High Spatial Resolution Satellite Images to Support Coastal Monitoring , 2012 .

[38]  Jocelyn Chanussot,et al.  Pansharpening Quality Assessment Using the Modulation Transfer Functions of Instruments , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Ardeshir Bahreininejad,et al.  Water cycle algorithm - A novel metaheuristic optimization method for solving constrained engineering optimization problems , 2012 .

[40]  Jocelyn Chanussot,et al.  Pansharpening Based on Semiblind Deconvolution , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Yang Xu,et al.  Pan-Sharpening Based on Multilevel Coupled Deep Network , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[42]  Rajesh Gogineni,et al.  Sparsity inspired pan-sharpening technique using multi-scale learned dictionary , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[43]  Luciano Alparone,et al.  A global quality measurement of pan-sharpened multispectral imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[44]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[45]  Jocelyn Chanussot,et al.  A Pansharpening Method Based on the Sparse Representation of Injected Details , 2015, IEEE Geoscience and Remote Sensing Letters.

[46]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[47]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[48]  Ardeshir Bahreininejad,et al.  Mine blast algorithm for optimization of truss structures with discrete variables , 2012 .

[49]  Ibrahim Eksin,et al.  A new optimization method: Big Bang-Big Crunch , 2006, Adv. Eng. Softw..

[50]  Jianya Gong,et al.  Multivariate statistical analysis of measures for assessing the quality of image fusion , 2010 .

[51]  Ardhendu Behera,et al.  Remote sensing image fusion via compressive sensing , 2019 .

[52]  Jun Liu,et al.  Human visual system consistent quality assessment for remote sensing image fusion , 2015 .

[53]  Liangpei Zhang,et al.  Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[54]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[55]  Richard Bamler,et al.  A Sparse Image Fusion Algorithm With Application to Pan-Sharpening , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[56]  F. Nencini,et al.  Fusion of Panchromatic and Multispectral Images by Genetic Algorithms , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[57]  Mohiy Hadhoud,et al.  Image Super-Resolution and Applications , 2012 .

[58]  Xin-She Yang,et al.  A New Metaheuristic Bat-Inspired Algorithm , 2010, NICSO.

[59]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[60]  Davide Cozzolino,et al.  Target-Adaptive CNN-Based Pansharpening , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[61]  Yun Zhang,et al.  Wavelet based image fusion techniques — An introduction, review and comparison , 2007 .

[62]  Wenzhong Shi,et al.  Area-to-point regression kriging for pan-sharpening , 2016 .

[63]  Li Yunsong,et al.  Subspace selection for hyperspectral pansharpening using sparse unmixing , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[64]  Xiaofeng Zhang,et al.  Sparse Representation over Shared Coefficients in Multispectral Pansharpening , 2018 .

[65]  Luciano Alparone,et al.  Image fusion—the ARSIS concept and some successful implementation schemes , 2003 .

[66]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[67]  A. Kaveh,et al.  A novel heuristic optimization method: charged system search , 2010 .

[68]  Brian Johnson,et al.  Effects of Pansharpening on Vegetation Indices , 2014, ISPRS Int. J. Geo Inf..

[69]  Xin-She Yang,et al.  Cuckoo Search via Lévy flights , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[70]  J. Schott,et al.  Resolution enhancement of multispectral image data to improve classification accuracy , 1993 .

[71]  Shuyuan Yang,et al.  Fusion of multispectral and panchromatic images via sparse representation and local autoregressive model , 2014, Inf. Fusion.

[72]  Christine Pohl,et al.  Remote Sensing Image Fusion: A Practical Guide , 2016 .

[73]  Zhenfeng Shao,et al.  Remote Sensing Image Fusion With Deep Convolutional Neural Network , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[74]  Thierry Ranchin,et al.  Liu 'Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details' , 2002 .

[75]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[76]  Haitao Yin.  PAN-Guided Cross-Resolution Projection for Local Adaptive Sparse Representation- Based Pansharpening , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[77]  Daniel L. Civco,et al.  Evaluating remote sensing image fusion algorithms for use in humanitarian crisis management , 2012, Remote Sensing.

[78]  Oguz Gungor,et al.  Determining the optimum image fusion method for better interpretation of the surface of the Earth , 2016 .

[79]  Min-Yuan Cheng,et al.  Symbiotic Organisms Search: A new metaheuristic optimization algorithm , 2014 .

[80]  Shuyuan Yang,et al.  Pan-sharpening via deep metric learning , 2018 .

[81]  Oguz Gungor,et al.  Fusion of very high-resolution UAV images with criteria-based image fusion algorithm , 2015, Arabian Journal of Geosciences.

[82]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[83]  Andrea Garzelli,et al.  Interband structure modeling for Pan-sharpening of very high-resolution multispectral images , 2005, Inf. Fusion.

[84]  Rafael García,et al.  Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[85]  Mehdi Mokhtarzade,et al.  A NOVEL IHS-GA FUSION METHOD BASED ON ENHANCEMENT VEGETATED AREA , 2015 .

[86]  Manfred Ehlers,et al.  FFT-enhanced IHS transform method for fusing high-resolution satellite images , 2007 .

[87]  Marguerite Madden,et al.  Comparison of Global- and Local-scale Pansharpening for Rapid Assessment of Humanitarian Emergencies , 2011 .

[88]  Peyman Kabiri,et al.  New intensity-hue-saturation pan-sharpening method based on texture analysis and genetic algorithm-adaption , 2014 .

[89]  H. Ghassemian,et al.  A multi-objective component-substitution-based pansharpening , 2017, 2017 3rd International Conference on Pattern Recognition and Image Analysis (IPRIA).

[90]  Maryam Imani.  Band Dependent Spatial Details Injection Based on Collaborative Representation for Pansharpening , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[91]  Y. Zhang,et al.  A new merging method and its spectral and spatial effects , 1999 .

[92]  Andrea Garzelli,et al.  PAN‐sharpening of very high resolution multispectral images using genetic algorithms , 2006 .

[93]  Wei Huang,et al.  Fusion of satellite images in urban area: Assessing the quality of resulting images , 2010, 2010 18th International Conference on Geoinformatics.

[94]  Karim Faez,et al.  A new pan-sharpening method using multiobjective particle swarm optimization and the shiftable contourlet transform , 2011 .

[95]  Shutao Li,et al.  Deep Hyperspectral Image Sharpening , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[96]  C. Padwick,et al.  WORLDVIEW-2 PAN-SHARPENING , 2010 .

[97]  Richard R. Forster,et al.  Fusion of hyperspectral and radar data using the IHS transformation to enhance urban surface features , 2003 .

[98]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[99]  Jocelyn Chanussot,et al.  Multi-band semiblind deconvolution for pansharpening applications , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[100]  Oguz Gungor.  Multi sensor multi resolution image fusion , 2008 .

[101]  A. Hegde,et al.  A Review of Quality Metrics for Fused Image , 2015 .

[102]  J. Rolet,et al.  Extraction of spectral information from Landsat TM data and merger with SPOT panchromatic imagery: a contribution to the study of geological structures , 1993 .

[103]  Daniel L. Civco,et al.  Evaluation of data fusion and image segmentation in earth observation based rapid mapping workflows , 2014 .

[104]  Bin Chen,et al.  Nearest-neighbor diffusion-based pan-sharpening algorithm for spectral images , 2014, Optical Engineering.

[105]  Gwendoline Blanchet,et al.  A Survey of Pansharpening Methods with A New Band-Decoupled Variational Model , 2016, ArXiv.

[106]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[107]  Ting Liu,et al.  Remote sensing image fusion via wavelet transform and sparse representation , 2015 .

[108]  Hongyu Li,et al.  SR-SIM: A fast and high performance IQA index based on spectral residual , 2012, 2012 19th IEEE International Conference on Image Processing.

[109]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[110]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[111]  Andrea Garzelli,et al.  Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[112]  Zong Woo Geem,et al.  A New Heuristic Optimization Algorithm: Harmony Search , 2001, Simul..

[113]  Haitao Yin,et al.  Sparse representation based pansharpening with details injection model , 2015, Signal Process..

[114]  Lei Cao,et al.  A GIHS-based spectral preservation fusion method for remote sensing images using edge restored spectral modulation , 2014 .

[115]  Wen Dou,et al.  Image Degradation for Quality Assessment of Pan-Sharpening Methods , 2018, Remote. Sens..

[116]  A. Kaveh,et al.  A new optimization method: Dolphin echolocation , 2013, Adv. Eng. Softw..

[117]  Hassan Ghassemian,et al.  Combining the spectral PCA and spatial PCA fusion methods by an optimal filter , 2016, Inf. Fusion.

[118]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.