Efficient Retrieval of Massive Ocean Remote Sensing Images via a Cloud-Based Mean-Shift Algorithm

The rapid development of remote sensing (RS) technology has resulted in the proliferation of high-resolution images. There are challenges involved in not only storing large volumes of RS images but also in rapidly retrieving the images for ocean disaster analysis such as for storm surges and typhoon warnings. In this paper, we present an efficient retrieval of massive ocean RS images via a Cloud-based mean-shift algorithm. Distributed construction method via the pyramid model is proposed based on the maximum hierarchical layer algorithm and used to realize efficient storage structure of RS images on the Cloud platform. We achieve high-performance processing of massive RS images in the Hadoop system. Based on the pyramid Hadoop distributed file system (HDFS) storage method, an improved mean-shift algorithm for RS image retrieval is presented by fusion with the canopy algorithm via Hadoop MapReduce programming. The results show that the new method can achieve better performance for data storage than HDFS alone and WebGIS-based HDFS. Speedup and scaleup are very close to linear changes with an increase of RS images, which proves that image retrieval using our method is efficient.

[1]  Dan Xu,et al.  The research of remote sensing image segmentation and release which are based on Tile Map Service , 2012, 2012 International Symposium on Geomatics for Integrated Water Resource Management.

[2]  Vo Thi Ngoc Chau,et al.  Fuzzy C-means for english sentiment classification in a distributed system , 2017, Applied Intelligence.

[3]  Ralf Lämmel,et al.  Google's MapReduce programming model - Revisited , 2007, Sci. Comput. Program..

[4]  Chunyong Yin,et al.  Parallel implementing improved k-means applied for image retrieval and anomaly detection , 2017, Multimedia Tools and Applications.

[5]  Xin Pan,et al.  A remote sensing image cloud processing system based on Hadoop , 2012, 2012 IEEE 2nd International Conference on Cloud Computing and Intelligence Systems.

[6]  Jan-Ming Ho,et al.  A novel content based image retrieval system using K-means with feature extraction , 2012, 2012 International Conference on Systems and Informatics (ICSAI2012).

[7]  Mayuri A. Mehta,et al.  A novel approach for efficient handling of small files in HDFS , 2015, 2015 IEEE International Advance Computing Conference (IACC).

[8]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[9]  Roberto Giachetta,et al.  A framework for processing large scale geospatial and remote sensing data in MapReduce environment , 2015, Comput. Graph..

[10]  Roshan Rajak,et al.  High Resolution Satellite Image Processing Using Hadoop Framework , 2015, 2015 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM).

[11]  Antonio J. Plaza,et al.  Recent Developments in High Performance Computing for Remote Sensing: A Review , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[12]  Chun-Chieh Chen,et al.  Fast K-means algorithm based on a level histogram for image retrieval , 2014, Expert Syst. Appl..

[13]  Feng-Cheng Lin,et al.  Storage and processing of massive remote sensing images using a novel cloud computing platform , 2013 .

[14]  Joseph Issa.  Performance characterization and analysis for Hadoop K-means iteration , 2016, Journal of Cloud Computing.

[15]  Zeng Zhiyong.  Novel Image Retrieval Method of Improved K-means Clustering Algorithm , 2013 .

[16]  Bo Li,et al.  Parallel K-Means Clustering of Remote Sensing Images Based on MapReduce , 2010, WISM.

[17]  Simone A. Ludwig.  MapReduce-based fuzzy c-means clustering algorithm: implementation and scalability , 2015, Int. J. Mach. Learn. Cybern..

[18]  Ermias Beyene Tesfamariam.  Distributed processing of large remote sensing images using MapReduce - A case of Edge Detection , 2011 .

[19]  Li Jun.  Method of Building Image Pyramid Model Based on GDAL and NetCDF , 2012 .

[20]  Mohamed H. Almeer.  Cloud Hadoop Map Reduce For Remote Sensing Image Analysis , 2012 .

[21]  Andrew McCallum,et al.  Efficient clustering of high-dimensional data sets with application to reference matching , 2000, KDD '00.

[22]  Rajashree Shettar,et al.  Multidimensional Canopy Clustering on Iterative MapReduce Framework Using Elefig Tool , 2015 .

[23]  Licheng Jiao,et al.  Parallel fast Global K-Means algorithm for synthetic aperture radar image change detection using OpenCL , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[24]  V. S. Murthy,et al.  Content Based Image Retrieval using Hierarchical and K-Means Clustering Techniques , 2010 .

[25]  Mohamed H. Almeer.  Hadoop Mapreduce for Remote Sensing Image Analysis , 2012 .

[26]  Sitalakshmi Venkatraman,et al.  MapReduce neural network framework for efficient content based image retrieval from large datasets in the cloud , 2012, 2012 12th International Conference on Hybrid Intelligent Systems (HIS).

[27]  Xubin He,et al.  Implementing WebGIS on Hadoop: A case study of improving small file I/O performance on HDFS , 2009, 2009 IEEE International Conference on Cluster Computing and Workshops.

[28]  Xiaohong Jiang,et al.  vHadoop: A Scalable Hadoop Virtual Cluster Platform for MapReduce-Based Parallel Machine Learning with Performance Consideration , 2012, 2012 IEEE International Conference on Cluster Computing Workshops.

[29]  Huang Dongmei,et al.  AN EFFECTIVE DETECTION OF SATELLITE IMAGES VIA K-MEANS CLUSTERING ON HADOOP SYSTEM , 2017 .

[30]  Yao Zheng,et al.  QDFS: A quality-aware distributed file storage service based on HDFS , 2011, 2011 IEEE International Conference on Computer Science and Automation Engineering.