Towards building a multi‐datacenter infrastructure for massive remote sensing image processing

Earth observation applications are now facing the challenges of managing and processing massive data sets from multiple sources from large‐scale distributed data centers (DCs). To solve this research problem, this paper presents an infrastructure of multiple data centers (MDC) for managing and processing massive remote sensing images. The proposed system is built on both groups of distributed DCs/clusters, which are equipped with DC or cluster resource manager. Access security and information service are introduced to support this architecture of MDC. We collaboratively organized the algorithm, and data belonged to the MDC in the manner of workflow. In practice, we succeeded in working out the concrete problems regarding procedures in processing applications collaboratively and transfer the massive remote sensing dataset fast and with stable cross‐MDC. On the basis of the previously mentioned research work, we will investigate the platform integration of MDC. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  Yan Ma,et al.  An Asynchronous Parallelized and Scalable Image Resampling Algorithm with Parallel I/O , 2009, ICCS.

[2]  J. P. Collins,et al.  Sailing on an Ocean of 0s and 1s , 2010, Science.

[3]  David A. Bader,et al.  High performance computing algorithms for land cover dynamics using remote sensing data , 2000, International Journal of Remote Sensing.

[4]  Mark H. Ellisman,et al.  Data-intensive e-science frontier research , 2003, CACM.

[5]  Darrel L. Williams,et al.  Landsat and Earth Systems Science : Development of terrestrial monitoring , 1997 .

[6]  Jianya Gong,et al.  OpenRS-Cloud: A remote sensing image processing platform based on cloud computing environment , 2010 .

[7]  Albert Y. Zomaya.  Parallel Computing for Bioinformatics and Computational Biology , 2005 .

[8]  Lizhe Wang,et al.  Hybrid modelling and simulation of huge crowd over a hierarchical Grid architecture , 2013, Future Gener. Comput. Syst..

[9]  Gabriele Garzoglio.  A Globally Distributed System for High Energy Physics Computation: Job, Data, and Information Handling for High-Energy Physics , 2009 .

[10]  Yong Zhao,et al.  Chimera: a virtual data system for representing, querying, and automating data derivation , 2002, Proceedings 14th International Conference on Scientific and Statistical Database Management.

[11]  Ian T. Foster,et al.  Virtual workspaces: Achieving quality of service and quality of life in the Grid , 2005, Sci. Program..

[12]  Lizhe Wang,et al.  Organization of CMS benchmarks in VDS Workflow on Virtual Machines , 2007, Sixth International Conference on Grid and Cooperative Computing (GCC 2007).

[13]  Xi He,et al.  Cloud Computing: a Perspective Study , 2010, New Generation Computing.

[14]  Gregor von Laszewski,et al.  Towards building a cloud for scientific applications , 2011, Adv. Eng. Softw..

[15]  Lizhe Wang,et al.  Virtual environments for grid computing , 2008 .

[16]  Liping Di,et al.  Introduction of Grid Computing Application Projects at the NASA Earth Science Technology Office , 2006, GPC.

[17]  John R. G. Townshend,et al.  Global data sets for land applications from the Advanced Very High Resolution Radiometer: an introduction , 1994 .

[18]  Francine Berman,et al.  Overview of the Book: Grid Computing – Making the Global Infrastructure a Reality , 2003 .

[19]  Wei Jie,et al.  Towards supporting multiple virtual private computing environments on computational Grids , 2009, Adv. Eng. Softw..

[20]  Lizhe Wang,et al.  Task Scheduling of Massive Spatial Data Processing across Distributed Data Centers: What's New? , 2011, 2011 IEEE 17th International Conference on Parallel and Distributed Systems.

[21]  Ian T. Foster,et al.  Condor-G: A Computation Management Agent for Multi-Institutional Grids , 2004, Cluster Computing.

[22]  Lizhe Wang,et al.  Virtual workflow system for distributed collaborative scientific applications on Grids , 2011, Comput. Electr. Eng..

[23]  Ian T. Foster,et al.  The Globus Replica Location Service: Design and Experience , 2009, IEEE Transactions on Parallel and Distributed Systems.