Workload and task management of Grid-enabled quantitative aerosol retrieval from remotely sensed data

As the quality and accuracy of remote sensing instruments improve, the ability to quickly process remotely sensed data is in increasing demand. Quantitative retrieval of aerosol properties from remotely sensed data is a data-intensive scientific application, where the complexities of processing, modeling and analyzing large volumes of remotely sensed data sets have significantly increased computation and data demands. While Grid computing has been a prominent technique to tackle computational issues, little work has been done on making Grid computing adapted to remote sensing applications. In this paper, we intended to demonstrate the usage of Grid computing for quantitative remote sensing retrieval applications. A workload estimation and task partition algorithm was developed, and it executes a generic remote sensing algorithm in parallel over partitioned datasets, which is embedded in a middleware framework for remote sensing retrieval named the Remote Sensing Information Service Grid Node (RSIN). A case study shows that significant improvement of system performance can be achieved with this implementation. It also gives a perspective on the potential of applying Grid computing practices to remote sensing problems.

[1]  Antonio J. Plaza,et al.  Heterogeneous Parallel Computing in Remote Sensing Applications: Current Trends and Future Perspectives , 2006, 2006 IEEE International Conference on Cluster Computing.

[2]  Antonio J. Plaza,et al.  FPGA-Based Hyperspectral Data Compression Using Spectral Unmixing and the Pixel Purity Index Algorithm , 2006, International Conference on Computational Science.

[3]  Keith C. Clarke,et al.  Geocomputation's future at the extremes: high performance computing and nanoclients , 2003, Parallel Comput..

[4]  Ying Wang,et al.  Quantitative Retrieval of Geophysical Parameters Using Satellite Data , 2008, Computer.

[5]  A. Kokhanovsky,et al.  Aerosol remote sensing over land: A comparison of satellite retrievals using different algorithms and instruments , 2007, Atmospheric Research.

[6]  J. Townshend,et al.  Characterizing land surface anisotropy from AVHRR data at a global scale using high performance computing , 2001 .

[7]  Chein-I Chang,et al.  High Performance Computing in Remote Sensing , 2007, HiPC 2007.

[8]  Miron Livny,et al.  Condor and the Grid , 2003 .

[9]  Imtiaz Ahmad,et al.  D-ISODATA: A Distributed Algorithm for Unsupervised Classification of Remotely Sensed Data on Network of Workstations , 1999, J. Parallel Distributed Comput..

[10]  Yanguang Wang,et al.  Preliminary study of Grid computing for remotely sensed information , 2005 .

[11]  Jarek Nabrzyski,et al.  Grid resource management: state of the art and future trends , 2004 .

[12]  Chuang Liu,et al.  Design and evaluation of a resource selection framework for Grid applications , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[13]  Philip H. Swain,et al.  Remote Sensing: The Quantitative Approach , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Richard Healey,et al.  Parallel Processing Algorithms for GIS , 1997 .

[15]  Ian T. Foster,et al.  The Anatomy of the Grid: Enabling Scalable Virtual Organizations , 2001, Int. J. High Perform. Comput. Appl..

[16]  Chao-Tung Yang,et al.  USING A BEOWULF CLUSTER FOR A REMOTE SENSING APPLICATION , 2001 .

[17]  Daniel C. Stanzione,et al.  Coven-a framework for high performance problem solving environments , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[18]  Antonio J. Plaza,et al.  Commodity cluster-based parallel processing of hyperspectral imagery , 2006, J. Parallel Distributed Comput..

[19]  David J. Diner,et al.  Simultaneous retrieval of aerosol and surface properties from a combination of AERONET and satellite data , 2007 .

[20]  Francine Berman,et al.  Grid Computing: Making the Global Infrastructure a Reality , 2003 .

[21]  Yves Robert,et al.  Optimal algorithms for scheduling divisible workloads on heterogeneous systems , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[22]  Miron Livny,et al.  Condor and preemptive resume scheduling , 2004 .

[23]  K.A. Hawick,et al.  Distributed High Performance Computation for Remote Sensing , 1997, ACM/IEEE SC 1997 Conference (SC'97).

[24]  Keith C. Clarke,et al.  Loose-Coupling a Cellular Automaton Model and GIS: Long-Term Urban Growth Prediction for San Francisco and Washington/Baltimore , 1998, Int. J. Geogr. Inf. Sci..

[25]  David A. Bader,et al.  High performance computing algorithms for land cover dynamics using remote sensing data , 2000, International Journal of Remote Sensing.

[26]  Ian J. Taylor,et al.  Workflows and e-Science: An overview of workflow system features and capabilities , 2009, Future Gener. Comput. Syst..

[27]  José M. García,et al.  Cluster Computing Using MPI and Windows NT to Solve the Processing of Remotely Sensed Imagery , 1999, PVM/MPI.

[28]  Rajesh Raman,et al.  Matchmaking: distributed resource management for high throughput computing , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[29]  Yong Xue,et al.  Aerosol optical thickness determination by exploiting the synergy of TERRA and AQUA MODIS , 2005 .

[30]  Tarek El-Ghazawi,et al.  High-performance automatic image registration for remote sensing , 1999 .