A System for GIS Polygonal Overlay Computation on Linux Cluster - An Experience and Performance Report

GIS polygon-based (also know as vector-based) spatial data overlay computation is much more complex than raster data computation. Processing of polygonal spatial data files has been a long standing research question in GIS community due to the irregular and data intensive nature of the underlying computation. The state-of-the-art software for overlay computation in GIS community is still desktop-based. We present a cluster-based distributed solution for end-to-end polygon overlay processing, modeled after our Windows Azure cloud-based Crayons system [1]. We present the details of porting Crayons system to MPI-based Linux cluster and show the improvements made by employing efficient data structures such as R-trees. We present performance report and show the scalability of our system, along with the remaining bottlenecks. Our experimental results show an absolute speedup of 15x for end-to-end overlay computation employing up to 80 cores.

[1]  Bruce Gittings,et al.  Towards a framework for High Performance Geocomputation: Handling Vector Topology within a Distributed Service Environment , 2000 .

[2]  Howard S. Seifert,et al.  Jet Propulsion Laboratory , 2008 .

[3]  Marc P. Armstrong,et al.  DOMAIN DECOMPOSITION FOR PARALLEL PROCESSING OF SPATIAL PROBLEMS , 1992 .

[4]  David Sun,et al.  UNIFORM GRIDS: A TECHNIQUE FOR INTERSECTION DETECTION ON SERIAL AND PARALLEL MACHINES , 2008 .

[5]  Thomas C. Waugh,et al.  An algorithm for polygon overlay using cooperative parallel processing , 1992, Int. J. Geogr. Inf. Sci..

[6]  John D. Hobby,et al.  Practical segment intersection with finite precision output , 1999, Comput. Geom..

[7]  Fangju Wang A parallel intersection algorithm for vector polygon overlay , 1993, IEEE Computer Graphics and Applications.

[8]  Richard Healey,et al.  Parallel Processing Algorithms for GIS , 1997 .

[9]  Dinesh Manocha,et al.  Fast computation of database operations using graphics processors , 2005, SIGGRAPH Courses.

[10]  T B Comstock,et al.  U. S. Geological Survey , 1907, Radiocarbon.

[11]  Timothy M. Chan A Simple Trapezoid Sweep Algorithm for Reporting Red/Blue Segment Intersections , 1994, CCCG.

[12]  Dinesh Agarwal Crayons: An Azure Cloud Based Parallel System for GIS Overlay Operations , 2012, 2012 SC Companion: High Performance Computing, Networking Storage and Analysis.

[13]  Antonin Guttman,et al.  R-trees: a dynamic index structure for spatial searching , 1984, SIGMOD '84.

[14]  Ershun Zhong,et al.  A parallel line segment intersection strategy based on uniform grids , 2009 .

[15]  Pankaj K. Agarwal,et al.  I/o-efficient efficient algorithms for computing contours on a terrain , 2008, SCG '08.