Parallel hyperspectral image processing on distributed multicluster systems

Computationally efficient processing of hyperspectral image cubes can be greatly beneficial in many application domains, including environmental modeling, risk/hazard prevention and response, and defense/security. As individual cluster computers often cannot satisfy the computational demands of emerging problems in hyperspectral imaging, there is a growing need for distributed supercomputing using multicluster systems. A well-known manner of obtaining speedups in hyperspectral imaging is to apply data parallel approaches, in which commonly used data structures (e.g., the image cubes) are being scattered among the available compute nodes. Such approaches work well for individual compute clusters, but—due to the inherently large wide-area communication overheads—these are generally not applied in distributed multi-cluster systems. Given the nature of many algorithmic approaches in hyperspectral imaging, however, and due to the increasing availability of high-bandwidth optical networks, wide-area data parallel execution may well be a feasible acceleration approach. This paper discusses the wide-area data parallel execution of two realistic and state-of-the-art algorithms for endmember extraction in hyperspectral unmixing applications: automatic morphological endmember extraction and orthogonal subspace projection. It presents experimental results obtained on a real-world multicluster system, and provides a feasibility analysis of the applied parallelization approaches. The two parallel algorithms evaluated in this work had been developed before for single-cluster execution, and were not changed. Because no further implementation efforts were required, the proposed methodology is easy to apply to already available algorithms, thus reducing complexity and enhancing standardization.