A high-throughput shared service to estimate evapotranspiration using Landsat imagery

Abstract The execution of workflows to estimate time series of evapotranspiration (ET) from Earth Observation Satellite (EOS) data is a computing intensive activity. When running these workflows on their own personal computers, users usually need to restrict one or more of the following dimensions: resolution of input EOS data, size of the area of interest, and/or duration of the time series. Recently, a large public cloud provider has made available a platform for the parallel execution of these workflows. Although this system addresses the performance issue, it still presents limitations. In particular, it provides little support for the management of metadata associated with the executions that generated the ET estimations, impairing the effective sharing of such data. Additionally, it does not allow the reuse of already available implementations of ET estimation algorithms. Moreover, the service governance model is entirely defined by that service provider, raising risks that users may not want to face. We address all these challenges by leveraging software containerization technology. Firstly, we use containers to facilitate the deployment of independent and customizable ET processing services on top of either public or private clouds. This not only addresses the performance issue, but also provides freedom for each service to define its own governance model, in consonance with the needs of users. Secondly, we define a simple protocol allowing the easy integration of different container-based implementations of the different stages of the workflow, including those already available. Finally, the service automatically collects provenance information required for the effective sharing of the output it generates. The paper presents the architecture of the proposed service, emphasizing how it addresses the above-mentioned challenges, and provides a performance assessment of a small deployment. It also includes a discussion on possible applications that can benefit from the proposed service.