Design of the Efficient Archival Warehouse for Weather Forecast System

The goal of this study is design of a database holding archives of weather forecast systems. The detailed description of the project and analysis of the database performance is presented, along with the experimental performance tests of key algorithms. A standard mode of data access is optimal for a sequential access to individual 2D grids (time-local and spatially-global), whereas new applications require reading long time series of localised data (spatially-local and time-global). The design goal is to increase a performance of access to archival spatially-local-time-global data, without visible degradation of performance of the standard access mode. The database is designed as two separate layers. The Format Translation Layer (FTL) is an interface between the database and the file-based output of the simulation and analysis programs. The Distributed Data Storage Layer (DDSL) is responsible for a secure data storage and an efficient access. The FTL reads output forecasts and converts them to the spatially local format. It splits large 2D arrays of data into small patches and forms 3D arrays, using time as the third dimension. The theoretical analysis of the performance shows that four orders of magnitudes improvement in comparison with the standard serial access and two orders of magnitude in comparison with parallelised version of the spatially-global access can be achieved.