Computational Infrastructure of SoilGrids 2.0

SoilGrids maps soil properties for the entire globe at medium spatial resolution (250 m cell side) using state-of-the-art machine learning methods. The expanding pool of input data and the increasing computational demands of predictive models required a prediction framework that could deal with large data. This article describes the mechanisms set in place for a geo-spatially parallelised prediction system for soil properties. The features provided by GRASS GIS – mapset and region – are used to limit predictions to a specific geographic area, enabling parallelisation. The Slurm job scheduler is used to deploy predictions in a high-performance computing cluster. The framework presented can be seamlessly applied to most other geo-spatial process requiring parallelisation. This framework can also be employed with a different job scheduler, GRASS GIS being the main requirement and engine.

[1]  Roger Bivand,et al.  Interface Between GRASS 6+ Geographical Information System and R , 2015 .

[2]  Jeremy Kepner,et al.  Scheduler technologies in support of high performance data analysis , 2016, 2016 IEEE High Performance Extreme Computing Conference (HPEC).

[3]  Nicolai Meinshausen,et al.  Quantile Regression Forests , 2006, J. Mach. Learn. Res..

[4]  Budiman Minasny,et al.  Digital soil mapping: A brief history and some lessons , 2016 .

[5]  Philippe Lagacherie,et al.  GlobalSoilMap: Toward a Fine-Resolution Global Grid of Soil Properties , 2014 .

[6]  J. Paul Goode,et al.  THE HOMOLOSINE PROJECTION: A NEW DEVICE FOR PORTRAYING THE EARTH'S SURFACE ENTIRE , 1925 .

[7]  Budiman Minasny,et al.  On digital soil mapping , 2003 .

[8]  Laura Poggio,et al.  Comparison of FOSS4G Supported Equal-Area Projections Using Discrete Distortion Indicatrices , 2019, ISPRS Int. J. Geo Inf..

[9]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[10]  Andy B. Yoo,et al.  Approved for Public Release; Further Dissemination Unlimited X-ray Pulse Compression Using Strained Crystals X-ray Pulse Compression Using Strained Crystals , 2002 .

[11]  Markus Neteler,et al.  Open Source GIS: A GRASS GIS Approach , 2007 .

[12]  J. Snyder Flattening the Earth: Two Thousand Years of Map Projections , 1994 .

[13]  Bruce Momjian,et al.  PostgreSQL: Introduction and Concepts , 2000 .

[14]  Niels H. Batjes,et al.  Standardised soil profile data to support global mapping and modelling (WoSIS snapshot 2019) , 2020 .