Building an Earth Observations Data Cube: lessons learned from the Swiss Data Cube (SDC) on generating Analysis Ready Data (ARD)

Abstract Pressures on natural resources are increasing and a number of challenges need to be overcome to meet the needs of a growing population in a period of environmental variability. Some of these environmental issues can be monitored using remotely sensed Earth Observations (EO) data that are increasingly available from a number of freely and openly accessible repositories. However, the full information potential of EO data has not been yet realized. They remain still underutilized mainly because of their complexity, increasing volume, and the lack of efficient processing capabilities. EO Data Cubes (DC) are a new paradigm aiming to realize the full potential of EO data by lowering the barriers caused by these Big data challenges and providing access to large spatio-temporal data in an analysis ready form. Systematic and regular provision of Analysis Ready Data (ARD) will significantly reduce the burden on EO data users. Nevertheless, ARD are not commonly produced by data providers and therefore getting uniform and consistent ARD remains a challenging task. This paper presents an approach to enable rapid data access and pre-processing to generate ARD using interoperable services chains. The approach has been tested and validated generating Landsat ARD while building the Swiss Data Cube.

[1]  Denisa Rodila,et al.  Live Monitoring of Earth Surface (LiMES): A framework for monitoring environmental changes from Earth Observations , 2017 .

[2]  F. Chapin,et al.  Planetary boundaries: Exploring the safe operating space for humanity , 2009 .

[3]  Zhenlong Li,et al.  Big Data and cloud computing: innovation opportunities and challenges , 2017, Int. J. Digit. Earth.

[4]  Amir Hossein Alavi,et al.  Machine learning in geosciences and remote sensing , 2016 .

[5]  Konstantinos Evangelidis,et al.  Geospatial services in the Cloud , 2014, Comput. Geosci..

[6]  C. Woodcock,et al.  Classification and Change Detection Using Landsat TM Data: When and How to Correct Atmospheric Effects? , 2001 .

[7]  Joseph Antony,et al.  The NCI High Performance Computing and High Performance Data Platform to Support the Analysis of Petascale Environmental Data Collections , 2014, ISESS.

[8]  Adam Lewis,et al.  Unlocking the Australian Landsat Archive – From dark data to High Performance Data infrastructures , 2015 .

[9]  Darrel L. Williams,et al.  Historical record of Landsat global coverage: mission operations, NSLRSDA, and International Cooperator stations , 2006 .

[10]  Michael A. Wulder,et al.  Opening the archive: How free data has enabled the science and monitoring promise of Landsat , 2012 .

[11]  Joanne C. White,et al.  Optical remotely sensed time series data for land cover classification: A review , 2016 .

[12]  Gregory Giuliani,et al.  Lifting the Information Barriers to Address Sustainability Challenges with Data from Physical Geography and Earth Observation , 2017 .

[13]  Michael A. Wulder,et al.  Landsat continuity: Issues and opportunities for land cover monitoring , 2008 .

[14]  Konstantinos Karantzalos,et al.  Scalable Geospatial Web Services through Efficient, Online and Near Real-Time Processing of Earth Observation Data , 2015, 2015 IEEE First International Conference on Big Data Computing Service and Applications.

[15]  Roberto Roncella,et al.  Bringing GEOSS Services into Practice: A Capacity Building Resource on Spatial Data Infrastructures (SDI) , 2017, Trans. GIS.

[16]  A. Gore The digital earth : Understanding our planet in the 21st century , 1998 .

[17]  Peter Baumann,et al.  Big Data Analytics for Earth Sciences: the EarthServer approach , 2016, Int. J. Digit. Earth.

[18]  Dimitar Misev,et al.  Fostering Cross-Disciplinary Earth Science Through Datacube Analytics , 2018 .

[19]  Huadong Guo,et al.  Earth observation satellite sensors for biodiversity monitoring: potentials and bottlenecks , 2014 .

[20]  Nicholas C. Coops,et al.  Virtual constellations for global terrestrial monitoring , 2015 .

[21]  Julia A. Barsi,et al.  The next Landsat satellite: The Landsat Data Continuity Mission , 2012 .

[22]  Feng Gao,et al.  Landsat Ecosystem Disturbance Adaptive Processing System LEDAPS algorithm description , 2013 .

[23]  Didier Tanré,et al.  Second Simulation of the Satellite Signal in the Solar Spectrum, 6S: an overview , 1997, IEEE Trans. Geosci. Remote. Sens..

[24]  Joachim Denzler,et al.  Multivariate anomaly detection for Earth observations: a comparison of algorithms and feature extraction techniques , 2016 .

[25]  Adam Lewis,et al.  Rapid, high-resolution detection of environmental change over continental scales from satellite data – the Earth Observation Data Cube , 2016, Int. J. Digit. Earth.

[26]  Katherine Anderson,et al.  Earth observation in service of the 2030 Agenda for Sustainable Development , 2017, Geo spatial Inf. Sci..

[27]  Ben Evans,et al.  The Australian Geoscience Data Cube - foundations and lessons learned , 2017 .

[28]  K. Meissner,et al.  How Essential Biodiversity Variables and remote sensing can help national biodiversity monitoring , 2017 .

[29]  L. Lymburner,et al.  Extracting the intertidal extent and topography of the Australian coastline from a 28 year time series of Landsat observations , 2017 .

[30]  David P. Roy,et al.  The global Landsat archive: Status, consolidation, and direction , 2016 .

[31]  M. Tulbure,et al.  Surface water extent dynamics from three decades of seasonally continuous Landsat time series at subcontinental scale in a semi-arid region , 2016 .

[32]  Peter Baumann,et al.  The RasDaMan approach to multidimensional database management , 1997, SAC '97.

[33]  Martha C. Anderson,et al.  Free Access to Landsat Imagery , 2008, Science.

[34]  C. Woodcock,et al.  Improvement and expansion of the Fmask algorithm: cloud, cloud shadow, and snow detection for Landsats 4–7, 8, and Sentinel 2 images , 2015 .

[35]  J. Douglas McCuistion,et al.  From observations to decision support: the new paradigm for satellite data , 2005 .

[36]  Stefano Nativi,et al.  Big Data challenges in building the Global Earth Observation System of Systems , 2015, Environ. Model. Softw..

[37]  Nancy D. Searby,et al.  GEOCAB Portal: A gateway for discovering and accessing capacity building resources in Earth Observation , 2017, Int. J. Appl. Earth Obs. Geoinformation.

[38]  Gustavo Camps-Valls,et al.  Machine learning in remote sensing data processing , 2009, 2009 IEEE International Workshop on Machine Learning for Signal Processing.

[39]  Chaowei Yang,et al.  Utilizing Cloud Computing to address big geospatial data challenges , 2017, Comput. Environ. Urban Syst..

[40]  Michael Dixon,et al.  Google Earth Engine: Planetary-scale geospatial analysis for everyone , 2017 .

[41]  L. Lymburner,et al.  Water observations from space: Mapping surface water from 25 years of Landsat imagery across Australia , 2016 .