Big Earth data: disruptive changes in Earth observation data management and analysis?

ABSTRACT Turning Earth observation (EO) data consistently and systematically into valuable global information layers is an ongoing challenge for the EO community. Recently, the term ‘big Earth data’ emerged to describe massive EO datasets that confronts analysts and their traditional workflows with a range of challenges. We argue that the altered circumstances must be actively intercepted by an evolution of EO to revolutionise their application in various domains. The disruptive element is that analysts and end-users increasingly rely on Web-based workflows. In this contribution we study selected systems and portals, put them in the context of challenges and opportunities and highlight selected shortcomings and possible future developments that we consider relevant for the imminent uptake of big Earth data.

[1]  Dana Petcu,et al.  Earth Observation Data Processing in Distributed Systems , 2010, Informatica.

[2]  Peter Baumann.  Datacube Standards and their Contribution to Analysis-Ready Data , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[3]  Adam Shepherd,et al.  Semantics all the way down: the Semantic Web and open science in big earth data , 2017 .

[4]  Adam Lewis,et al.  Unlocking the Australian Landsat Archive – From dark data to High Performance Data infrastructures , 2015 .

[5]  Peter Baumann,et al.  Big Data Analytics for Earth Sciences: the EarthServer approach , 2016, Int. J. Digit. Earth.

[6]  Nigel Waters,et al.  Using Social Media and Satellite Data for Damage Assessment in Urban Areas During Emergencies , 2017 .

[7]  Dirk Tiede,et al.  Architecture and prototypical implementation of a semantic querying system for big Earth observation image bases , 2017, European journal of remote sensing.

[8]  E. C. Barrett,et al.  Introduction to Environmental Remote Sensing. , 1978 .

[9]  Michael Dixon,et al.  Google Earth Engine: Planetary-scale geospatial analysis for everyone , 2017 .

[10]  M. Marquis,et al.  Accessing the MODIS snow and ice products at the NSIDC DAAC , 2000, IGARSS 2000. IEEE 2000 International Geoscience and Remote Sensing Symposium. Taking the Pulse of the Planet: The Role of Remote Sensing in Managing the Environment. Proceedings (Cat. No.00CH37120).

[11]  Peter Baumann,et al.  Datacubes: A Technology Survey , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[12]  P. Kempeneers,et al.  Optimizing Sentinel-2 image selection in a Big Data context , 2017 .

[13]  A. Thieken,et al.  Sendai Framework for Disaster Risk Reduction – Success or Warning Sign for Paris? , 2015 .

[14]  Dimitar Misev,et al.  Fostering Cross-Disciplinary Earth Science Through Datacube Analytics , 2018 .

[15]  H. Barlow.  Vision: A computational investigation into the human representation and processing of visual information: David Marr. San Francisco: W. H. Freeman, 1982. pp. xvi + 397 , 1983 .

[16]  C. Justice,et al.  High-Resolution Global Maps of 21st-Century Forest Cover Change , 2013, Science.

[17]  M. Weideman,et al.  European Space Agency , 2019, The Grants Register 2022.

[18]  Denisa Rodila,et al.  Building an Earth Observations Data Cube: lessons learned from the Swiss Data Cube (SDC) on generating Analysis Ready Data (ARD) , 2017 .

[19]  Fuzhong Weng,et al.  Evaluation of Special Sensor Microwave Imager/Sounder (SSMIS) Environmental Data Records , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[20]  L. Dijkstra,et al.  Big earth data analytics on Sentinel-1 and Landsat imagery in support to global human settlements mapping , 2017 .

[21]  Adam Lewis,et al.  Rapid, high-resolution detection of environmental change over continental scales from satellite data – the Earth Observation Data Cube , 2016, Int. J. Digit. Earth.

[22]  A. Barrett,et al.  National Operational Hydrologic Remote Sensing Center SNOw Data Assimilation System (SNODAS) Products at NSIDC , 2003 .

[23]  L. Lymburner,et al.  Digital earth Australia – unlocking new value from earth observation data , 2017 .

[24]  Scott C. Burleigh,et al.  Environmental Studies with the Sensor Web: Principles and Practice , 2005, Sensors (Basel, Switzerland).

[25]  Matei Ripeanu,et al.  Amazon S3 for science grids: a viable solution? , 2008, DADC '08.

[26]  Peter Baumann,et al.  The RasDaMan approach to multidimensional database management , 1997, SAC '97.

[27]  Huadong Guo,et al.  Scientific big data and Digital Earth , 2014 .

[28]  Ben Evans,et al.  The Australian Geoscience Data Cube - foundations and lessons learned , 2017 .

[29]  Max J. Egenhofer,et al.  Fields as a Generic Data Type for Big Spatial Data , 2014, GIScience.

[30]  Guido Cervone,et al.  Enhancing the temporal resolution of satellite-based flood extent generation using crowdsourced data for disaster monitoring , 2016 .

[31]  Dirk Tiede,et al.  Semantic and syntactic interoperability in online processing of big Earth observation data , 2017, Int. J. Digit. Earth.

[32]  Paul G. Brown,et al.  Overview of sciDB: large scale array storage, processing and analysis , 2010, SIGMOD Conference.

[33]  Michael A. Wulder,et al.  Opening the archive: How free data has enabled the science and monitoring promise of Landsat , 2012 .

[34]  Gilberto Câmara,et al.  An Algebra for Spatiotemporal Data: From Observations to Events , 2014, Trans. GIS.

[35]  Vipin Kumar,et al.  Big Data in Climate: Opportunities and Challenges for Machine Learning , 2016, SIGIR.

[36]  Claudia Notarnicola,et al.  Addressing Grand Challenges in Earth Observation Science: The Earth Observation Data Centre for Water Resources Monitoring , 2014 .

[37]  Julia Wagemann,et al.  Geospatial web services pave new ways for server-based on-demand access and processing of Big Earth Data , 2018, Int. J. Digit. Earth.

[38]  Stefano Nativi,et al.  A view-based model of data-cube to support big earth data systems interoperability , 2017 .

[39]  Dirk Tiede,et al.  Towards a GEOBIA 2.0 manifesto - achievements and open challenges in information & knowledge extraction from big Earth data , 2018 .

[40]  George Percivall.  Big Geospatial Data – an OGC White Paper , 2017 .

[41]  Fulvio Mazzocchi,et al.  Could Big Data be the end of theory in science? , 2015, EMBO reports.

[42]  J. Pekel,et al.  High-resolution mapping of global surface water and its long-term changes , 2016, Nature.

[43]  Thomas Esch,et al.  Exploiting big earth data from space – first experiences with the timescan processing chain , 2018 .

[44]  Chen Chen,et al.  Opinion: Big data has big potential for applications to climate change adaptation , 2016, Proceedings of the National Academy of Sciences.

[45]  Hao Jiang,et al.  Big Earth Data: a new challenge and opportunity for Digital Earth’s development , 2017, Int. J. Digit. Earth.

[46]  Thomas Blaschke,et al.  Geographic Object-Based Image Analysis – Towards a new paradigm , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[47]  Forrest R. Stevens,et al.  Multitemporal settlement and population mapping from Landsat using Google Earth Engine , 2015, Int. J. Appl. Earth Obs. Geoinformation.

[48]  Geoffrey Boulton,et al.  The challenges of a Big Data Earth , 2018 .

[49]  David P. Roy,et al.  Demonstration of Percent Tree Cover Mapping Using Landsat Analysis Ready Data (ARD) and Sensitivity with Respect to Landsat ARD Processing Level , 2018, Remote. Sens..

[50]  Stefano Nativi,et al.  Mind the gap: big data vs. interoperability and reproducibility of science , 2018 .

[51]  Jiancheng Shi,et al.  The Soil Moisture Active Passive (SMAP) Mission , 2010, Proceedings of the IEEE.

[52]  Un-Habitat,et al.  neW URBAn AGendA , 2016 .

[53]  C. Lynch.  Big data: How do your data grow? , 2008, Nature.

[54]  Peter Baumann.  Standardizing big earth datacubes , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[55]  Stefan Dech,et al.  A semi-automated approach for the generation of a new land use and land cover product for Germany based on Landsat time-series and Lucas in-situ data , 2017 .

[56]  V. Marx.  Biology: The big challenges of big data , 2013, Nature.

[57]  Un Desa.  Transforming our world : The 2030 Agenda for Sustainable Development , 2016 .

[58]  Stefan Adriaensen,et al.  Remote Sensing Letters , 2013 .

[59]  Simon Scheider,et al.  Why good data analysts need to be critical synthesists. Determining the role of semantics in data analysis , 2017, Future Gener. Comput. Syst..

[60]  Vahid Naeimi,et al.  Geophysical parameters retrieval from sentinel-1 SAR data: a case study for high performance computing at EODC , 2016, SpringSim.

[61]  John A. Olson.  Data as a Service: Are We in the Clouds? , 2009 .

[62]  Michael F. Goodchild,et al.  Spatial cloud computing: how can the geospatial sciences use and help shape cloud computing? , 2011, Int. J. Digit. Earth.