Geospatial Big Data for Environmental and Agricultural Applications

Earth observation (EO) and environmental geospatial datasets are growing at an unprecedented rate in size, variety and complexity, thus, creating new challenges and opportunities as far as their access, archiving, processing and analytics are concerned. Currently, huge imaging streams are reaching several petabytes in many satellite archives worldwide. In this chapter, we review the current state-of-the-art in big data frameworks able to access, handle, process, analyse and deliver geospatial data and value-added products. Operational services that feature efficient implementations and different architectures allowing in certain cases the online and near real-time processing and analytics are detailed. Based on the current status, state-of-the-art and emerging challenges, the present study highlights certain issues, insights and future directions towards the efficient exploitation of EO big data for important engineering, environmental and agricultural applications.

[1]  Konstantinos Karantzalos,et al.  Vehicle detection and traffic density monitoring from very high resolution satellite video data , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[2]  Konstantinos Karantzalos,et al.  Scalable Geospatial Web Services through Efficient, Online and Near Real-Time Processing of Earth Observation Data , 2015, 2015 IEEE First International Conference on Big Data Computing Service and Applications.

[3]  Guan Le,et al.  Survey on NoSQL database , 2011, 2011 6th International Conference on Pervasive Computing and Applications.

[4]  Mladen A. Vouk,et al.  Cloud computing — Issues, research and implementations , 2008, ITI 2008 - 30th International Conference on Information Technology Interfaces.

[5]  Ian Gorton,et al.  The Changing Paradigm of Data-Intensive Computing , 2009, Computer.

[6]  Mihai Datcu,et al.  Earth-Observation Image Retrieval Based on Content, Semantics, and Metadata , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Antonio J. Plaza,et al.  Recent Developments in High Performance Computing for Remote Sensing: A Review , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Xiaodong Zhang,et al.  Digital Northern Great Plains: A Web-Based System Delivering Near Real Time Remote Sensing Data for Precision Agriculture , 2010, Remote. Sens..

[9]  Cees T. A. M. de Laat,et al.  Addressing Big Data challenges for Scientific Data Infrastructure , 2012, 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings.

[10]  Albert Y. Zomaya,et al.  Remote sensing big data computing: Challenges and opportunities , 2015, Future Gener. Comput. Syst..

[11]  Jim Gray,et al.  Distributed Computing Economics , 2004, ACM Queue.

[12]  Peter Baumann,et al.  PetaScope: An Open-Source Implementation of the OGC WCS Geo Service Standards Suite , 2010, SSDBM.

[13]  Mihai Datcu,et al.  Big, Linked and Open Data: Applications in the German Aerospace Center , 2014, ESWC.

[14]  Genshe Chen,et al.  Scalable sentiment classification for Big Data analysis using Naïve Bayes Classifier , 2013, 2013 IEEE International Conference on Big Data.

[15]  S. G. Nelson,et al.  Relationship Between Remotely-sensed Vegetation Indices, Canopy Attributes and Plant Physiological Processes: What Vegetation Indices Can and Cannot Tell Us About the Landscape , 2008, Sensors.

[16]  Walid G. Aref,et al.  Supporting views in data stream management systems , 2010, TODS.

[17]  Hal Finkel,et al.  HACC , 2016, Commun. ACM.

[18]  Peter Baumann.  rasdaman: Array Databases Boost Spatio-Temporal Analytics , 2014, 2014 Fifth International Conference on Computing for Geospatial Research and Application.

[19]  Douglas Nebert,et al.  OpenGIS® Catalogue Services Specification , 2007 .

[20]  Borko Furht,et al.  Handbook of Cloud Computing , 2010 .

[21]  Liping Di,et al.  A Geospatial Web Service Approach for Creating On-Demand Cropland Data Layer Thematic Maps , 2014 .

[22]  Peter D. Hunter,et al.  Spectral discrimination of phytoplankton colour groups: The effect of suspended particulate matter and sensor spectral resolution , 2008 .

[23]  Dimitar Misev,et al.  PlanetServer: Innovative approaches for the online analysis of hyperspectral satellite data from Mars , 2014 .

[24]  Peter Baumann,et al.  A Database Array Algebra for Spatio-Temporal Data and Beyond , 1999, NGITS.

[25]  Joel H. Saltz,et al.  Hadoop-GIS: A High Performance Spatial Data Warehousing System over MapReduce , 2013, Proc. VLDB Endow..

[26]  Tim Menzies,et al.  Software Analytics: So What? , 2013, IEEE Softw..

[27]  Albert Y. Zomaya,et al.  Task-Tree Based Large-Scale Mosaicking for Massive Remote Sensed Imageries with Dynamic DAG Scheduling , 2014, IEEE Transactions on Parallel and Distributed Systems.

[28]  Martha C. Anderson,et al.  Mapping daily evapotranspiration at field scales over rainfed and irrigated agricultural areas using remote sensing data fusion , 2014 .

[29]  Heiko Balzter,et al.  Validation of Envisat MERIS algorithms for chlorophyll retrieval in a large, turbid and optically-complex shallow lake , 2015 .

[30]  N. Pettorelli,et al.  Using the satellite-derived NDVI to assess ecological responses to environmental change. , 2005, Trends in ecology & evolution.

[31]  Jin Chen,et al.  Global land cover mapping at 30 m resolution: A POK-based operational approach , 2015 .

[32]  Peter Baumann,et al.  The OGC web coverage processing service (WCPS) standard , 2010, GeoInformatica.

[33]  Peng Yue,et al.  The Geoprocessing Web , 2012, Comput. Geosci..

[34]  Martin L. Kersten,et al.  Data Vaults: A Symbiosis between Database Technology and Scientific File Repositories , 2012, SSDBM.

[35]  Nikos Pelekis,et al.  Mobility Data Management and Exploration , 2014, Springer New York.

[36]  Antonio J. Plaza,et al.  Cloud Implementation of a Full Hyperspectral Unmixing Chain Within the NASA Web Coverage Processing Service for EO-1 , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[37]  Martin L. Kersten,et al.  MonetDB: Two Decades of Research in Column-oriented Database Architectures , 2012, IEEE Data Eng. Bull..

[38]  Konstantinos Karantzalos,et al.  A Scalable Geospatial Web Service for Near Real-Time, High-Resolution Land Cover Mapping , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[39]  Luiz André Barroso,et al.  Web Search for a Planet: The Google Cluster Architecture , 2003, IEEE Micro.

[40]  J. A. Schell,et al.  Monitoring vegetation systems in the great plains with ERTS , 1973 .

[41]  Ying Zhang,et al.  SciQL: bridging the gap between science and relational DBMS , 2011, IDEAS '11.

[42]  Jaegul Choo,et al.  Customizing Computational Methods for Visual Analytics with Big Data , 2013, IEEE Computer Graphics and Applications.

[43]  Peter Baumann,et al.  Management of multidimensional discrete data , 1994, The VLDB Journal.

[44]  Rajkumar Buyya,et al.  Big Data computing and clouds: Trends and future directions , 2013, J. Parallel Distributed Comput..

[45]  Sean D Dessureault,et al.  Understanding big data , 2016 .

[46]  Ying Zhang,et al.  Astronomical Data Processing Using SciQL, an SQL Based Query Language for Array Data , 2011 .

[47]  Paul Zikopoulos,et al.  Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data , 2011 .

[48]  Michael Zeiler,et al.  Modeling our world : the ESRI guide to geodatabase design , 1999 .

[49]  Yong Zhao,et al.  Cloud Computing and Grid Computing 360-Degree Compared , 2008, GCE 2008.

[50]  Bryan C. Pijanowski,et al.  A big data urban growth simulation at a national scale: Configuring the GIS and neural network based Land Transformation Model to run in a High Performance Computing (HPC) environment , 2014, Environ. Model. Softw..

[51]  A. Adamov.  Distributed file system as a basis of data-intensive computing , 2012, 2012 6th International Conference on Application of Information and Communication Technologies (AICT).

[52]  Jens Lehmann,et al.  GeoKnow: Leveraging Geospatial Data in the Web of Data , 2013 .

[53]  Peng Yue,et al.  Intelligent services for discovery of complex geospatial features from remote sensing imagery , 2013 .

[54]  C. Justice,et al.  High-Resolution Global Maps of 21st-Century Forest Cover Change , 2013, Science.

[55]  Liping Di,et al.  GeoPW: Laying Blocks for the Geospatial Processing Web , 2010, Trans. GIS.

[56]  Adam T. Carpenter,et al.  A User-Driven Approach to Determining Critical Earth Observation Priorities for Societal Benefit , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[57]  Antonio J. Plaza,et al.  Special issue on architectures and techniques for real-time processing of remotely sensed images , 2009, Journal of Real-Time Image Processing.

[58]  Konstantinos Karantzalos,et al.  Benchmarking server-side software modules for handling and processing remote sensing data through Rasdaman , 2015, 2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[59]  Chein-I Chang,et al.  High Performance Computing in Remote Sensing , 2007, HiPC 2007.

[60]  Stefan Manegold,et al.  Real-time wildfire monitoring using scientific database and linked data technologies , 2013, EDBT '13.

[61]  John H. Prueger,et al.  Value of Using Different Vegetative Indices to Quantify Agricultural Crop Characteristics at Different Growth Stages under Varying Management Practices , 2010, Remote. Sens..

[62]  Konstantinos Evangelidis,et al.  Geospatial services in the Cloud , 2014, Comput. Geosci..

[63]  Minha Choi,et al.  Seasonal trends of satellite-based evapotranspiration algorithms over a complex ecosystem in East Asia , 2013 .

[64]  Mihai Datcu,et al.  Assessment of dimensionality reduction based on communication channel model; application to immersive information visualization , 2013, 2013 IEEE International Conference on Big Data.

[65]  B. E. Davis.  Gis a Visual Approach , 1996 .

[66]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[67]  Edmon Begoli,et al.  Design Principles for Effective Knowledge Discovery from Big Data , 2012, 2012 Joint Working IEEE/IFIP Conference on Software Architecture and European Conference on Software Architecture.

[68]  Peter Baumann,et al.  The multidimensional database system RasDaMan , 1998, SIGMOD '98.

[69]  I. Creed,et al.  Understanding variation in trophic status of lakes on the Boreal Plain: A 20 year retrospective using Landsat TM imagery , 2007 .

[70]  Rajiv Ranjan,et al.  Towards building a data-intensive index for big data computing - A case study of Remote Sensing data processing , 2015, Inf. Sci..

[71]  Konstantinos Karantzalos,et al.  RemoteAgri: processing online big earth observation data for precision agriculture , 2015 .

[72]  Daniel M. Batista,et al.  A Survey of Large Scale Data Management Approaches in Cloud Environments , 2011, IEEE Communications Surveys & Tutorials.

[73]  Ghassem R. Asrar,et al.  Estimates of leaf area index from spectral reflectance of wheat under different cultural practices and solar angle , 1985 .