Big Data and Cloud Computing

Big data emerged as a new paradigm to provide unprecedented content and value for Digital Earth. Big Earth data are increasing tremendously with growing heterogeneity, posing grand challenges for the data management lifecycle of storage, processing, analytics, visualization, sharing, and applications. During the same time frame, cloud computing emerged to provide crucial computing support to address these challenges. This chapter introduces Digital Earth data sources, analytical methods, and architecture for data analysis and describes how cloud computing supports big data processing in the context of Digital Earth.

[1]  Michael Stonebraker,et al.  A Demonstration of SciDB: A Science-Oriented DBMS , 2009, Proc. VLDB Endow..

[2]  J. Blachowski,et al.  Application of GIS spatial regression methods in assessment of land subsidence in complicated mining conditions: case study of the Walbrzych coal mine (SW Poland) , 2016, Natural Hazards.

[3]  Weisong Shi,et al.  Edge Computing: Vision and Challenges , 2016, IEEE Internet of Things Journal.

[4]  Cyrus Shahabi,et al.  Supporting spatial aggregation in sensor network databases , 2004, GIS '04.

[5]  Anna Söderberg,et al.  Turning Smart Water Meter Data Into Useful Information : A case study on rental apartments in Södertälje , 2017 .

[6]  Mohamed Sarwat,et al.  GeoSpark: a cluster computing framework for processing large-scale spatial data , 2015, SIGSPATIAL/GIS.

[7]  Zhou Guo,et al.  Using multi-scale and hierarchical deep convolutional features for 3D semantic classification of TLS point clouds , 2018, Int. J. Geogr. Inf. Sci..

[8]  Ruixin Yang.  A Systematic Classification Investigation of Rapid Intensification of Atlantic Tropical Cyclones with the SHIPS Database , 2016 .

[9]  Myanna Lahsen,et al.  Toward a Sustainable Future Earth , 2016 .

[10]  Chaowei Phil Yang,et al.  Evaluating the Open Source Data Containers for Handling Big Geospatial Raster Data , 2018, ISPRS Int. J. Geo Inf..

[11]  Dan Xu,et al.  Spatial data cube: provides better support for spatial data mining , 2005, Proceedings. 2005 IEEE International Geoscience and Remote Sensing Symposium, 2005. IGARSS '05..

[12]  Wenwen Li,et al.  Automated terrain feature identification from remote sensing imagery: a deep learning approach , 2018, Int. J. Geogr. Inf. Sci..

[13]  Guangwen Yang,et al.  Adaptive Indexing for Distributed Array Processing , 2014, 2014 IEEE International Congress on Big Data.

[14]  Zhenlong Li,et al.  Building Model as a Service to support geosciences , 2017, Comput. Environ. Urban Syst..

[15]  Maged N. Kamel Boulos,et al.  On the Internet of Things, smart cities and the WHO Healthy Cities , 2014, International Journal of Health Geographics.

[16]  Roberto Roncella,et al.  Bringing GEOSS Services into Practice: A Capacity Building Resource on Spatial Data Infrastructures (SDI) , 2017, Trans. GIS.

[17]  David O'Sullivan,et al.  Detecting ethnic residential clusters using an optimisation clustering method , 2012, Int. J. Geogr. Inf. Sci..

[18]  Russ Rew,et al.  NetCDF: an interface for scientific data access , 1990, IEEE Computer Graphics and Applications.

[19]  Liang Guo,et al.  Trust evaluation model of cloud manufacturing service platform , 2014 .

[20]  Huadong Guo,et al.  Big Earth data: A new frontier in Earth and information sciences , 2017 .

[21]  Michael F. Goodchild,et al.  Spatial cloud computing: how can the geospatial sciences use and help shape cloud computing? , 2011, Int. J. Digit. Earth.

[22]  Thomas S. Huang,et al.  Towards intelligent geospatial data discovery: a machine learning framework for search ranking , 2018, Int. J. Digit. Earth.

[23]  Yixiang Chen,et al.  A trajectory clustering approach based on decision graph and data field for detecting hotspots , 2017, Int. J. Geogr. Inf. Sci..

[24]  Bin Jiang,et al.  Geospatial Big Data Handling Theory and Methods: A Review and Research Challenges , 2015, ArXiv.

[25]  Tharam S. Dillon,et al.  Cloud Computing: Issues and Challenges , 2010, 2010 24th IEEE International Conference on Advanced Information Networking and Applications.

[26]  Yong Zhao,et al.  Cloud Computing and Grid Computing 360-Degree Compared , 2008, GCE 2008.

[27]  Abbas Rajabifard,et al.  A framework for a microscale flood damage assessment and visualization for a building using BIM–GIS integration , 2016, Int. J. Digit. Earth.

[28]  Michael J. Franklin,et al.  Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing , 2012, NSDI.

[29]  Thomas S. Huang,et al.  A Smart Web-Based Geospatial Data Discovery System with Oceanographic Data as an Example , 2018, ISPRS Int. J. Geo Inf..

[30]  Konstantina Bereta,et al.  The Copernicus App Lab project: Easy Access to Copernicus Data , 2019, EDBT.

[31]  François Clemens,et al.  Interpolation in Time Series : An Introductive Overview of Existing Methods, Their Performance Criteria and Uncertainty Assessment , 2017 .

[32]  Zhenlong Li,et al.  A spatiotemporal indexing approach for efficient processing of big array-based climate data with MapReduce , 2017, Int. J. Geogr. Inf. Sci..

[33]  Mark Gahegan,et al.  Geospatial Cyberinfrastructure: Past, present and future , 2010, Comput. Environ. Urban Syst..

[34]  Stefano Nativi,et al.  A view-based model of data-cube to support big earth data systems interoperability , 2017 .

[35]  Tanu Malik.  GeoBase: Indexing NetCDF Files for Large-Scale Data Analysis , 2014 .

[36]  Manzhu Yu,et al.  Big Data in Natural Disaster Management: A Review , 2018 .

[37]  S. Schubert,et al.  MERRA: NASA’s Modern-Era Retrospective Analysis for Research and Applications , 2011 .

[38]  Carol A. Gotway,et al.  Statistical Methods for Spatial Data Analysis , 2004 .

[39]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[40]  Alexander Novikov,et al.  Transparent Data Cube for Spatiotemporal Data Mining and Visualization , 2011, Grid and Cloud Database Management.

[41]  Edzer Pebesma,et al.  Spatial aggregation and soil process modelling , 1999 .

[42]  Shaowen Wang,et al.  TeraGrid GIScience Gateway: Bridging cyberinfrastructure and GIScience , 2009, Int. J. Geogr. Inf. Sci..

[43]  Dimitris Kofinas,et al.  A methodology for synthetic household water consumption data generation , 2018, Environ. Model. Softw..

[44]  Han Qi,et al.  Research on mobile cloud computing: Review, trend and perspectives , 2012, 2012 Second International Conference on Digital Information and Communication Technology and it's Applications (DICTAP).

[45]  Peter Baumann,et al.  Big Data Analytics for Earth Sciences: the EarthServer approach , 2016, Int. J. Digit. Earth.

[46]  Mohamed El-Mekawy,et al.  INTEGRATING BIM AND GIS FOR 3D CITY MODELLING The Case of IFC and CityGML , 2010 .

[47]  Marimuthu Palaniswami,et al.  Real-Time Urban Microclimate Analysis Using Internet of Things , 2018, IEEE Internet of Things Journal.

[48]  Josef Weidendorfer,et al.  Considering GPGPU for HPC Centers: Is It Worth the Effort? , 2010, Facing the Multicore-Challenge.

[49]  Michel Krämer,et al.  A modular software architecture for processing of big geospatial data in the cloud , 2015, Comput. Graph..

[50]  Mitchell M. Tseng,et al.  Design Considerations for Building Distributed Supply Chain Management Systems Based on Cloud Computing , 2015 .

[51]  Jing Li,et al.  A service visualization tool for spatial web portal , 2011, COM.Geo.

[52]  Hao Jiang,et al.  Big Earth Data: a new challenge and opportunity for Digital Earth’s development , 2017, Int. J. Digit. Earth.

[53]  Christopher N. Eichelberger,et al.  GeoMesa: a distributed architecture for spatio-temporal fusion , 2015, Defense + Security Symposium.

[54]  Stefano Nativi,et al.  Big Data challenges in building the Global Earth Observation System of Systems , 2015, Environ. Model. Softw..

[55]  Chaowei Phil Yang,et al.  ClimateSpark: An in-memory distributed computing framework for big climate data analytics , 2018, Comput. Geosci..

[56]  Chaowei Yang,et al.  Enabling Big Geoscience Data Analytics with a Cloud-Based, MapReduce-Enabled and Service-Oriented Workflow Framework , 2015, PloS one.

[57]  Carlos Maltzahn,et al.  SciHadoop: Array-based query processing in Hadoop , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[58]  Luca Trani,et al.  Data cube and cloud resources as platform for seamless geospatial computation , 2018, CF.

[59]  T. Yoshimi,et al.  Proposal of object management system for applying to existing object storage furniture , 2011, 2011 IEEE/SICE International Symposium on System Integration (SII).

[60]  Marvin Mc Cutchan.  Linked Data for a Digital Earth: Spatial Forecasting with Next Generation Geographical Data , 2017, COSIT.

[61]  Bo Li,et al.  Parallel Accessing Massive NetCDF Data Based on MapReduce , 2010, WISM.

[62]  Xiaowei Yang,et al.  CloudCmp: comparing public cloud providers , 2010, IMC '10.

[63]  Guangwen Yang,et al.  SciHive: Array-Based Query Processing with HiveQL , 2013, 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications.

[64]  N. Lam.  Spatial Interpolation Methods: A Review , 1983 .

[65]  Wenwen Li,et al.  The GEOSS clearinghouse high performance search engine , 2011, 2011 19th International Conference on Geoinformatics.

[66]  Bernard Marr,et al.  Big Data: Using SMART Big Data, Analytics and Metrics To Make Better Decisions and Improve Performance , 2015 .

[67]  Luis Ramirez,et al.  Big Data Analysis Using Modern Statistical and Machine Learning Methods in Medicine , 2014, International neurourology journal.

[68]  M. R. Rahman,et al.  Climate change in Bangladesh: a spatio-temporal analysis and simulation of recent temperature and rainfall data using GIS and time series analysis model , 2017, Theoretical and Applied Climatology.

[69]  David Semmelroth,et al.  Statistics for Big Data For Dummies , 2015 .

[70]  P. Mell,et al.  SP 800-145. The NIST Definition of Cloud Computing , 2011 .

[71]  Annekatrin Metz,et al.  Earth observation-supported service platform for the development and provision of thematic information on the built environment — the TEP-Urban project , 2017, 2017 Joint Urban Remote Sensing Event (JURSE).

[72]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[73]  B. Segal,et al.  Grid computing: the European Data Grid Project , 2000, 2000 IEEE Nuclear Science Symposium. Conference Record (Cat. No.00CH37149).

[74]  Michael F. Goodchild,et al.  Towards geospatial semantic search: exploiting latent semantic relations in geospatial data , 2014, Int. J. Digit. Earth.

[75]  Chitra Balakrishna,et al.  Enabling Technologies for Smart City Services and Applications , 2012, 2012 Sixth International Conference on Next Generation Mobile Applications, Services and Technologies.

[76]  Nor Badrul Anuar,et al.  The role of big data in smart city , 2016, Int. J. Inf. Manag..

[77]  Ramin Yahyapour,et al.  Design and evaluation of job scheduling strategies for grid computing , 2000, GRID.

[78]  Tomasz Imielinski,et al.  Mining association rules between sets of items in large databases , 1993, SIGMOD Conference.

[79]  Asunción Gómez-Pérez,et al.  Integrating geographical information in the Linked Digital Earth , 2014, Int. J. Digit. Earth.

[80]  Syed Mohd Ali,et al.  Comparative analysis of SpatialHadoop and GeoSpark for geospatial big data analytics , 2016, 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I).

[81]  H. Barbas,et al.  Pathways for Emotions and Attention Converge on the Thalamic Reticular Nucleus in Primates , 2012, The Journal of Neuroscience.

[82]  Wo L. Chang,et al.  NIST Big Data Interoperability Framework: Volume 1, Big Data Definitions [Version 2] , 2015 .

[83]  Chaowei Yang,et al.  Utilizing Cloud Computing to address big geospatial data challenges , 2017, Comput. Environ. Urban Syst..

[84]  Zhenlong Li,et al.  Big Data and cloud computing: innovation opportunities and challenges , 2017, Int. J. Digit. Earth.

[85]  Ruixin Yang,et al.  Association Rule Data Mining Applications for Atlantic Tropical Cyclone Intensity Changes , 2011 .