A big data placement method using NSGA-III in meteorological cloud platform

Meteorological cloud platforms (MCP) are gradually replacing the traditional meteorological information systems to provide information analysis services such as weather forecasting, disaster warning, and scientific research. However, the explosive growth of meteorological data resources has brought new challenges to the placement and management of big data in MCP. On the one hand, managers of MCP need to save energy to achieve cost savings. On the other hand, users need shorter data access time to improve user’s experience. Hence, a big data placement method in MCP is proposed in this paper to deal with challenges above. First, the resource utilization, the data access time, and the energy consumption in MCP with the fat-tree topology are analyzed. Then, a corresponding data placement method, using the improved non-dominated sorting genetic algorithm III (NSGA-III), is designed to optimize the resource usage, energy saving, and efficient data access. Finally, extensive experimental evaluations validate the efficiency and effectiveness of our proposed method.

[1]  Luciana Arantes,et al.  MRA++: Scheduling and data placement on MapReduce for heterogeneous environments , 2015, Future Gener. Comput. Syst..

[2]  Qiang He,et al.  An IoT-Oriented data placement method with privacy preservation in cloud environment , 2018, J. Netw. Comput. Appl..

[3]  Wei-Jen Lee,et al.  Forecasting the Wind Generation Using a Two-Stage Network Based on Meteorological Information , 2009, IEEE Transactions on Energy Conversion.

[4]  Misbah Mubarak,et al.  Predicting the Performance Impact of Different Fat-Tree Configurations , 2017, SC17: International Conference for High Performance Computing, Networking, Storage and Analysis.

[5]  Yan Wang,et al.  Levitin-Polyak well-posedness of generalized vector quasi-equilibrium problems with functional constraints , 2012, J. Glob. Optim..

[6]  Mohan K. Ramamurthy,et al.  Unlocking the Potential of NEXRAD Data through NOAA’s Big Data Partnership , 2017 .

[7]  Lei Ren,et al.  Multi-scale Dense Gate Recurrent Unit Networks for bearing remaining useful life prediction , 2019, Future Gener. Comput. Syst..

[8]  Jianping Pan,et al.  Location-aware associated data placement for geo-distributed data-intensive applications , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[9]  João Paiva.  Scalable Self-Tuning Data Placement in Distributed Key-value Stores , 2013 .

[10]  Abinet Tesfaye Eseye,et al.  Short-term photovoltaic solar power forecasting using a hybrid Wavelet-PSO-SVM model based on SCADA and Meteorological information , 2018 .

[11]  Zhiwei Xu,et al.  The convergence of conjugate gradient method with nonmonotone line search , 2010, Appl. Math. Comput..

[12]  Laurence T. Yang,et al.  A Tensor-Based Big-Data-Driven Routing Recommendation Approach for Heterogeneous Networks , 2019, IEEE Network.

[13]  Xuyun Zhang,et al.  A computation offloading method over big data for IoT-enabled cloud-edge computing , 2019, Future Gener. Comput. Syst..

[14]  Minimal generating reflexive lattices of projections in finite von Neumann algebras , 2012 .

[15]  Derivations of a class of Kadison–Singer algebras☆ , 2015 .

[16]  Kohei Arai,et al.  Aerosol data assimilation using data from Himawari‐8, a next‐generation geostationary meteorological satellite , 2016 .

[17]  Tao Huang,et al.  Structural Evaluation for Distribution Networks with Distributed Generation Based on Complex Network , 2017, Complex..

[18]  Maciej Stachura,et al.  Overview of Small Fixed-Wing Unmanned Aircraft for Meteorological Sampling , 2015 .

[19]  Suryakant A. Sawant,et al.  Interoperable agro-meteorological observation and analysis platform for precision agriculture: A case study in citrus crop water requirement estimation , 2017, Comput. Electron. Agric..

[20]  I A Botygin,et al.  Distributed Information System for Processing and Storage of Meteorological Data , 2016 .

[21]  Ching-Hsien Hsu,et al.  Service Composition in Cyber-Physical-Social Systems , 2020, IEEE Transactions on Emerging Topics in Computing.

[22]  Jiguo Yu,et al.  Privacy-Preserving and Scalable Service Recommendation Based on SimHash in a Distributed Cloud Environment , 2017, Complex..

[23]  Aiju Dong,et al.  On some automorphisms of a class of Kadison–Singer algebras , 2012 .

[24]  Ludovic Favre,et al.  Convergence of Multi-Criteria Optimization of a Building Energetic Resources by Genetic Algorithm , 2018, 2018 International Conference on Smart Grid and Clean Energy Technologies (ICSGCE).

[25]  Jie Zhang,et al.  Data Placement for Privacy-Aware Applications over Big Data in Hybrid Clouds , 2017, Secur. Commun. Networks.

[26]  R. Draxler,et al.  NOAA’s HYSPLIT Atmospheric Transport and Dispersion Modeling System , 2015 .

[27]  Xuyun Zhang,et al.  A balanced virtual machine scheduling method for energy-performance trade-offs in cyber-physical cloud systems , 2017, Future Gener. Comput. Syst..

[28]  Dunwei Gong,et al.  A Set-Based Genetic Algorithm for Interval Many-Objective Optimization Problems , 2018, IEEE Transactions on Evolutionary Computation.

[29]  Xuyun Zhang,et al.  An edge computing-enabled computation offloading method with privacy preservation for internet of connected vehicles , 2019, Future Gener. Comput. Syst..

[30]  Tao Huang,et al.  An energy-aware computation offloading method for smart edge computing in wireless metropolitan area networks , 2019, J. Netw. Comput. Appl..

[31]  Laurence T. Yang,et al.  A Cloud-Edge Computing Framework for Cyber-Physical-Social Services , 2017, IEEE Communications Magazine.

[32]  M. Drinkwater,et al.  The advanced scatterometer (ASCAT) on the meteorological operational (MetOp) platform: A follow on for European wind scatterometers , 2002 .

[33]  Ludovic Favre,et al.  Convergence of Multi-Criteria Optimization of a Building Energetic Resources by Genetic Algorithm , 2018 .

[34]  Luís E. T. Rodrigues,et al.  AutoPlacer: Scalable Self-Tuning Data Placement in Distributed Key-Value Stores , 2015, TAAS.

[35]  Fayez F. Boctor,et al.  An efficient genetic algorithm to solve the resource-constrained project scheduling problem with transfer times: The single mode case , 2018, Eur. J. Oper. Res..

[36]  Yiju Wang,et al.  A new CQ method for solving split feasibility problem , 2010 .

[37]  Stephen S. Yau,et al.  Towards Green Service Composition Approach in the Cloud , 2018, IEEE Transactions on Services Computing.

[38]  Biao Qu,et al.  Merit functions for general mixed quasi-variational inequalities , 2010 .

[39]  R. Eisen,et al.  Meteorological conditions associated with increased incidence of West Nile virus disease in the United States, 2004-2012. , 2015, The American journal of tropical medicine and hygiene.

[40]  张杰,et al.  LEVITIN-POYAK WELL-POSEDNESS OF GENERALIZED VECTOR EQUILIBRIUM PROBLEMS WITH FUNCTIONAL CONSTRAINTS , 2010 .

[41]  Haibin Chen,et al.  A Family of higher-order convergent iterative methods for computing the Moore-Penrose inverse , 2011, Appl. Math. Comput..

[42]  A. A. Sorokin,et al.  The VolSatView information system for Monitoring the Volcanic Activity in Kamchatka and on the Kuril Islands , 2016, Journal of Volcanology and Seismology.

[43]  A. Okuyama,et al.  An Introduction to Himawari-8/9— Japan’s New-Generation Geostationary Meteorological Satellites , 2016 .

[44]  Budiman Minasny,et al.  Using Google's cloud-based platform for digital soil mapping , 2015, Comput. Geosci..

[45]  Gang Wang,et al.  Feasibility-solvbility theorems for generalized vector equilibrium problem in reflexive banach spaces , 2012 .

[46]  Maja Pokric,et al.  ekoNET - Environmental Monitoring Using Low-Cost Sensors for Detecting Gases, Particulate Matter, and Meteorological Parameters , 2014, 2014 Eighth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing.