A Stochastic Process Approach to Model Distributed Computing on Complex Networks

In this work we present analytic expressions for the expected values of the performance metrics of parallel applications when the distributed computing infrastructure has a complex topology. Through active probing tests we analyse the structure of a real distributed computing environment. From the resulting network we both validate the analytic expressions and explore the performance metrics under different conditions through Monte Carlo simulations. In particular we gauge computing paradigms with different hierarchical structures in computing services. Fully decentralised (i.e., peer-to-peer) environments provide the best performance. Moreover, we show that it is possible to improve significantly the parallel efficiency by implementing more intelligent configurations of computing services and task allocation strategies (e.g., by using a betweenness centrality measure). We qualitatively reproduce results of previous works and provide closed-form solutions that link topology, application’s structure and allocation parameters when job dependencies and a complex network structure are considered.

[1]  Laurence T. Yang,et al.  A Complex Network-Based Approach for Job Scheduling in Grid Environments , 2007, HPCC.

[2]  Vijaya Ramachandran,et al.  The diameter of sparse random graphs , 2007, Random Struct. Algorithms.

[3]  Stefano Zanero,et al.  Optimal number of nodes for computation in grid environments , 2004, 12th Euromicro Conference on Parallel, Distributed and Network-Based Processing, 2004. Proceedings..

[4]  V. Blondel,et al.  Distance distribution in random graphs and application to network exploration. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[6]  Bob Jones.  An Overview of the EGEE Project , 2004, DELOS.

[7]  T. P. Singh,et al.  The Distributed Computing Paradigms: P2P, Grid, Cluster, Cloud, and Jungle , 2013, ArXiv.

[8]  Rajkumar Buyya,et al.  Cloud Computing Principles and Paradigms , 2011 .

[9]  Youcef Derbal,et al.  Entropic Grid Scheduling , 2006, Journal of Grid Computing.

[10]  Anees Shaikh,et al.  Issues with inferring Internet topological attributes , 2002, SPIE ITCom.

[11]  Carolyn McCreary,et al.  A comparison of heuristics for scheduling DAGs on multiprocessors , 1994, Proceedings of 8th International Parallel Processing Symposium.

[12]  Daniel M. Batista,et al.  Self-Adjusting Grid Networks , 2007, 2007 IEEE International Conference on Communications.

[13]  Walter Willinger,et al.  The origin of power laws in Internet topologies revisited , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[14]  Jean-Jacques Pansiot,et al.  On routes and multicast trees in the Internet , 1998, CCRV.

[15]  Lorenza Saitta,et al.  Characterization of a computational grid as a complex system , 2009, GMAC '09.

[16]  G Kendall Maurice,et al.  The Advanced Theory Of Statistics Vol-i , 1943 .

[17]  Federico Boccardi,et al.  Load & backhaul aware decoupled downlink/uplink access in 5G systems , 2014, 2015 IEEE International Conference on Communications (ICC).

[18]  B. Arnold,et al.  A first course in order statistics , 2008 .

[19]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[20]  Yang Zhang,et al.  Grid-centric scheduling strategies for workflow applications , 2010 .

[21]  Agata Fronczak,et al.  Average path length in random networks. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[23]  Rajkumar Buyya,et al.  GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing , 2002, Concurr. Comput. Pract. Exp..

[24]  Narayanaswamy Balakrishnan,et al.  A First Course in Order Statistics (Classics in Applied Mathematics) , 2008 .

[25]  A. Forti,et al.  Dag scheduling for grid computing systems , 2006 .

[26]  Carlos A. Ruggiero,et al.  Complex grid computing , 2003, cond-mat/0312603.

[27]  Mark E. J. Newman,et al.  Power-Law Distributions in Empirical Data , 2007, SIAM Rev..

[28]  Bruce S. Davie,et al.  Computer Networks ISE: A Systems Approach , 2007 .

[29]  Hagit Attiya,et al.  Wiley Series on Parallel and Distributed Computing , 2004, SCADA Security: Machine Learning Concepts for Intrusion Detection and Prevention.

[30]  Ian T. Foster,et al.  Locating Data in (Small-World?) Peer-to-Peer Scientific Collaborations , 2002, IPTPS.

[31]  Hai Jin,et al.  DAGMap: efficient and dependable scheduling of DAG workflow job in Grid , 2010, The Journal of Supercomputing.

[32]  Michalis Faloutsos,et al.  File-sharing in the Internet: A characterization of P2P traffic in the backbone , 2003 .

[33]  S. L. Hakimi.  Graph Theory and Combinatorics with Applications to Digital Systems and Networks , 1977 .

[34]  B. Arnold,et al.  A first course in order statistics , 1994 .

[35]  Albert-László Barabási,et al.  Scale-Free Networks: A Decade and Beyond , 2009, Science.

[36]  Adam Arbree,et al.  Mapping Abstract Complex Workflows onto Grid Environments , 2003, Journal of Grid Computing.

[37]  Bruce S. Davie,et al.  Computer Networks: A Systems Approach , 1996 .

[38]  Béla Bollobás,et al.  Modern Graph Theory , 2002, Graduate Texts in Mathematics.

[39]  Hesham H. Ali,et al.  Task scheduling in parallel and distributed systems , 1994, Prentice Hall series in innovative technology.

[40]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.

[41]  Walter Willinger,et al.  Mathematics and the Internet: A Source of Enormous Confusion and Great Potential , 2009, The Best Writing on Mathematics 2010.

[42]  Ian T. Foster,et al.  The Anatomy of the Grid: Enabling Scalable Virtual Organizations , 2001, Int. J. High Perform. Comput. Appl..

[43]  Michalis Faloutsos,et al.  Power laws and the AS-level internet topology , 2003, TNET.