Application resource requirement estimation in a parallel-pipeline model of execution

We propose a massively parallel framework termed a parallel-pipeline model of execution that can be employed on a homogeneous computational cluster. We show that speedups near-linear in the number of processors are achievable for applications involving reduction operations based on a novel, parallel-pipeline model of execution. As computational clusters become viable alternative platforms for solving large computational problems, the research community acknowledges that the cluster environment can be used effectively when adaptive resource management is employed. This requires the ability to estimate the resource requirements of applications before scheduling decisions are made. We propose a resource estimation model for applications executed in the parallel-pipeline model of execution. We develop a performance model that predicts the resource utilization (i.e., computation and communication complexity) for applications executing under the parallel-pipeline model on a homogeneous computational cluster. This performance prediction model can provide information to a cluster scheduler.

[1]  Susan T. Dumais,et al.  Latent Semantic Indexing (LSI) and TREC-2 , 1993, TREC.

[2]  Haym Hirsh,et al.  Using LSI for text classification in the presence of background text , 2001, CIKM '01.

[3]  Eric Brill,et al.  A Simple Rule-Based Part of Speech Tagger , 1992, HLT.

[4]  Yossi Matias,et al.  Can shared-memory model serve as a bridging model for parallel computation? , 1997, SPAA '97.

[5]  Susan T. Dumais,et al.  Using LSI for information filtering: TREC-3 experiments , 1995 .

[6]  Susan T. Dumais,et al.  LSI meets TREC: A Status Report , 1992, TREC.

[7]  Charles L. Seitz,et al.  Myrinet: A Gigabit-per-Second Local Area Network , 1995, IEEE Micro.

[8]  David A. Patterson,et al.  Computer Architecture: A Quantitative Approach , 1969 .

[9]  Brian D. Davison,et al.  Assessing the Impact of Sparsification on LSI Performance , 2004 .

[10]  William M. Pottenger,et al.  The role of associativity and commutativity in the detection and transformation of loop-level parallelism , 1998, ICS '98.

[11]  Yossi Matias,et al.  Can shared-memory model serve as a bridging model for parallel computation? , 1997, SPAA '97.

[12]  Joseph Mohan.  Performance of parallel programs , 1984 .

[13]  Paul D. Gader,et al.  Image algebra techniques for parallel image processing , 1987 .

[14]  Robert L. Grossman,et al.  Data Mining for Scientific and Engineering Applications , 2001, Massive Computing.

[15]  Sang Cheol Kim,et al.  Measurement and Prediction of Communication Delays in Myrinet Networks , 2001, J. Parallel Distributed Comput..

[16]  Leslie G. Valiant,et al.  A bridging model for parallel computation , 1990, CACM.

[17]  William M. Pottenger,et al.  Massively parallel distributed feature extraction in textual data mining using HDDI/sup TM/ , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[18]  Ramesh Subramonian,et al.  LogP: a practical model of parallel computation , 1996, CACM.

[19]  M. J. Quinn,et al.  Analytical performance prediction on multicomputers , 1993, Supercomputing '93.

[20]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[21]  Rajkumar Buyya,et al.  High Performance Cluster Computing: Programming and Applications , 1999 .

[22]  Mark M. Mathis,et al.  A performance model of non-deterministic particle transport on large-scale systems , 2003, Future Gener. Comput. Syst..

[23]  F JáJáJoseph,et al.  The Block Distributed Memory Model , 1996 .

[24]  David A. Patterson,et al.  Computer architecture (2nd ed.): a quantitative approach , 1996 .

[25]  Eric Brill,et al.  A corpus-based approach to language learning , 1993 .

[26]  J. Schopf,et al.  Structural Prediction Models for High-Performance Distributed Applications , 1997 .

[27]  William M. Pottenger.  Theory, techniques, and experiments in solving recurrences in computer programs , 1997 .

[28]  Rajkumar Buyya,et al.  High Performance Cluster Computing , 1999 .

[29]  Leonard Kleinrock.  On the modeling and analysis of computer networks , 1993 .

[30]  Eric Brill,et al.  Some Advances in Transformation-Based Part of Speech Tagging , 1994, AAAI.

[31]  Vikram S. Adve,et al.  Analyzing the behavior and performance of parallel programs , 1993 .

[32]  Stephen F. Lundstrom,et al.  Predicting Performance of Parallel Computations , 1990, IEEE Trans. Parallel Distributed Syst..

[33]  William M. Pottenger,et al.  The Role of the HDDI Collection Builder in Hierarchical Distributed Dynamic Indexing , 2004 .

[34]  Daniel A. Menascé,et al.  A Methodology for Performance Evaluation of Parallel Applications on Multiprocessors , 1992, J. Parallel Distributed Comput..

[35]  Ken Kennedy,et al.  Performance of parallel processors , 1989, Parallel Comput..

[36]  Chris J. Scheiman,et al.  LogGP: Incorporating Long Messages into the LogP Model for Parallel Computation , 1997, J. Parallel Distributed Comput..