Performance based design and analysis of multimicrocomputer networks

A new paradigm for parallel computation based on large numbers of interconnected microcomputer nodes has recently emerged. Each network node, fabricated as one or two VLSI chips would contain a processor with some local memory, a communication controller that routes messages without delaying the processor, and a few connections to other network nodes. The cooperating tasks of a parallel algorithm would execute asynchronously on different nodes and communicate via message passing. This approach to parallel processing poses several new and interesting problems in network performance evaluation, distributed task scheduling, and parallel algorithm design. The absence of shared memory makes the evaluation and design of an interconnection network capable of efficiently supporting internode communication patterns crucial. A network model based on the asymptotic properties of closed queueing networks representing the effects of the network topology, node and communication link speeds, and the internode communication patterns is developed. With this model, it is possible to compare the performance of different network topologies processing the same workload, determine the range of network sizes over which a given topology can meet specified performance requirements, and calculate the size of computation quanta below which communication delays negate possible gains due to increased parallelism. Because of communication delays, no node can possess an exact description of the entire network state; all scheduling decisions must be made using incomplete and possibly inaccurate status information. The efficacy of distributed scheduling heuristics as a function of network topology, status information accuracy, and the amount of computation represented by each task are examined. Knowledge of a small area surrounding each node is shown to be sufficient to make acceptable scheduling decisions. Finally, the importance of partial differential equations as models of many phenomena has motivated the search for solution algorithms suited to multimicrocomputer networks. This work has sought to relax the severe synchronization constraints imposed by most algorithms and to determine an appropriate number of discretization points to place in each network node. This has led to a class of solution algorithms spanning the spectrum from completely sequential and synchronous to completely parallel and fully asynchronous.