Construction Principles for Efficient Parallel Structures

This chapter provides an overview of the examination and comparison of network structures. It discusses the variety of possible types of parallelism, possible sources of speedup from parallelism and hardware and software design, and the range of possibilities between tightly and loosely coupled networks. It also presents some applications of graph theory that have produced a number of interesting new possibilities for networks. These graph theory results are narrowly focused on one or two formal global characteristics, chiefly the density of packing of nodes. But local structural characteristics, in particular a close fit of the anatomy of the network to the physiology of the program's flow of procedures, are almost certainly of overriding importance. In theory, a single system might incorporate virtually every type of parallelism. However, that would be too expensive in practice and overall efficiency would probably not be increased by simply adding all together.