Program partitioning and synchronization on multiprocessor systems

Since the mid 1970's, vector machines have dominated the supercomputer market. Because of technological limitations, faster circuits and more levels of pipelining of vector processors can no longer satisfy the increasing demand for high-speed computation. Multiprocessing problems in parallel is a natural trend. Five essential issues are identified in solving problems on multiprocessor systems: control structure, program partitioning, scheduling, synchronization, and memory access. The solutions of these problems determine the performance and efficiency of future multiprocessor machines. This thesis introduces new solutions for the synchronization and partitioning problems. The bit-map method synchronizes concurrent executing processes at the data level. Each synchronized data element has an attached sync field, and each synchronization memory operation contains a mask value. The data can be accessed only when the mask matches the sync. A proper referencing order for the data can be maintained. Two factors are considered when partitioning a program into processes executing in parallel: amount of parallelism and memory access and synchronization overhead. This thesis introduces a minimum distance method which partitions a recurrence loop into independent execution sets. This method uses the minimum dependence distance of each dimension of all dependence cycles to divide the index set of the loop into independent partitions. When a loop does not have a sufficient number of independent sets, the block and the interleaved methods partition the loop using a proper synchronization mechanism. A programmer assistance tool helps programmers in using these partitioning and synchronization methods. A simulator in the tool compares different partitioning strategies. This programmer assistance methodology allows users to explore several algorithms and select one which fits the appropriate multiprocessor architecture.