Techniques for Improving the Performance of Parallel Computations

Developing parallel implementations of applications which utilise an acceptably large fraction of the peak performance of current high performance computers has proved a di cult task. The lack of success in this endeavour is perceived as a major impediment to the general acceptance of high performance computing in industry. Even for structured, `static', scienti c and engineering applications coded in FORTRAN, where performance is apparently predictable, success has been limited. This Thesis argues that, while the development of high performance applications for parallel systems remains an experimental task suitable only for the expert programmer, systematic techniques, which maximise the bene t of programmer e ort, can be employed in order to develop `good' parallel implementations rapidly. A framework for such a method is presented, and a set of supporting techniques is developed, by means of a series of examples on the Kendall Square Research, KSR1. The method requires the achieved performance of an implementation to be described in terms of an `ideal' parallel performance, plus a small number of (parallel) overhead terms. Once the magnitude of each overhead term has been quanti ed, a systematic, iterative, process of overhead minimisation can take place. The source of each targeted overhead is analysed, and an alternative implementation, which reduces the overhead, is developed. Analysis of the overheads requires a mixture of experiment and modelling. 7 Declaration No portion of the work referred to in this thesis has been submitted in support of an application for another degree or quali cation of this or any other university or other institution of learning. 8 Copyright Copyright in text of this thesis rests with the Author. Copies (by any process) either in full, or of extracts, may be made only in accordance with instructions given by the Author and lodged in the John Rylands University Library of Manchester. Details may be obtained from the Librarian. This page must form part of any such copies made. Further copies (by any process) of copies made in accordance with such instructions may not be made without the permission (in writing) of the Author. The ownership of any intellectual property rights which may be described in this thesis is vested in the University of Manchester, subject to any prior agreement to the contrary, and may not be made available for use by third parties without the written permission of the University, which will prescribe the terms and conditions of any such agreement. Further information on the conditions under which disclosures and exploitation may take place is available from the head of Department of Computer Science. 9 Education and Research The author graduated from the University of Manchester in 1978 with a BSc(Hons) in Physics. After obtaining a PGCE from Christ College Liverpool, and completing his probationary teaching year, he moved into the area of real-time systems simulation in industry, rst with Redifusion Flight Simulation in Crawley, then with Ferranti Computer Systems in Cheadle, Manchester, before joining the Centre for Novel Computing in the Department of Computer Science at the University of Manchester in 1990. 10 Acknowledgements I would like to thank the following| CNC people past and present for the stimulating tea breaks, in particular Mark, Rupert, Rob and Andy. Also John, for both the opportunity and his patience; Mum and Dad for all their support and encouragement through the years; and nally Pat, for making it all worthwhile. 11 Chapter