Performance of On-Line Learning Methods in Predicting MultiprocessorMemory Access Patterns

Shared memory multiprocessors require reconfigurable interconnection networks (INs) for scalability. These INs are reconfigured by an IN control unit. However, these INs are often plagued by undesirable reconfiguration time that is primarily due to control latency, the amount of time delay that the control unit takes to decide on a desired new IN configuration. To reduce control latency, a trainable prediction unit (PU) was devised and added to the IN controller. The PU’s job is to anticipate and reduce control configuration time, the major component of the control latency. Three different on-line prediction techniques were tested to learn and predict repetitive memory access patterns for three typical parallel processing applications, the 2-D relaxation algorithm, matrix multiply and Fast Fourier Transform. The predictions were then used by a routing control algorithm to reduce control latency by configuring the IN to provide needed memory access paths before they were requested. Three prediction techniques were used and tested: 1). a Markov predictor, 2). a linear predictor and 3). a time delay neural network (TDNN) predictor. As expected, different predictors performed best on different applications, however, the TDNN produced the best overall results.

[1]  Donald M. Chiarulli,et al.  Online prediction of multiprocessor memory access patterns , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[2]  Alexander V. Veidenbaum,et al.  Compiler-directed data prefetching in multiprocessors with memory hierarchies , 1990 .

[3]  Kirk L. Johnson.  The impact of communication locality on large-scale multiprocessor performance , 1992, ISCA '92.

[4]  Bell Cg,et al.  Multis: a new class of multiprocessor computers. , 1985 .

[5]  Yoshiyasu Takefuji,et al.  Comparisons of Seven Neural Network Models on Traffic Control Problems in Multistage Interconnection Networks , 1993, IEEE Trans. Computers.

[6]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[7]  Marcus Dormanns,et al.  Partitioning and mapping of large FEM-graphs by self-organization , 1995, Proceedings Euromicro Workshop on Parallel and Distributed Processing.

[8]  C. Lee Giles,et al.  Using recurrent neural networks to learn the structure of interconnection networks , 1995, Neural Networks.

[9]  D M Chiarulli,et al.  Locality-based control algorithms for reconfigurable optical interconnection networks. , 1994, Applied optics.

[10]  Howard Jay Siegel.  Interconnection Network for Large-Scale Parallel Processing , 1990 .

[11]  Willard Korfhage,et al.  Process scheduling using genetic algorithms , 1995, Proceedings.Seventh IEEE Symposium on Parallel and Distributed Processing.

[12]  Howard Jay Siegel,et al.  Interconnection networks for large-scale parallel processing: theory and case studies (2nd ed.) , 1985 .

[13]  C. L. Giles,et al.  Routing in Optical Multistage Interconnection Networks: a Neural Network Solution UMIACS-TR-94-21 AN , 1994 .

[14]  J. E. Jensen,et al.  Neural network controller for adaptive routing in survivable communications networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[15]  Donald M. Chiarulli,et al.  Predictive control of opto-electronic reconfigurable interconnection networks using neural networks , 1995, Proceedings of Second International Workshop on Massively Parallel Processing Using Optical Interconnections.

[16]  C. G. Bell.  Multis: A New Class of Multiprocessor Computers , 1985, Science.

[17]  Franciszek Seredynski.  Dynamic mapping and load balancing with parallel genetic algorithms , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[18]  T. Fritsch,et al.  Communication network routing using neural nets-numerical aspects and alternative approaches , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.