Simulating modular neural networks on message-passing multiprocessors

While simulation programs for single neural networks, running on parallel machines, always use a fixed problem decomposition and mapping strategy, we show that this is not possible for modular neural networks. We demonstrate this by analysing decomposition and mapping issues for a particular modular neural network model: the entropy-driven artificial neural network. The classic approach to simulations consists of two steps: first a data structure is built, describing the problem to be simulated. For a neural network this data structure contains the network topology, the interconnection strengths, etc. In a second step, this data structure is read into the simulation program, which performs a fixed decomposition and mapping before simulation can take place. Since this approach cannot be used any more for simulations of modular networks, we propose a new, three-step approach, in which decomposition and mapping are taken out of the simulation program. A compiler is used to prepare the problem data structure, a splitter program takes care of problem decomposition, and the simulator program takes the decomposed problem as its input. Since all decisions with respect to decomposition and mapping are taken by the splitter, the simulator program is independent of decomposition and mapping, and hence it can handle any decomposition and mapping. Following this approach, a machine-independent simulation environment was designed, and this design was implemented on a transputer system. To show that our approach is generic (i.e. not limited to simulations of modular networks) an implementation of a Hopfield network for image restoration is described. In spite of the classic preconceptions about generic software, performance analysis and benchmark results show that our novel, generic approach can be implemented efficiently on transputer arrays.