Self-Organised Modular Neural Networks for Encoding Data

It is shown how a neural network can be optimised so that multiple interlinked network modules emerge by self-organisation. The processing task chosen to illustrate this is encoding high-dimensional data, such as images, where multiple network modules implement a factorial encoder, in which the high-dimensional data space is broken up into a number of lowdimensional subspaces, each of which is separately encoded. This type of factorial encoder emerges through a process of self-organisation, provided that the input data lies on a curved manifold, as is indeed the case in image processing applications.