Combining unsupervised classifiers : a multimodal case study

Neural computing systems are trained on the principle that if a network can compute then it will learn to compute. Multi-net neural computing systems are trained on the principle that if two or more networks compute, then the multi-net will learn to compute. In this thesis I will present a novel multi-net systems that can classify a set of objects using different sets of attributes of the objects: each set is used in the training of each unsupervised self-organised network and a Hebbian network simultaneously learns the association of the most highly active neurons in the constituent unsupervised networks of the multi-net system. The performance of the multi-net system is compared with a single net system trained with a single complex vector. The comparison suggests that for certain tasks, particularly classification, the multi-net system has a better retrieval effectiveness than the monolithic single net system. Furthermore, we demonstrate the effectiveness of using conventional statistical clustering techniques, especially k-means and hierarchical clustering techniques, in clustering the output map of an unsupervised network. Such sequential clustering, that is first clustering using the output map of an unsupervised network and then clustering the output map, facilitates in automatically clustering and in visualising the clusters that are otherwise implicit in the output map. One important use of our multi-net architecture is in using a partial query that activates some of the neurons in only one constituent unsupervised network and the Hebbian link then activates neurons in the other unsupervised network. I have trained a multi-net systems and a single net system on a collection of images, and associated collateral texts describing the contents of the image. Different training regimens were used, by changing the topology of the unsupervised network, specifically Kohonen's self-organising feature map, and the number of training cycles. Three testing regimens were developed for evaluating the performance of the multi-net system. The multi-net system can be used in the automatic annotation of images and the automatic illustration of keywords. The multi-net approach was also tried on a set of "real-world" images, images used in training of scene of crime officers: the initial results, especially the comparison between single net neural networks and the multi-net system suggest that the multi-net is a better classifier. Our approach is relevant to the needs of workers in the fields of multi-net community on the one hand and to needs of workers in multi-modal data fusion on the other.