Feature extraction in shared weights neural networksDick

Since 1989 there have been numerous reports about shared weights neural networks: feed-forward networks performing very well in image analysis problems. As an explanation of these results, the authors of the corresponding papers propose that the networks somehow extract local features from the image, combining them at deeper levels in the network. Experiments on a large real-life dataset of handwritten digits show these networks to be able to achieve a low test set error. However, the lters or templates implemented by trained networks are uninterpretable and do not conform to conventional approaches in the image processing eld. To study the feature extraction process, we trained a simple network on a small 2-class digit dataset. The results indicate that a comparison between these networks and traditional image processing techniques is not straightforward, since the two are based on diierent mechanisms and goals.