Training Recurrent Nets of Hardware Realisable Sigma-PI Units

The dynamics and training of recurrent nets which make use of probabilistic nodes based on Boolean functions are explored. These are shown to be equivalent to sigma-pi units when using binary inputs and have the advantage of an immediate implementation in readily available digital hardware. The underlying hypercube structure of the Boolean address space is used to describe the process of generalisation and also sheds light on the role of hidden units. This is made more precise with the aid of the concept of hypercube order statistics. Training algorithms are described which make use of a node time-integration apparatus to promote near optimal cube structures in a natural way.