Nonlinear dynamics of artificial neural systems

Now that significant progress has been made in developing algorithms for training hidden units, we suggest that it is time to reevaluate the nonlinear discriminate approach, which once fell into disfavor due to the problem of proliferation of high order terms. We show that there are many powerful techniques for reducing the number of spurious terms, and that the high order approach has many advantages over a cascaded slab approach in certain problem areas. Advantages include increased expressive ability, decreased architectural complexity, and dramatically increased learning rates.