Product Units with Trainable Exponents and Multi-Layer Networks

This chapter reviews and examines a variant type of computational unit which we have recently proposed for use in multi-layer neural networks [3]. Instead of the output of this unit depending on a weighted sum of the inputs, it depends on a weighted product. In justifying the introduction of a new type of unit we explore at some length the rationale behind the use of multi-layer neural networks, and the properties of the computational units within them. At the end of the chapter we discuss a biological model for a single complex neve cell with active dendritic membrane that uses the product units.