Giving meaning to cycles to go beyond the limitations of fixed point attractors

This chapter focuses on associativememories in recurrent artificial neural networks using the same kind of very simple neurons usually found in neural nets. The past 25 years have dedicated much of the research endeavor on coding the information in fixed point attractors. From a cognitive or neurophysiological point of view, this choice is rather arbitrary. This paper justifies the need to switch to another encoding mechanism exploiting limit cycles and complex dynamics in the background rather than fixed points. It is shown how these attractors encompass in many aspects the limitations of fixed points: better correspondence with neurophysiological facts, increase of the encoding capacity, improved robustness during the retrieval phase, decrease in the number of spurious attractors. However, how to exploit and learn these cycles for encoding the relevant information is still an open issue. In this paper two learning strategies are proposed, tested and compared, one rather classical, very reminiscent of the usual supervised hebbian learning, the other one, rather original since allowing the coding attractor to be chosen by the network itself. Computer experiments of these two learning strategies will be presented and explained. The second learning mechanism will be advocated both for its highly cognitive relevance and on account of its much better performance in encoding and retrieving the information. Since the kind of dynamics observed in our experiments (cyclic attractor when a stimulus is presented and a weak background chaos in the absence of such stimulation) faithfully reminds neurophysiological data and although no straightforward applications have been found so far, we limit our justification to this qualitative mapping with brain observations and the need to better explore how a physical device such as a brain can store and retrieve in a robust way a huge quantity of information.