On the Information Capacity of Auto-associative RAM-based Neural Networks

The network studied in this work is based on the idea of a General Neural Unit (GNU), which can be configured as a feed-forward network for pattern classification or as a feedback system with associative memory properties [1]. Storing patterns in a GNU in the auto-associative mode consists on creating re-entrant states in which the network stabilises in the retrieval phase as a function of the excitation pattern. When the system is not fully interconnected, a collision may occur between patterns ξr and ξs if, after storing ξr storing ξs makes ξr no longer stable. Collisions cause the formation of dynamic attractors instead of fixed and stable attractors. The cycle size of the dynamic attractor formed increases exponentially with the number of nodes where collisions occurred [2]. Therefore, the prediction of the expected number of collisions is an important issue when estimating the storage capacity [3]. A statistical approach to estimate the number of collisions of such networks is presented, allowing the prediction of the information capacity as a function of the main parameters of the system.