Using genetic algorithms for sparse distributed memory initialization

We describe the use of genetic algorithms to initialize a set of hard locations that constitutes the storage space of Sparse Distributed Memory (SDM). SDM is an associative memory technique that uses binary spaces, and relies on close memory items tending to be clustered together, with some level of abstraction. An important factor in the physical implementation of SDM is how many hard locations are used, which greatly affects the memory capacity. It is also dependent on the dimension of the binary space used. For the SDM system to function appropriately, the hard locations should be uniformly distributed over the binary space. We represented a set of hard locations of SDM as population members, and employed GA to search for the best (fittest) distribution of hard locations over the vast binary space. Accordingly, fitness is based on how far each hard location is from all other hard locations, which measures the uniformity of the distribution. The preliminary results are very promising, with the GA significantly outperforming random initialization used in most existing SDM implementations. This use of GA, which is similar to the Michigan approach, differs from the standard approach in that the object of the search is the entire population.

[1]  John J. Grefenstette,et al.  A Parallel Genetic Algorithm , 1987, ICGA.

[2]  Richard W. Prager,et al.  The modified Kanerva model for automatic speech recognition , 1989 .

[3]  Larry M. Manevitz,et al.  Assigning meaning to data: Using sparse distributed memory for multilevel cognitive tasks , 1997, Neurocomputing.

[4]  Robert L. Brown Two Demonstrators and a Simulator for a Sparse, Distributed Memory , 1987 .

[5]  Dipankar Dasgupta,et al.  Metacognition in Software Agents Using Classifier Systems , 1998, AAAI/IAAI.

[6]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms in Engineering Applications , 1997, Springer Berlin Heidelberg.

[7]  Arthur C. Graesser,et al.  Natural language sensing for autonomous agents , 1998, Proceedings. IEEE International Joint Symposia on Intelligence and Systems (Cat. No.98EX174).

[8]  David R. Jefferson,et al.  Selection in Massively Parallel Genetic Algorithms , 1991, ICGA.

[9]  James D. Keeler,et al.  Capacity for Patterns and Sequences in Kanerva's SDM as Compared to Other Associative Memory Models , 1987, NIPS.

[10]  Jukka Saarinen,et al.  VLSI architectures of sparse distributed memory , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[11]  Shumeet Baluja,et al.  A Massively Distributed Parallel Genetic Algorithm , 1992 .

[12]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[13]  Pentti Kanerva A cerebellar-model associative memory as a generalized random-access memory , 1989, Digest of Papers. COMPCON Spring 89. Thirty-Fourth IEEE Computer Society International Conference: Intellectual Leverage.

[14]  John J. Grefenstette Predictive Models Using Fitness Distributions of Genetic Operators , 1994, FOGA.

[15]  David Willshaw,et al.  Ceded Memories: Sparse distributed memory by Pentti Kanerva. Cambridge, Mass.: M.I.T. Press, 1988, 176 pages, ISBN 0 262 11132 2, £17.95 , 1990 .

[16]  Roland Karlsson Evaluation of a Fast Activation Mechanism for the Kanerva SDM Memory , 1995 .

[17]  Günther Görz,et al.  Kanerva's Sparse Distributed Memory: An Object-Oriented Implementation on the Connection Machine , 1995, IJCAI.

[18]  Bernard Manderick,et al.  Fine-Grained Parallel Genetic Algorithms , 1989, ICGA.

[19]  John J. Grefenstette,et al.  Credit Assignment in Genetic Learning Systems , 1988, AAAI.

[20]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[21]  Chris R. Fuller,et al.  Sparse distributed associative memory for the identification of aerospace acoustic sources , 1993 .

[22]  Timo Hämäläinen,et al.  Parallel realizations of Kanerva's sparse distributed memory on a tree-shaped computer , 1997, Concurr. Pract. Exp..

[23]  Douglas G. Danforth An empirical investigation of sparse distributed memory using discrete speech recognition , 1990 .

[24]  David Rogers Kanerva's Sparse Distributed Memory: an Associative Memory Algorithm Well-Suited to the Connection Machine , 1989, Int. J. High Speed Comput..

[25]  Bernard Manderick,et al.  A Massively Parallel Genetic Algorithm: Implementation and First Analysis , 1991, ICGA.

[26]  J. David Schaffer,et al.  Multi-Objective Learning via Genetic Algorithms , 1985, IJCAI.

[27]  S. Franklin,et al.  An Architecture for Emotion , 1998 .

[28]  David Rogers Using data tagging to improve the performance of Kanerva's sparse distributed memory , 1988 .

[29]  Michael J. Flynn,et al.  Sparse distributed memory prototype: principles and operation , 1989 .

[30]  Arthur C. Graesser,et al.  Is it an Agent, or Just a Program?: A Taxonomy for Autonomous Agents , 1996, ATAL.

[31]  David Rogers Statistical Prediction with Kanerva's Sparse Distributed Memory , 1988, NIPS.

[32]  S.-S. Chen,et al.  Character recognition in a sparse distributed memory , 1991, IEEE Trans. Syst. Man Cybern..

[33]  Pattie Maes,et al.  Learning Navigational Behaviors using a Predictive Sparse Distributed Memory , 1996 .

[34]  Pentti Kanerva The organization of an autonomous learning system , 1988 .

[35]  Jukka Saarinen,et al.  Parallel realizations of Kanerva's sparse distributed memory on a tree-shaped computer , 1997, Concurr. Pract. Exp..

[36]  Kuo-Chin Fan,et al.  A genetic sparse distributed memory approach to the application of handwritten character recognition , 1997, Pattern Recognit..

[37]  Lee McCauley,et al.  IDA: a cognitive agent architecture , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[38]  Pentti Kanerva,et al.  Sparse distributed memory and related models , 1993 .

[39]  Umesh Dwarkanath Joglekar Learning to read aloud: A neural network approach using sparse distributed memory , 1989 .

[40]  John H. Andreae,et al.  Improving the performance of Kanerva's associate memory , 1995, IEEE Trans. Neural Networks.

[41]  J. Keeler Comparison Between Kanerva's SDM and Hopfield-Type Neural Networks , 1988, Cogn. Sci..

[42]  Kimmo Kaski,et al.  Highly Parallel Realization of Sparse Distributed Memory System , 1991, The Sixth Distributed Memory Computing Conference, 1991. Proceedings.

[43]  Kimmo Kaski,et al.  Kanerva's sparse distributed memory with multiple hamming thresholds , 1992 .

[44]  Alvin J. Surkan,et al.  Relating number of processing elements in a sparse distributed memory model to learning rate and generalization , 1991, APL '91.

[45]  Gillian M. Hayes,et al.  A new approach to Kanerva's sparse distributed memory , 1997, IEEE Trans. Neural Networks.

[46]  Louis A. Jaeckel A class of designs for a sparse distributed memory , 1989 .

[47]  Stan Franklin,et al.  Autonomous Agents as Embodied Ai , 1997, Cybern. Syst..

[48]  A. Glenberg,et al.  What memory is for: Creating meaning in the service of action , 1997, Behavioral and Brain Sciences.

[49]  Pentti Kanerva,et al.  Sparse Distributed Memory , 1988 .

[50]  Heinz Mühlenbein,et al.  The parallel genetic algorithm as function optimizer , 1991, Parallel Comput..

[51]  John J. Grefenstette,et al.  Multilevel Credit Assignment in a Genetic Learning System , 1987, International Conference on Genetic Algorithms.