Symbolic rule extraction with a scaled conjugate gradient version of CLARION

This paper presents a hybrid intelligent system made up of two modules. The bottom sub-symbolic module is a multi-layer feed-forward neural network trained by a modified Q-learning methodology that employs the scaled conjugate gradient algorithm. The top module is a symbolic system (implemented with a neural network built on-line) where rules are extracted from the bottom module during training, in a fashion similar to the CLARION system. The two modules augment each other in an effort to obtain a better performance than both of the modules acting alone in solving a problem. The originality of this work lies in the use of the advanced scaled conjugate learning algorithm in such a hybrid system. It is expected that the use of this algorithm would provide significant improvements in the performance of the overall system and also make it less dependent on user-selected parameters. This paper emphasises the implementation details, since the system is currently under development, rather that concrete experimental results.

[1]  Farid U. Dowla,et al.  Backpropagation Learning for Multilayer Feed-Forward Neural Networks Using the Conjugate Gradient Method , 1991, Int. J. Neural Syst..

[2]  Ron Sun,et al.  Autonomous learning of sequential tasks: experiments and analyses , 1998, IEEE Trans. Neural Networks.

[3]  C. Charalambous,et al.  Conjugate gradient algorithm for efficient training of artifi-cial neural networks , 1990 .

[4]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[5]  Stefan Wermter,et al.  Hybrid neural systems: from simple coupling to fully integrated neural networks , 1999 .

[6]  Stefan Wermter,et al.  An Overview of Hybrid Neural Systems , 1998, Hybrid Neural Systems.

[7]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[8]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[9]  M. Møller A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1990 .

[10]  Melanie Hilario,et al.  An Overview of Strategies for Neurosymbolic Integration , 1995 .

[11]  Andreas Stafylopatis,et al.  The impact of the error function selection in neural network-based classifiers , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[12]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[13]  Etienne Barnard,et al.  Optimization for training neural nets , 1992, IEEE Trans. Neural Networks.