Data-Driven Theory Refinement Using KBDistAl

Knowledge based artificial neural networks offer an attractive approach to extending or modifying incomplete knowledge bases or domain theories through a process of data-driven theory refinement. We present an efficient algorithm for data-driven knowledge discovery and theory refinement using DistAl, a novel (inter-pattern distance based, polynomial time) constructive neural network learning algorithm. The initial domain theory comprising of propositional rules is translated into a knowledge based network. The domain theory is modified using DistAl which adds new neurons to the existing network as needed to reduce classification errors associated with the incomplete domain theory on labeled training examples. The proposed algorithm is capable of handling patterns represented using binary, nominal, as well as numeric (real-valued) attributes. Results of experiments on several datasets for financial advisor and the human genome project indicate that the performance of the proposed algorithm compares quite favorably with other algorithms for connectionist theory refinement (including those that require substantially more computational resources) both in terms of generalization accuracy and network size.

[1]  Zoran Obradovic,et al.  Combining Prior Symbolic Knowledge and Constructive Neural Network Learning , 1993 .

[2]  Vasant Honavar,et al.  Generative learning structures and processes for generalized connectionist networks , 1993, Inf. Sci..

[3]  Raju S. Bapi,et al.  Artificial intelligence and neural networks: Steps toward principled integration , 1996 .

[4]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.

[5]  Jude Shavlik,et al.  Refinement ofApproximate Domain Theories by Knowledge-Based Neural Networks , 1990, AAAI.

[6]  Pat Langley,et al.  Elements of Machine Learning , 1995 .

[7]  Enrico Gobbetti,et al.  Encyclopedia of Electrical and Electronics Engineering , 1999 .

[8]  Vasant Honavar,et al.  Books-Received - Artificial Intelligence and Neural Networks - Steps Toward Principled Integration , 1994 .

[9]  David W. Opitz,et al.  Dynamically adding symbolically meaningful nodes to knowledge-based neural networks , 1995, Knowl. Based Syst..

[10]  Munindar P. Singh,et al.  Agents on the Web: Mobile Agents , 1997, IEEE Internet Comput..

[11]  Vasant Honavar,et al.  Generative learning structures for generalized connectionist networks , 1990 .

[12]  M.H. Hassoun,et al.  Fundamentals of Artificial Neural Networks , 1996, Proceedings of the IEEE.

[13]  Eric B. Baum,et al.  Constructing Hidden Units Using Examples and Queries , 1990, NIPS.

[14]  DistAl: An inter-pattern distance-based constructive learning algorithm , 1999, Intell. Data Anal..

[15]  Vasant Honavar.  Machine learning: Principles and applications , 1999 .

[16]  Jihoon Yang,et al.  Constructive Neural Network Learning Algorithms , 1996, AAAI/IAAI, Vol. 2.

[17]  Dekang Lin,et al.  An Information-Theoretic Definition of Similarity , 1998, ICML.

[18]  Raymond J. Mooney,et al.  Comparing Methods for Refining Certainty-Factor Rule-Bases , 1994, ICML.

[19]  David W. Opitz,et al.  Connectionist Theory Refinement: Genetically Searching the Space of Network Topologies , 1997, J. Artif. Intell. Res..

[20]  Allen Ginsberg,et al.  Theory Reduction, Theory Revision, and Retranslation , 1990, AAAI.

[21]  Raymond J. Mooney,et al.  Theory Refinement Combining Analytical and Empirical Methods , 1994, Artif. Intell..

[22]  Jude Shavlik,et al.  A Framework for Combining Symbolic and Neural Learning , 1992 .

[23]  Larry A. Rendell,et al.  Rerepresenting and Restructuring Domain Theories: A Constructive Induction Approach , 1994, J. Artif. Intell. Res..

[24]  George F. Luger,et al.  Artificial Intelligence and the Design of Expert Systems , 1990 .

[25]  Vasant Honavar,et al.  Distributed knowledge networks , 1998, 1998 IEEE Information Technology Conference, Information Environment for the Future (Cat. No.98EX228).

[26]  I. Kononenko,et al.  INDUCTION OF DECISION TREES USING RELIEFF , 1995 .

[27]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[28]  Ronen Feldman,et al.  Bias-Driven Revision of Logical Domain Theories , 1993, J. Artif. Intell. Res..

[29]  J. Shavlik,et al.  Re nement of Approximate Domain Theories byKnowledge-Based Neural Networks , 1990 .

[30]  Jihoon Yang,et al.  DistAl: an inter-pattern distance-based constructive learning algorithm , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[31]  Li-Min Fu,et al.  Knowledge-based connectionism for revising domain theories , 1993, IEEE Trans. Syst. Man Cybern..

[32]  Rajesh Parekh,et al.  Constructive Theory Reenement in Knowledge Based Neural Networks , 1998 .

[33]  Raymond J. Mooney,et al.  Automated refinement of first-order horn-clause domain theories , 2005, Machine Learning.

[34]  Mark Craven,et al.  Extracting comprehensible models from trained neural networks , 1996 .

[35]  Rajesh Parekh,et al.  Constructive theory refinement in knowledge based neural networks , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[36]  Fu,et al.  Integration of neural heuristics into knowledge-based inference , 1989 .

[37]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[38]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[39]  Michael R. Genesereth,et al.  Software agents , 1994, CACM.

[40]  Sebastian Thrun,et al.  Lifelong Learning: A Case Study. , 1995 .