A HYBrid symbolic-connectionist processor of natural language semantic relations

In the field of Natural Language Processing (NLP), there are symbolic and connectionist approaches to account for semantic issues, such as the thematic role relationships between sentence constituents. A “hybrid” option merges both methods: a symbolic thematic theory is used to supply the connectionist network with initial knowledge. This way, benefits of connectionism, such as learning, generalization and fault tolerance are combined with representational symbolic features. A symbolic-connectionist hybrid system called HYBãPRED (HYBrid symbolic-connectionist thematic (ã) PREDictor) is proposed here. Its main purpose is to reveal the thematic grid assigned to a sentence. The connectionist architecture comprises, as input, a featural representation of the words (based on the verb/noun WordNet classification and on the classical semantic microfeature representation), and, as output, the thematic grid assigned to that sentence. HYBãPRED “predicts” thematic (semantic) roles assigned to words in a sentence context, adopting a psycholinguistic view of thematic theory.

[1]  Randall C. O'Reilly,et al.  Biologically Plausible Error-Driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm , 1996, Neural Computation.

[2]  Todd R. Ferretti,et al.  Thematic Roles as Verb-specific Concepts , 1997 .

[3]  Krysia Broda,et al.  Symbolic knowledge extraction from trained neural networks: A sound approach , 2001, Artif. Intell..

[4]  James L. McClelland,et al.  Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences , 1986 .

[5]  João Luís Garcia Rosa,et al.  Hybrid Thematic Role Processor: Symbolic Linguistic Relations Revised by Connectionist Learning , 1999, IJCAI.

[6]  Geoffrey E. Hinton,et al.  Learning Representations by Recirculation , 1987, NIPS.

[7]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[8]  Joydeep Ghosh,et al.  Symbolic Interpretation of Artificial Neural Networks , 1999, IEEE Trans. Knowl. Data Eng..

[9]  Vasant Honavar,et al.  Integrating Symbol Processing Systems and Connectionist Networks , 1995 .

[10]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation (3rd Edition) , 2007 .

[11]  Francis Crick,et al.  The recent excitement about neural networks , 1989, Nature.

[12]  Li-Min Fu,et al.  Knowledge-based connectionism for revising domain theories , 1993, IEEE Trans. Syst. Man Cybern..

[13]  James H. Martin,et al.  Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , 2000 .

[14]  Samuel W. K. Chan,et al.  Symbolic connectionism in natural language disambiguation , 1998, IEEE Trans. Neural Networks.

[15]  Christiane Fellbaum,et al.  English Verbs as a Semantic Net , 1990 .

[16]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[17]  Jude W. Shavlik,et al.  Extracting refined rules from knowledge-based neural networks , 2004, Machine Learning.

[18]  Lawrence O. Hall,et al.  A hybrid/symbolic connectionist production system , 1992, Proceedings Fourth International Conference on Tools with Artificial Intelligence TAI '92.

[19]  Ajay N. Jain,et al.  Parsing Complex Sentences with Structured Connectionist Networks , 1991, Neural Computation.

[20]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[21]  James L. McClelland,et al.  Sentence comprehension: A parallel distributed processing approach , 1989, Language and Cognitive Processes.

[22]  Noam Chomsky.  Lectures on Government and Binding: The Pisa Lectures , 1993 .

[23]  Huan Liu,et al.  Symbolic Representation of Neural Networks , 1996, Computer.

[24]  Marvin Oliver Schneider,et al.  Application and development of biologically plausible neural networks in a multiagent artificial life system , 2009, Neural Computing and Applications.

[25]  João Luís Garcia Rosa,et al.  A Connectionist Thematic Grid Predictor for Pre-parsed Natural Language Sentences , 2007, ISNN.

[26]  Risto Miikkulainen,et al.  Subsymbolic Case-Role Analysis of Sentences With Embedded Clauses , 1993, Cogn. Sci..

[27]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[28]  Jude W. Shavlik,et al.  Combining Symbolic and Neural Learning , 1994, Machine Learning.

[29]  C. Lee Giles,et al.  Rule Revision With Recurrent Neural Networks , 1996, IEEE Trans. Knowl. Data Eng..

[30]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[31]  Janyce Wiebe,et al.  Empirical acquisition of conceptual distinctions via dictionary definitions , 2005 .

[32]  João Luís Garcia Rosa,et al.  SABIO: A BIOLOGICALLY PLAUSIBLE CONNECTIONIST APPROACH TO AUTOMATIC TEXT SUMMARIZATION , 2008, Appl. Artif. Intell..

[33]  Liliane Haegeman,et al.  Introduction to Government and Binding Theory , 1991 .

[34]  George A. Miller,et al.  Nouns in WordNet: A Lexical Inheritance System , 1990 .

[35]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .