Biologically plausible connectionist prediction of natural language thematic relations

In Natural Language Processing (NLP) symbolic systems, several linguistic phenomena, for instance, the thematic role relationships between sentence constituents, such as AGENT and PATIENT, can be accounted for by the employment of a rule-based grammar. Another approach to NLP concerns the use of the connectionist model, which has the benefits of learning, generalization and fault tolerance, among others. Inspired on neuroscience, it is proposed a connectionist system called BIOθPRED (BIOlogically plausible thematic (θ) PREDictor), designed to reveal the thematic grid assigned to a sentence. Its architecture comprises, as input, a featural representation of the words (based on the verb/noun WordNet classification and on the classical semantic microfeature representation), and, as output, the thematic grid assigned to the sentence. BIOθPRED is designed to “predict” thematic (semantic) roles assigned to words in a sentence context, employing biologically inspired training algorithm and architecture, and adopting a psycholinguistic view of thematic theory.

[1]  A. Alishahi,et al.  Please Scroll down for Article Language and Cognitive Processes a Computational Model of Learning Semantic Roles from Child-directed Language a Computational Model of Learning Semantic Roles from Child-directed Language , 2022 .

[2]  Carlo Cecchetto,et al.  Introduction to Government and Binding Theory , 1996 .

[3]  Geoffrey E. Hinton,et al.  Learning distributed representations of concepts. , 1989 .

[4]  James L. McClelland,et al.  Sentence comprehension: A parallel distributed processing approach , 1989, Language and Cognitive Processes.

[5]  D. Caplan,et al.  The role of animacy and thematic relationships in processing active English sentences: Evidence from event-related potentials , 2007, Brain and Language.

[6]  Douglas L. T. Rohde,et al.  Language acquisition in the absence of explicit negative evidence: how important is starting small? , 1999, Cognition.

[7]  João Luís Garcia Rosa,et al.  A biologically inspired connectionist system for natural language processing , 2002, VII Brazilian Symposium on Neural Networks, 2002. SBRN 2002. Proceedings..

[8]  Geoffrey E. Hinton.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1991 .

[9]  Risto Miikkulainen.  Subsymbolic Case-Role Analysis of Sentences with Embedded Clauses , 1993 .

[10]  Noam Chomsky.  Knowledge of language: its nature, origin, and use , 1988 .

[11]  Noam Chomsky.  Lectures on Government and Binding: The Pisa Lectures , 1993 .

[12]  Randall C. O'Reilly,et al.  Biologically Plausible Error-Driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm , 1996, Neural Computation.

[13]  Todd R. Ferretti,et al.  Thematic Roles as Verb-specific Concepts , 1997 .

[14]  Stuart C. Shapiro.  Review of Knowledge representation: logical, philosophical, and computational foundations by John F. Sowa. Brooks/Cole 2000. , 2001 .

[15]  Alexander H. Waibel,et al.  Incremental Parsing by Modular Recurrent Connectionist Networks , 1989, NIPS.

[16]  C. Lee Giles,et al.  Rule Revision With Recurrent Neural Networks , 1996, IEEE Trans. Knowl. Data Eng..

[17]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[18]  Janyce Wiebe,et al.  Empirical acquisition of conceptual distinctions via dictionary definitions , 2005 .

[19]  Risto Miikkulainen,et al.  Subsymbolic natural language processing - an integrated model of scripts, lexicon, and memory , 1993, Neural network modeling and connectionism.

[20]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[21]  João Luís Garcia Rosa,et al.  Linguistic Relations Encoding in a Symbolic-Connectionist Hybrid Natural Language Processor , 2000, IBERAMIA-SBIA.

[22]  Jude W. Shavlik,et al.  Combining Symbolic and Neural Learning , 1994, Machine Learning.

[23]  Risto Miikkulainen,et al.  Natural Language Processing With Modular PDP Networks and Distributed Lexicon , 1991, Cogn. Sci..

[24]  Krysia Broda,et al.  Symbolic knowledge extraction from trained neural networks: A sound approach , 2001, Artif. Intell..

[25]  Huan Liu,et al.  Symbolic Representation of Neural Networks , 1996, Computer.

[26]  James L. McClelland,et al.  Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences , 1986 .

[27]  João Luís Garcia Rosa,et al.  Hybrid Thematic Role Processor: Symbolic Linguistic Relations Revised by Connectionist Learning , 1999, IJCAI.

[28]  A. Andrews.  The major functions of the noun phrase , 2007 .

[29]  Geoffrey E. Hinton,et al.  Learning Representations by Recirculation , 1987, NIPS.

[30]  Tim van Gelder,et al.  Defining ‘Distributed Representation’ , 1992 .

[31]  Jordan B. Pollack,et al.  Massively Parallel Parsing: A Strongly Interactive Model of Natural Language Interpretation , 1988, Cogn. Sci..

[32]  Li-Min Fu,et al.  Knowledge-based connectionism for revising domain theories , 1993, IEEE Trans. Syst. Man Cybern..

[33]  Vasant Honavar,et al.  Integrating Symbol Processing Systems and Connectionist Networks , 1995 .

[34]  João Luís Garcia Rosa,et al.  A HYBrid symbolic-connectionist processor of natural language semantic relations , 2009, 2009 IEEE Workshop on Hybrid Intelligent Models and Applications.

[35]  Juan Manuel Adán Coello,et al.  Biologically Plausible Connectionist Prediction of Natural Language Thematic Relations , 2010, J. Univers. Comput. Sci..

[36]  Geoffrey E. Hinton.  Mapping Part-Whole Hierarchies into Connectionist Networks , 1990, Artif. Intell..

[37]  De Vries.  Book review: R.C. O'Reilly and Y. Munakata: Computational explorations in cognitive neuroscience: understanding the mind by stimulating the brain. Cambridge, Mass: The MIT Press. , 2002 .

[38]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain , 2000 .

[39]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing , 2000 .

[40]  João Luís Garcia Rosa.  A Biologically Motivated and Computationally Efficient Natural Language Processor , 2004, MICAI.

[41]  João Luís Garcia Rosa,et al.  A Connectionist Thematic Grid Predictor for Pre-parsed Natural Language Sentences , 2007, ISNN.

[42]  James H. Martin,et al.  Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , 2000 .

[43]  Peter Ford Dominey,et al.  Neural network processing of natural language: II. Towards a unified model of corticostriatal function in learning sentence comprehension and non-linguistic sequencing , 2009, Brain and Language.

[44]  Francis Crick,et al.  The recent excitement about neural networks , 1989, Nature.

[45]  Sandiway Fong,et al.  Natural Language Grammatical Inference with Recurrent Neural Networks , 2000, IEEE Trans. Knowl. Data Eng..

[46]  Liliane Haegeman,et al.  Introduction to Government and Binding Theory , 1991 .

[47]  J.L.G. Rosa,et al.  Thematic role assignment through a biologically plausible symbolic-connectionist hybrid system , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[48]  George A. Miller,et al.  Nouns in WordNet: A Lexical Inheritance System , 1990 .

[49]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[50]  Jude W. Shavlik,et al.  Extracting refined rules from knowledge-based neural networks , 2004, Machine Learning.

[51]  D. Rumelhart.  Parallel Distributed Processing Volume 1: Foundations , 1987 .

[52]  Ajay N. Jain,et al.  Parsing Complex Sentences with Structured Connectionist Networks , 1991, Neural Computation.

[53]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[54]  Christiane Fellbaum,et al.  English Verbs as a Semantic Net , 1990 .

[55]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[56]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[57]  Joydeep Ghosh,et al.  Symbolic Interpretation of Artificial Neural Networks , 1999, IEEE Trans. Knowl. Data Eng..

[58]  J. Elman.  An alternative view of the mental lexicon , 2004, Trends in Cognitive Sciences.

[59]  David R. Dowty.  On the Semantic Content of the Notion of ‘Thematic Role’ , 1989 .

[60]  John F. Sowa,et al.  Knowledge representation: logical, philosophical, and computational foundations , 2000 .

[61]  J. Elman.  Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[62]  Christopher R. Johnson,et al.  Background to Framenet , 2003 .

[63]  Daniel Jurafsky,et al.  Automatic Labeling of Semantic Roles , 2002, CL.

[64]  Neville Ryant,et al.  A Large-scale Classication of English Verbs , 2006 .

[65]  Geoffrey E. Hinton,et al.  Proceedings of the 1988 Connectionist Models Summer School , 1989 .