Nonmonotonic Inferences and Neural Networks

There is a gap between two different modes of computation: the symbolic mode and the subsymbolic (neuron-like) mode. The aim of this paper is to overcome this gap by viewing symbolism as a high-level description of the properties of (a class of) neural networks. Combining methods of algebraic semantics and nonmonotonic logic, the possibility of integrating both modes of viewing cognition is demonstrated. The main results are (a) that certain activities of connectionist networks can be interpreted as non-monotonic inferences, and (b) that there is a strict correspondence between the coding of knowledge in Hopfield networks and the knowledge representation in weight-annotated Poole systems. These results show the usefulness of non-monotonic logic as a descriptive and analytic tool for analyzing emerging properties of connectionist networks. Assuming an exponential development of the weight function, the present account relates to optimality theory — a general framework that aims to integrate insights from symbolism and connectionism. The paper concludes with some speculations about extending the present ideas.

[1]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[2]  Barbara H. Partee,et al.  Montague Grammar , 1997, Handbook of Logic and Language.

[3]  Mary-Louise Kean,et al.  The theory of markedness in generative grammar , 1980 .

[4]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[5]  Géraldine Legendre,et al.  Principles for an Integrated Connectionist/Symbolic Theory of Higher Cognition ; CU-CS-600-92 , 1992 .

[6]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[7]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[8]  James A. Hendler,et al.  Developing Hybrid Symbolic/Connectionist Models , 1991 .

[9]  Krysia Broda,et al.  Symbolic knowledge extraction from trained neural networks: A sound approach , 2001, Artif. Intell..

[11]  Christian Balkenius,et al.  Nonmonotonic Inferences in Neural Networks , 1991, KR.

[12]  Michael N. Vrahatis,et al.  Artificial nonmonotonic neural networks , 2001, Artif. Intell..

[13]  Michael C. Mozer,et al.  Mathematical Perspectives on Neural Networks , 1996 .

[14]  D. Dennett.  Darwin's Dangerous Idea , 1995 .

[15]  Geoffrey E. Hinton,et al.  Schemata and Sequential Thought Processes in PDP Models , 1986 .

[16]  Lokendra Shastri,et al.  Seeking coherent explanations -- a fusion of structured connectionism, temporal synchrony, and evident reasoning - eScholarship , 2000 .

[17]  P. Smolensky.  On the proper treatment of connectionism , 1988, Behavioral and Brain Sciences.

[18]  Mark Derthick,et al.  Mundane Reasoning by Settling on a Plausible Model , 1990, Artif. Intell..

[19]  David Poole,et al.  A Logical Framework for Default Reasoning , 1988, Artif. Intell..

[20]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[21]  Paul Smolensky,et al.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1990, Artif. Intell..

[22]  Johan van Benthem,et al.  Handbook of Logic and Language , 1996 .

[23]  Paul Smolensky.  Grammar-based connectionist approaches to language , 1999 .

[24]  S. Pinker,et al.  On language and connectionism: Analysis of a parallel distributed processing model of language acquisition , 1988, Cognition.

[25]  R. Parasuraman.  The attentive brain , 1998 .

[26]  Noam Chomsky,et al.  The Sound Pattern of English , 1968 .

[27]  James A. Hendler,et al.  Editorial: On The Need for Hybrid Systems , 1989 .

[28]  W. Freeman.  Second Commentary: On the proper treatment of connectionism by Paul Smolensky (1988) - Neuromachismo Rekindled , 1989 .

[29]  Boicho Kokinov,et al.  Micro-Level Hybridization in the Cognitive Architecture DUAL , 1999 .

[30]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[31]  Dov M. Gabbay,et al.  Theoretical Foundations for Non-Monotonic Reasoning in Expert Systems , 1989, Logics and Models of Concurrent Systems.

[32]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[33]  Lauri Karttunen,et al.  The Proper Treatment of Optimality in Computational Phonology , 1998, ArXiv.

[34]  Giorgio Satta,et al.  Optimality Theory and the Generative Complexity of Constraint Violability , 1998, CL.

[35]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[36]  Gadi Pinkas,et al.  Reasoning, Nonmonotonicity and Learning in Connectionist Networks that Capture Propositional Knowledge , 1995, Artif. Intell..

[37]  P. Boersma,et al.  Empirical Tests of the Gradual Learning Algorithm , 2001, Linguistic Inquiry.

[38]  Sarit Kraus,et al.  Nonmonotonic Reasoning, Preferential Models and Cumulative Logics , 1990, Artif. Intell..

[39]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[40]  W. Bechtel,et al.  Connectionism and the Mind , 1991 .

[41]  P. Smolensky,et al.  Optimality Theory: Constraint Interaction in Generative Grammar , 2004 .