Applying category theory to improve the performance of a neural architecture

A recently developed mathematical semantic theory explains the relationship between knowledge and its representation in connectionist systems. The semantic theory is based upon category theory, the mathematical theory of structure. A product of its explanatory capability is a set of principles to guide the design of future neural architectures and enhancements to existing designs. We claim that this mathematical semantic approach to network design is an effective basis for advancing the state of the art. We offer two experiments to support this claim. One of these involves multispectral imaging using data from a satellite camera.

[1]  Keith E. Williamson,et al.  Industrial Applications of Software Synthesis via Category Theory—Case Studies Using Specware , 2004, Automated Software Engineering.

[2]  Kai-Uwe Kühnberger,et al.  Learning Symbolic Inferences with Neural Networks , 2005 .

[3]  Thomas P. Caudell,et al.  Neural Networks, Knowledge and Cognition: A Mathematical Semantic Model Based upon Category Theory , 2004 .

[4]  Viggo Stoltenberg-Hansen,et al.  Mathematical theory of domains , 1994, Cambridge tracts in theoretical computer science.

[5]  D. Corfield.  Conceptual mathematics: a first introduction to categories , 2002 .

[6]  Roy L. Crole,et al.  Categories for Types , 1994, Cambridge mathematical textbooks.

[7]  Thomas P. Caudell,et al.  A categorical semantic analysis of ART architectures , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[8]  Antony Browne,et al.  Connectionist inference models , 2001, Neural Networks.

[9]  Thomas P. Caudell,et al.  eLoom and Flatland: specification, simulation and visualization engines for the study of arbitrary hierarchical neural architectures , 2003, Neural Networks.

[10]  José Meseguer,et al.  General Logics , 2006 .

[11]  Michael Georgiopoulos,et al.  Properties of learning related to pattern diversity in ART1 , 1991, Neural Networks.

[12]  Richard Jüllig,et al.  Diagrams for software synthesis , 1993, Proceedings of 8th Knowledge-Based Software Engineering Conference.

[13]  S. Vickers.  Topology via Logic , 1989 .

[14]  Michael J. Healy.  A Topological Semantics for Rule Extraction with Neural Networks , 1999, Connect. Sci..

[15]  Thomas P. Caudell,et al.  Ontologies and Worlds in Category Theory: Implications for Neural Systems , 2006 .

[16]  Nikola IL Kasabov.  Adaptable neuro production systems , 1996, Neurocomputing.

[17]  Jude W. Shavlik,et al.  Learning Symbolic Rules Using Artificial Neural Networks , 1993, ICML.

[18]  Antony Browne,et al.  Preprint of Paper ‘ Connectionist Inference Models ’ to appear in Neural Networks , 2001 .

[19]  J HealyMichael,et al.  Ontologies and worlds in category theory , 2006 .

[20]  G. Bartfai,et al.  Hierarchical clustering with ART neural networks , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[21]  Joseph A. Goguen,et al.  Institutions: abstract model theory for specification and programming , 1992, JACM.

[22]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[23]  S. Lane.  Categories for the Working Mathematician , 1971 .

[24]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[25]  Thomas P. Caudell,et al.  Acquiring rule sets as a product of learning in a logical neural architecture , 1997, IEEE Trans. Neural Networks.

[26]  M. J. Healy,et al.  Modification of the ART-1 architecture based on category theoretic design principles , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[27]  Benjamin C. Peirce,et al.  Basic Category Theory for Computer Scientists , 1991 .

[28]  Robert Rosen,et al.  A relational theory of biological systems II , 1958 .

[29]  A C Ehresmann,et al.  Information processing and symmetry-breaking in memory evolutive systems. , 1997, Bio Systems.