Towards integrated neural–symbolic systems for human-level AI: Two research programs helping to bridge the gaps

Abstract After a human-level AI-oriented overview of the status quo in neural–symbolic integration, two research programs aiming at overcoming long-standing challenges in the field are suggested to the community: The first program targets a better understanding of foundational differences and relationships on the level of computational complexity between symbolic and subsymbolic computation and representation, potentially providing explanations for the empirical differences between the paradigms in application scenarios and a foothold for subsequent attempts at overcoming these. The second program suggests a new approach and computational architecture for the cognitively-inspired anchoring of an agent’s learning, knowledge formation, and higher reasoning abilities in real-world interactions through a closed neural–symbolic acting/sensing–processing–reasoning cycle, potentially providing new foundations for future agent architectures, multi-agent systems, robotics, and cognitive systems and facilitating a deeper understanding of the development and interaction in human-technological settings.

[1]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[2]  Ralph Adolphs,et al.  The unsolved problems of neuroscience , 2015, Trends in Cognitive Sciences.

[3]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[4]  Geoffrey E. Hinton A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[5]  Tarek R. Besold Computational Models of Analogy-Making An Overview Analysis of Computational Approaches to Analogical Reasoning , 2011 .

[6]  Alan Bundy,et al.  A Higher Order Approach to Ontology Evolution in Physics , 2013, Journal on Data Semantics.

[7]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[8]  Herbert Jaeger,et al.  Controlling Recurrent Neural Networks by Conceptors , 2014, ArXiv.

[9]  Krysia Broda,et al.  Symbolic knowledge extraction from trained neural networks: A sound approach , 2001, Artif. Intell..

[10]  Wolfgang Maass,et al.  On the Computational Power of Threshold Circuits with Sparse Activity , 2006, Neural Computation.

[11]  Hannes Leitgeb,et al.  Interpreted Dynamical Systems and Qualitative Laws: from Neural Networks to Evolutionary Systems , 2005, Synthese.

[12]  Heribert Vollmer,et al.  Complexity of non-monotonic logics , 2010, Bull. EATCS.

[13]  Dov M. Gabbay,et al.  Neural-Symbolic Cognitive Reasoning , 2008, Cognitive Technologies.

[14]  Luc De Raedt,et al.  Neural-Symbolic Learning and Reasoning: Contributions and Challenges , 2015, AAAI Spring Symposia.

[15]  Artur S. d'Avila Garcez,et al.  A Neural-Symbolic Cognitive Agent for Online Learning and Reasoning , 2011, IJCAI.

[16]  Alessandro Saffiotti,et al.  Cooperative anchoring in heterogeneous multi-robot systems , 2008, 2008 IEEE International Conference on Robotics and Automation.

[17]  L. Valiant Probably Approximately Correct: Nature's Algorithms for Learning and Prospering in a Complex World , 2013 .

[18]  G. Fauconnier,et al.  The Way We Think: Conceptual Blending and the Mind''s Hidden Complexities. Basic Books , 2002 .

[19]  Kai-Uwe Kühnberger,et al.  Anchoring Knowledge in Interaction: Towards a Harmonic Subsymbolic/Symbolic Framework and Architecture of Computational Cognition , 2015, AGI.

[20]  Thomas G. Dietterich Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[21]  Thomas L. Griffiths,et al.  One and Done? Optimal Decisions From Very Few Samples , 2014, Cogn. Sci..

[22]  R. Diaconescu Institution-independent model theory , 2008 .

[23]  Alan Bundy,et al.  The interaction of representation and reasoning , 2013, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[24]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Stefan Poslad,et al.  Ubiquitous Computing: Smart Devices, Environments and Interactions , 2009 .

[26]  Zhi-Hua Zhou,et al.  Extracting symbolic rules from trained neural network ensembles , 2003, AI Commun..

[27]  D. Whitteridge,et al.  Learning and Relearning , 1959, Science's STKE.

[28]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[29]  Julian D. Olden,et al.  Illuminating the “black box”: a randomization approach for understanding variable contributions in artificial neural networks , 2002 .

[30]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[31]  Peter van Emde Boas,et al.  Machine Models and Simulation , 1990, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[32]  Hava T. Siegelmann,et al.  Neural networks and analog computation - beyond the Turing limit , 1999, Progress in theoretical computer science.

[33]  Stefan Habenschuss,et al.  Stochastic Computations in Cortical Microcircuit Models , 2013, PLoS Comput. Biol..

[34]  Martin H. Fischer,et al.  A hierarchical view of grounded, embodied, and situated numerical cognition , 2012, Cognitive Processing.

[35]  Christian Lebiere,et al.  THE B-I-C-A OF BIOLOGICALLY INSPIRED COGNITIVE ARCHITECTURES , 2010 .

[36]  Whitney Tabor,et al.  A dynamical systems perspective on the relationship between symbolic and non-symbolic computation , 2009, Cognitive Neurodynamics.

[37]  James A. Reggia,et al.  A generalized LSTM-like training algorithm for second-order recurrent neural networks , 2012, Neural Networks.

[38]  A. Triller,et al.  The Dynamic Synapse , 2013, Neuron.

[39]  Dirk Kraft,et al.  A Survey of the Ontogeny of Tool Use: From Sensorimotor Experience to Planning , 2013, IEEE Transactions on Autonomous Mental Development.

[40]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[41]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[42]  Diane J. Cook,et al.  Author's Personal Copy Pervasive and Mobile Computing Ambient Intelligence: Technologies, Applications, and Opportunities , 2022 .

[43]  Rineke Verbrugge,et al.  Logic and Complexity in Cognitive Science , 2014, Johan van Benthem on Logic and Information Dynamics.

[44]  Iris van Rooij,et al.  The Tractable Cognition Thesis , 2008, Cogn. Sci..

[45]  John Dinsmore,et al.  The symbolic and connectionist paradigms : closing the gap , 1992 .

[46]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[47]  Jakub Szymanik,et al.  Exploring the tractability border in epistemic tasks , 2012, Synthese.

[48]  Alessandro Saffiotti,et al.  Anchoring Symbols to Sensor Data: Preliminary Report , 2000, AAAI/IAAI.

[49]  Nina Gierasimczuk,et al.  A note on a generalization of the Muddy Children puzzle , 2011, TARK XIII.

[50]  Michèle Sebag,et al.  Unsupervised Layer-Wise Model Selection in Deep Neural Networks , 2010, ECAI.

[51]  Pekka Orponen,et al.  General-Purpose Computation with Neural Networks: A Survey of Complexity Theoretic Results , 2003, Neural Computation.

[52]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  O. Sporns,et al.  Towards the virtual brain: network modeling of the intact and the damaged brain. , 2010, Archives italiennes de biologie.

[54]  Alexei V. Samsonovich,et al.  On a roadmap for the BICA Challenge , 2012, BICA 2012.

[55]  Jonathan Evans In two minds: dual-process accounts of reasoning , 2003, Trends in Cognitive Sciences.

[56]  P. Boas Machine models and simulations , 1991 .

[57]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[58]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[59]  Fei-Fei Li,et al.  Deep visual-semantic alignments for generating image descriptions , 2015, CVPR.

[60]  Peter Gärdenfors,et al.  Conceptual spaces - the geometry of thought , 2000 .

[61]  Salvatore Gaglio,et al.  Anchoring symbols to conceptual spaces: the case of dynamic scenarios , 2003, Robotics Auton. Syst..