Towards Unsupervised Knowledge Extraction

Integration of symbolic and sub-symbolic approaches is rapidly emerging as an Artificial Intelligence (AI) paradigm. This paper presents a proof-of-concept approach towards an unsupervised learning method, based on Restricted Boltzmann Machines (RBMs), for extracting semantic associations among prominent entities within data. Validation of the approach is performed in two datasets that connect language and vision, namely Visual Genome and GQA. A methodology to formally structure the extracted knowledge for subsequent use through reasoning engines is also offered.

[1]  David E. Irwin,et al.  Finding a "Kneedle" in a Haystack: Detecting Knee Points in System Behavior , 2011, 2011 31st International Conference on Distributed Computing Systems Workshops.

[2]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[3]  Gorjan Alagic,et al.  #p , 2019, Quantum information & computation.

[4]  Ole Winther,et al.  Recurrent Relational Networks , 2017, NeurIPS.

[5]  Timothy A. Miller,et al.  Learning Patient Representations from Text , 2018, *SEM@NAACL-HLT.

[6]  Michael S. Bernstein,et al.  Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations , 2016, International Journal of Computer Vision.

[7]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[8]  Chuang Gan,et al.  The Neuro-Symbolic Concept Learner: Interpreting Scenes Words and Sentences from Natural Supervision , 2019, ICLR.

[9]  Anik De Ribaupierre,et al.  Piaget's Theory of Cognitive Development , 2015 .

[10]  Tijmen Tieleman,et al.  Training restricted Boltzmann machines using approximations to the likelihood gradient , 2008, ICML '08.

[11]  Wen Yu,et al.  Data-Driven Fuzzy Modeling Using Restricted Boltzmann Machines and Probability Theory , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[12]  이화영.  X , 1960, Chinese Plants Names Index 2000-2009.

[13]  Honglak Lee,et al.  Learning hierarchical representations for face verification with convolutional deep belief networks , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Trevor Darrell,et al.  Modeling Relationships in Referential Expressions with Compositional Modular Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  S. Tran,et al.  Knowledge Extraction from Deep Belief Networks for Images , 2013 .

[16]  Geoffrey E. Hinton,et al.  The Recurrent Temporal Restricted Boltzmann Machine , 2008, NIPS.

[17]  Artur S. d'Avila Garcez,et al.  A Neural-Symbolic Cognitive Agent for Online Learning and Reasoning , 2011, IJCAI.

[18]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[19]  Pascal Hitzler,et al.  On the Capabilities of Logic Tensor Networks for Deductive Reasoning , 2019, AAAI Spring Symposium Combining Machine Learning with Knowledge Engineering.

[20]  Christopher D. Manning,et al.  GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Fan Chung Graham,et al.  A Combinatorial Laplacian with Vertex Weights , 1996, J. Comb. Theory, Ser. A.

[22]  Richard Evans,et al.  Learning Explanatory Rules from Noisy Data , 2017, J. Artif. Intell. Res..

[23]  Paolo Favaro,et al.  Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles , 2016, ECCV.

[24]  Stefano Faralli,et al.  Large-scale taxonomy induction using entity and word embeddings , 2017, WI.

[25]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[26]  Eric P. Xing,et al.  Harnessing Deep Neural Networks with Logic Rules , 2016, ACL.

[27]  Yang Yu,et al.  Tunneling Neural Perception and Logic Reasoning through Abductive Learning , 2018, ArXiv.

[28]  Marvin Minsky,et al.  Logical Versus Analogical or Symbolic Versus Connectionist or Neat Versus Scruffy , 1991, AI Mag..

[29]  Michael Philippsen,et al.  Automatic Clustering of Code Changes , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).