DRILL- Deep Reinforcement Learning for Refinement Operators in ALC

Approaches based on refinement operators have been successfully applied to class expression learning on RDF knowledge graphs. These approaches often need to explore a large number of concepts to find adequate hypotheses. This need arguably stems from current approaches relying on myopic heuristic functions to guide their search through an infinite concept space. In turn, deep reinforcement learning provides effective means to address myopia by estimating how much discounted cumulated future reward states promise. In this work, we leverage deep reinforcement learning to accelerate the learning of concepts in ALC by proposing Drill—a novel class expression learning approach that uses a convolutional deep Q-learning model to steer its search. By virtue of its architecture, Drill is able to compute the expected discounted cumulated future reward of more than 10 class expressions in a second on standard hardware. We evaluate Drill on four benchmark datasets against state-of-the-art approaches. Our results suggest that Drill converges to goal states at least 2.7× faster than state-of-the-art models on all benchmark datasets. We provide an open-source implementation of our approach, including training and evaluation scripts as well as pretrained models.

[1]  Markus Krötzsch,et al.  Getting the Most Out of Wikidata: Semantic Technology Usage in Wikipedia's Knowledge Graph , 2018, SEMWEB.

[2]  Axel-Cyrille Ngonga Ngomo,et al.  A Holistic Natural Language Generation Framework for the Semantic Web , 2019, RANLP.

[3]  Jason Yosinski,et al.  Estimating Q(s, s') with Deep Deterministic Dynamics Gradients , 2020, ICML.

[4]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[5]  M. Behlol,et al.  Concept of Learning , 2010 .

[6]  Georg Langs,et al.  Causability and explainability of artificial intelligence in medicine , 2019, WIREs Data Mining Knowl. Discov..

[7]  Luigi Iannone,et al.  An Algorithm Based on Counterfactuals for Concept Learning in the Semantic Web , 2005, IEA/AIE.

[8]  Klaus-Robert Müller,et al.  Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models , 2017, ArXiv.

[9]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[10]  Jason Weston,et al.  Question Answering with Subgraph Embeddings , 2014, EMNLP.

[11]  Nicola Fanizzi,et al.  Boosting DL Concept Learners , 2019, ESWC.

[12]  Nicola Fanizzi,et al.  DL-FOIL Concept Learning in Description Logics , 2008, ILP.

[13]  Wojciech Samek,et al.  Explainable AI: Interpreting, Explaining and Visualizing Deep Learning , 2019, Explainable AI.

[14]  Jens Lehmann,et al.  Learning OWL Class Expressions , 2010, Studies on the Semantic Web.

[15]  Jens Lehmann,et al.  Towards SPARQL-Based Induction for Large-Scale RDF Data Sets , 2016, ECAI.

[16]  Jens Lehmann,et al.  A Refinement Operator Based Learning Algorithm for the ALC Description Logic , 2007, ILP.

[17]  Wenhan Xiong,et al.  DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning , 2017, EMNLP.

[18]  Alexander J. Smola,et al.  Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning , 2017, ICLR.

[19]  Liviu Badea,et al.  A Refinement Operator for Description Logics , 2000, ILP.

[20]  Klaus-Robert Müller,et al.  Towards Explainable Artificial Intelligence , 2019, Explainable AI.

[21]  Yongfeng Zhang,et al.  Reinforcement Knowledge Graph Reasoning for Explainable Recommendation , 2019, SIGIR.

[22]  Paul Buitelaar,et al.  Utilizing Knowledge Graphs for Neural Machine Translation Augmentation , 2019, K-CAP.

[23]  Jens Lehmann,et al.  Class expression learning for ontology engineering , 2011, J. Web Semant..

[24]  Nicholas Jing Yuan,et al.  DRN: A Deep Reinforcement Learning Framework for News Recommendation , 2018, WWW.

[25]  Steffen Staab,et al.  Knowledge graphs , 2021, Commun. ACM.

[26]  Richard Socher,et al.  Multi-Hop Knowledge Graph Reasoning with Reward Shaping , 2018, EMNLP.

[27]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[28]  Jens Lehmann,et al.  Concept learning in description logics using refinement operators , 2009, Machine Learning.

[29]  Giuseppe Carenini,et al.  Proceedings of the 25th International Conference on Intelligent User Interfaces , 2015, IUI 2015.

[30]  Alexander Borgida,et al.  Computing Least Common Subsumers in Description Logics , 1992, AAAI.

[31]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[32]  Jens Lehmann,et al.  DL-Learner - A framework for inductive learning on the Semantic Web , 2016, J. Web Semant..

[33]  Axel-Cyrille Ngonga Ngomo,et al.  Convolutional Complex Knowledge Graph Embeddings , 2021, ESWC.

[34]  Muhammad Saleem,et al.  Big linked cancer data: Integrating linked TCGA and PubMed , 2014, J. Web Semant..

[35]  Carri W. Chan,et al.  Stochastic Depletion Problems: Effective Myopic Policies for a Class of Dynamic Optimization Problems , 2008, Math. Oper. Res..

[36]  Axel-Cyrille Ngonga Ngomo,et al.  A shallow neural model for relation prediction , 2021, 2021 IEEE 15th International Conference on Semantic Computing (ICSC).

[37]  Margaret Burnett Explaining AI: fairly? well? , 2019, IUI Workshops.

[38]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.