Ontology-based n-ball Concept Embeddings Informing Few-shot Image Classification

We propose a novel framework named ViOCE that integrates ontology-based background knowledge in the form of n-ball concept embeddings into a neural network based vision architecture. The approach consists of two main components converting symbolic knowledge of an ontology into continuous space by learning n-ball embeddings that capture properties of subsumption and disjointness, and guiding the training and inference of a vision model using the learnt embeddings. We evaluate ViOCE using the task of few-shot image classification, where it demonstrates superior performance on two standard benchmarks.

[1]  P. Alam ‘A’ , 2021, Composites Engineering: An A–Z Guide.

[2]  Michael J. Watts,et al.  IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS Publication Information , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Artur S. d'Avila Garcez,et al.  Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge , 2016, NeSy@HLAI.

[4]  Maxat Kulmanov,et al.  EL Embeddings: Geometric construction of models for the Description Logic EL ++ , 2019, IJCAI.

[5]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[6]  Vincent Gripon,et al.  Leveraging the Feature Distribution in Transfer-based Few-Shot Learning , 2020, ICANN.

[7]  Yong Dou,et al.  Spatial Attention Network for Few-Shot Learning , 2019, ICANN.

[8]  Bijan Parsia,et al.  Measuring similarity in ontologies: a new family of measures , 2014, SEMWEB.

[9]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Edward Szczerbicki,et al.  From Knowledge based Vision Systems to Cognitive Vision Systems: A Review , 2018, KES.

[12]  Uli Sattler,et al.  Visual-Semantic Embedding Model Informed by Structured Knowledge , 2020, STAIRS@ECAI.

[13]  Abhinav Gupta,et al.  The More You Know: Using Knowledge Graphs for Image Classification , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Boris Motik,et al.  OWL 2 Web Ontology Language: structural specification and functional-style syntax , 2008 .

[15]  Abhinav Gupta,et al.  Zero-Shot Recognition via Semantic Embeddings and Knowledge Graphs , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Wei Shen,et al.  Few-Shot Image Recognition by Predicting Parameters from Activations , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[18]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[19]  Marc'Aurelio Ranzato,et al.  DeViSE: A Deep Visual-Semantic Embedding Model , 2013, NIPS.

[20]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[21]  Boris Motik,et al.  HermiT: A Highly-Efficient OWL Reasoner , 2008, OWLED.

[22]  Tingting Mu,et al.  Adaptive Data Embedding Framework for Multiclass Classification , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Yue Wang,et al.  Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need? , 2020, ECCV.

[24]  Cory Henson,et al.  An Evaluation of Knowledge Graph Embeddings for Autonomous Driving Data: Experience and Practice , 2020, AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering.

[25]  P. Alam ‘T’ , 2021, Composites Engineering: An A–Z Guide.

[26]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[27]  Eric P. Xing,et al.  Harnessing Deep Neural Networks with Logic Rules , 2016, ACL.

[28]  Danna Zhou,et al.  d. , 1840, Microbial pathogenesis.

[29]  Qiang Ji Combining knowledge with data for efficient and generalizable visual learning , 2019, Pattern Recognit. Lett..

[30]  Yu-Chiang Frank Wang,et al.  A Closer Look at Few-shot Classification , 2019, ICLR.

[31]  Mirantha Jayathilaka Enhancing Generalization of First-Order Meta-Learning , 2019 .

[32]  Joshua B. Tenenbaum,et al.  Meta-Learning for Semi-Supervised Few-Shot Classification , 2018, ICLR.