TAdaNet: Task-Adaptive Network for Graph-Enriched Meta-Learning

Annotated data samples in real-world applications are often limited. Meta-learning, which utilizes prior knowledge learned from related tasks and generalizes to new tasks of limited supervised experience, is an effective approach for few-shot learning. However, standard meta-learning with globally shared knowledge cannot handle the task heterogeneity problem well, i.e., tasks lie in different distributions. Recent advances have explored several ways to trigger task-dependent initial parameters or metrics, in order to customize task-specific information. These approaches learn task contextual information from data, but ignore external domain knowledge that can help in the learning process. In this paper, we propose a task-adaptive network (TAdaNet) that makes use of a domain-knowledge graph to enrich data representations and provide task-specific customization. Specifically, we learn a task embedding that characterizes task relationships and tailors task-specific parameters, resulting in a task-adaptive metric space for classification. Experimental results on a few-shot image classification problem show the effectiveness of the proposed method. We also apply it on a real-world disease classification problem, and show promising results for clinical decision support.

[1]  Marcin Andrychowicz,et al.  Learning to learn by gradient descent by gradient descent , 2016, NIPS.

[2]  Tao Zhang,et al.  A Hybrid Approach with Optimization and Metric-based Meta-Learner for Few-Shot Learning , 2019, ArXiv.

[3]  Joseph J. Lim,et al.  Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation , 2019, NeurIPS.

[4]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[5]  Jiayu Zhou,et al.  MetaPred: Meta-Learning for Clinical Risk Prediction with Limited Patient Electronic Health Records , 2019, KDD.

[6]  Lu Wang,et al.  Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation , 2018, KDD.

[7]  Yu-Chiang Frank Wang,et al.  A Closer Look at Few-shot Classification , 2019, ICLR.

[8]  Le Song,et al.  GRAM: Graph-based Attention Model for Healthcare Representation Learning , 2016, KDD.

[9]  Razvan Pascanu,et al.  Meta-Learning with Latent Embedding Optimization , 2018, ICLR.

[10]  Yong Wang,et al.  Meta-Learning for Low-Resource Neural Machine Translation , 2018, EMNLP.

[11]  Thomas L. Griffiths,et al.  Recasting Gradient-Based Meta-Learning as Hierarchical Bayes , 2018, ICLR.

[12]  Lina Yao,et al.  Prototype Propagation Networks (PPN) for Weakly-supervised Few-shot Learning on Category Graph , 2019, IJCAI.

[13]  Fenglong Ma,et al.  KAME: Knowledge-based Attention Model for Diagnosis Prediction in Healthcare , 2018, CIKM.

[14]  Joshua B. Tenenbaum,et al.  Meta-Learning for Semi-Supervised Few-Shot Classification , 2018, ICLR.

[15]  Nitesh V. Chawla,et al.  Graph Few-shot Learning via Knowledge Transfer , 2020, AAAI.

[16]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[17]  Fenglong Ma,et al.  Risk Prediction on Electronic Health Records with Prior Medical Knowledge , 2018, KDD.

[18]  Tsendsuren Munkhdalai,et al.  Rapid Adaptation with Conditionally Shifted Neurons , 2017, ICML.

[19]  Hang Li,et al.  Meta-SGD: Learning to Learn Quickly for Few Shot Learning , 2017, ArXiv.

[20]  Alexandre Lacoste,et al.  TADAM: Task dependent adaptive metric for improved few-shot learning , 2018, NeurIPS.

[21]  Eunho Yang,et al.  Learning to Propagate Labels: Transductive Propagation Network for Few-Shot Learning , 2018, ICLR.

[22]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[23]  Ying Wei,et al.  Hierarchically Structured Meta-learning , 2019, ICML.

[24]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[25]  Feiyue Huang,et al.  LGM-Net: Learning to Generate Matching Networks for Few-Shot Learning , 2019, ICML.

[26]  Ping Zhang,et al.  Risk Prediction with Electronic Health Records: A Deep Learning Approach , 2016, SDM.

[27]  Hyunsouk Cho,et al.  MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation , 2019, KDD.

[28]  Xianfeng Tang,et al.  Learning from Multiple Cities: A Meta-Learning Approach for Spatial-Temporal Prediction , 2019, WWW.

[29]  Xian Wu,et al.  Automated Relational Meta-learning , 2020, ICLR.

[30]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[31]  Aram Galstyan,et al.  Multitask learning and benchmarking with clinical time series data , 2017, Scientific Data.

[32]  Sung Whan Yoon,et al.  TapNet: Neural Network Augmented with Task-Adaptive Projection for Few-Shot Learning , 2019, ICML.

[33]  Jing Jiang,et al.  Learning to Propagate for Graph Meta-Learning , 2019, NeurIPS.

[34]  Xiaowei Wang,et al.  Sequential Scenario-Specific Meta Learner for Online Recommendation , 2019, KDD.

[35]  Seungjin Choi,et al.  Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace , 2018, ICML.

[36]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[37]  Pieter Abbeel,et al.  A Simple Neural Attentive Meta-Learner , 2017, ICLR.

[38]  Tao Xiang,et al.  Learning to Compare: Relation Network for Few-Shot Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Xin Wang,et al.  Few-Shot Object Detection via Feature Reweighting , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[40]  Joan Bruna,et al.  Few-Shot Learning with Graph Neural Networks , 2017, ICLR.