Mix&Match - Agent Curricula for Reinforcement Learning

We introduce MixM using our method to progress through an action-space curriculum we achieve both faster training and better final performance than one obtains using traditional methods. (2) We further show that M&M can be used successfully to progress through a curriculum of architectural variants defining an agents internal state. (3) Finally, we illustrate how a variant of our method can be used to improve agent performance in a multitask setting.

[1]  Shane Legg,et al.  IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures , 2018, ICML.

[2]  Wojciech Zaremba,et al.  Learning to Execute , 2014, ArXiv.

[3]  Yuanzhi Li,et al.  Convergence Analysis of Two-layer Neural Networks with ReLU Activation , 2017, NIPS.

[4]  Alex Graves,et al.  Automated Curriculum Learning for Neural Networks , 2017, ICML.

[5]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[6]  J. Elman.  Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[7]  Yuval Tassa,et al.  Emergence of Locomotion Behaviours in Rich Environments , 2017, ArXiv.

[8]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[9]  Rich Caruana,et al.  Model compression , 2006, KDD '06.

[10]  Max Jaderberg,et al.  Population Based Training of Neural Networks , 2017, ArXiv.

[11]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[12]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[13]  Rich Caruana,et al.  Do Deep Nets Really Need to be Deep? , 2013, NIPS.

[14]  Razvan Pascanu,et al.  Policy Distillation , 2015, ICLR.

[15]  Ruslan Salakhutdinov,et al.  Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning , 2015, ICLR.

[16]  Razvan Pascanu,et al.  Sobolev Training for Neural Networks , 2017, NIPS.

[17]  Huchuan Lu,et al.  Deep Mutual Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Yee Whye Teh,et al.  Distral: Robust multitask reinforcement learning , 2017, NIPS.

[19]  Changhu Wang,et al.  Network Morphism , 2016, ICML.

[20]  Tom Schaul,et al.  Reinforcement Learning with Unsupervised Auxiliary Tasks , 2016, ICLR.

[21]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[22]  Tianqi Chen,et al.  Net2Net: Accelerating Learning via Knowledge Transfer , 2015, ICLR.

[23]  Wojciech Jaskowski,et al.  ViZDoom: A Doom-based AI research platform for visual reinforcement learning , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[24]  Razvan Pascanu,et al.  Learning to Navigate in Complex Environments , 2016, ICLR.

[25]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.