Multi-task Learning by Pareto Optimality

Deep Neural Networks (DNNs) are often criticized because they lack the ability to learn more than one task at a time: Multitask Learning is an emerging research area whose aim is to overcome this issue. In this work, we introduce the Pareto Multitask Learning framework as a tool that can show how effectively a DNN is learning a shared representation common to a set of tasks. We also experimentally show that it is possible to extend the optimization process so that a single DNN simultaneously learns how to master two or more Atari games: using a single weight parameter vector, our network is able to obtain sub-optimal results for up to four games.

[1]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[2]  Risto Miikkulainen,et al.  Designing neural networks through neuroevolution , 2019, Nat. Mach. Intell..

[3]  Massimiliano Pontil,et al.  Sparse coding for multitask and transfer learning , 2012, ICML.

[4]  Giuseppe Nicosia,et al.  Computational energy-based redesign of robust proteins , 2011, Comput. Chem. Eng..

[5]  Sebastian Thrun,et al.  Learning to Learn , 1998, Springer US.

[6]  Daphne Koller,et al.  Self-Paced Learning for Latent Variable Models , 2010, NIPS.

[7]  Jürgen Schmidhuber,et al.  Ultimate Cognition à la Gödel , 2009, Cognitive Computation.

[8]  Jaime G. Carbonell,et al.  Self-Paced Multitask Learning with Shared Knowledge , 2017, IJCAI.

[9]  Kenneth O. Stanley,et al.  Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents , 2017, NeurIPS.

[10]  Massimiliano Pontil,et al.  Multilinear Multitask Learning , 2013, ICML.

[11]  Qiang Yang,et al.  An Overview of Multi-task Learning , 2018 .

[12]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[13]  Anne Auger,et al.  Hypervolume-based multiobjective optimization: Theoretical foundations and practical implications , 2012, Theor. Comput. Sci..

[14]  Sebastian Ruder,et al.  An Overview of Multi-Task Learning in Deep Neural Networks , 2017, ArXiv.

[15]  Carlos M. Fonseca,et al.  An Improved Dimension-Sweep Algorithm for the Hypervolume Indicator , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[16]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[17]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents , 2012, J. Artif. Intell. Res..

[18]  Demis Hassabis,et al.  A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play , 2018, Science.