$A^2T$: Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources

The ability to transfer knowledge from source tasks to a new target task can be very useful in speeding up a Reinforcement Learning agent. Such transfer has been receiving a lot of attention lately, yet the application of transfer poses two serious challenges which have not been adequately addressed. First, the agent should be able to avoid negative transfer, which happens when the transfer hampers or slows down the learning instead of helping it. Second, the agent should be able to do selective transfer, which is the ability to select and transfer from different and multiple source tasks for different parts of the state space of the target task. We propose AT (Attend, Adapt and Transfer), an attentive deep architecture for adaptive transfer, which addresses these challenges. AT is generic enough to effect transfer of either policies or value functions. Empirical evaluations on different learning algorithms show that AT is an effective architecture for transfer learning by being able to avoid negative transfer while transferring selectively from multiple sources.