Hierarchical Multi-Task Learning: a Cascade Approach Based on the Notion of Task Relatedness

Multi-task learning can be shown to improve the generalization performance of single tasks under certain conditions. Typically, the algorithmic and theoretical analysis of multi-task learning deals with a two-level structure, including a group of tasks and a single task. In many situations, however, it is benecial to consider varying degrees of relatedness among tasks, assuming that some tasks are closely related and should contribute more to the learning process, while other tasks are less related but can still contribute some information to the learning process. The extension of current approaches to the multi-level setting may not be trivial. We propose a general framework for a full hierarchical multi-task setting. We dene an explicit notion of hierarchical tasks relatedness, where at each level we assume that some aspects of the learning problem are shared. We suggest a cascade approach, where at each level of the hierarchy a learner learns jointly the uniquely shared aspects of the tasks by nding a single shared hypothesis. This shared hypothesis is used to bootstrap the preceding level in the hierarchy, forming a hypothesis search space. We analyze sufcient conditions for our approach to reach optimality, and provide generalization guarantees in an empirical risk minimization setting.