Image Transformation: Inductive Transfer between Multiple Tasks Having Multiple Outputs

Previous research has investigated inductive transfer for single output modeling problems such as classification or prediction of a scalar. Little research has been done in the area of inductive transfer applied to tasks with multiple outputs. We report the results of using Multiple Task Learning (MTL) neural networks and Context-sensitive Multiple Task Learning (csMTL) on a domain of image transformation tasks. Models are developed to transform synthetic images of neutral (passport) faces to that of corresponding images of angry, happy and sad faces. The results are inconclusive for MTL, however they demonstrate that inductive transfer with csMTL is beneficial. When the secondary tasks have sufficient numbers of training examples from which to provide transfer, csMTL models are able to transform images more accurately than standard single task learning models.

[1]  Abhijit S. Pandya,et al.  Neural networks for face recognition , 1999 .

[2]  Daniel L. Silver,et al.  Sequential Consolidation of Learned Task Knowledge , 2004, Canadian Conference on AI.

[3]  Tony Jebara,et al.  Multi-task feature and kernel selection for SVMs , 2004, ICML.

[4]  Tom Heskes,et al.  Empirical Bayes for Learning to Learn , 2000, ICML.

[5]  Robert E. Mercer,et al.  The Task Rehearsal Method of Life-Long Learning: Overcoming Impoverished Data , 2002, Canadian Conference on AI.

[6]  Jonathan Baxter,et al.  Learning Model Bias , 1995, NIPS.

[7]  Robert E. Mercer,et al.  The Parallel Transfer of Task Knowledge Using Dynamic Learning Rates Based on a Measure of Relatedness , 1998, Learning to Learn.

[8]  Daniel L. Silver,et al.  Context-Sensitive MTL Networks for Machine Lifelong Learning , 2007, FLAIRS Conference.

[9]  Paul E. Utgoff,et al.  Machine Learning of Inductive Bias , 1986 .

[10]  Xinghuo Yu,et al.  AI 2004: Advances in Artificial Intelligence, 17th Australian Joint Conference on Artificial Intelligence, Cairns, Australia, December 4-6, 2004, Proceedings , 2004, Australian Conference on Artificial Intelligence.

[11]  Sebastian Thrun,et al.  Lifelong Learning Algorithms , 1998, Learning to Learn.

[12]  Sebastian Thrun,et al.  Learning to Learn , 1998, Springer US.

[13]  Rich Caruana,et al.  Multitask Learning: A Knowledge-Based Source of Inductive Bias , 1993, ICML.

[14]  Lorien Y. Pratt,et al.  Discriminability-Based Transfer between Neural Networks , 1992, NIPS.

[15]  Charles A. Micchelli,et al.  Kernels for Multi--task Learning , 2004, NIPS.

[16]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.