Bias learning, knowledge sharing

Biasing the hypothesis space of a learner has been shown to improve generalisation performances. Methods for achieving this goal are proposed, that range from deriving and introducing a bias into a learner to automatically learning the bias. In the latter case, most methods learn the bias by simultaneously training several related tasks derived from the same domain and imposing constraints on their parameters. We extend some of the ideas presented in this field and describe a new model that parametrizes the parameters of each task as a function of an affine manifold defined in parameter space and a point lying on the manifold. An analysis of variance on a class of learning tasks is performed that shows some significantly improved performances when using the model.

[1]  Geoffrey E. Hinton,et al.  A time-delay neural network architecture for isolated word recognition , 1990, Neural Networks.

[2]  Tom Heskes,et al.  Solving a Huge Number of Similar Tasks: A Combination of Multi-Task Learning and a Hierarchical Bayesian Approach , 1998, ICML.

[3]  Hilan Bensusan.  Odd bites into bananas don''t make you blind: learning about simplicity and attribute addition , 1998 .

[4]  Yoshua Bengio,et al.  Multi-Task Learning for Stock Selection , 1996, NIPS.

[5]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[6]  Sebastian Thrun,et al.  Learning to Learn , 1998, Springer US.

[7]  Robert E. Mercer,et al.  The Parallel Transfer of Task Knowledge Using Dynamic Learning Rates Based on a Measure of Relatedness , 1998, Learning to Learn.

[8]  Joseph Sill,et al.  Monotonicity Hints , 1996, NIPS.

[9]  P. McCullagh,et al.  Generalized Linear Models , 1972, Predictive Analytics.

[10]  Raymond J. Mooney,et al.  Improving Shared Rules in Multiple Category Domain Theories , 1991, ML.

[11]  William W. Cohen.  Compiling prior knowledge into an explicit basis , 1992, ICML 1992.

[12]  Tom M. Mitchell,et al.  Experience with a learning personal assistant , 1994, CACM.

[13]  Rich Caruana,et al.  Algorithms and Applications for Multitask Learning , 1996, ICML.

[14]  K Y Liang,et al.  Longitudinal data analysis for discrete and continuous outcomes. , 1986, Biometrics.

[15]  Jonathan Baxter,et al.  Learning Model Bias , 1995, NIPS.

[16]  Nathan Intrator,et al.  How to Make a Low-Dimensional Representation Suitable for Diverse Tasks , 1996 .

[17]  Stephen M. Omohundro.  Family Discovery , 1995, NIPS.

[18]  Sebastian Thrun,et al.  Discovering Structure in Multiple Learning Tasks: The TC Algorithm , 1996, ICML.

[19]  Geoffrey E. Hinton,et al.  Simplifying Neural Networks by Soft Weight-Sharing , 1992, Neural Computation.

[20]  Yaser S. Abu-Mostafa,et al.  Hints and the VC Dimension , 1993, Neural Computation.