Relatedness measures to aid the transfer of building blocks among multiple tasks

Multitask Learning is a learning paradigm that deals with multiple different tasks in parallel and transfers knowledge among them. XOF, a Learning Classifier System using tree-based programs to encode building blocks (meta-features), constructs and collects features with rich discriminative information for classification tasks in an Observed List. This paper seeks to facilitate the automation of feature transferring in between tasks by utilising the Observed List. We hypothesise that the best discriminative features of a classification task carry its characteristics. Therefore, the relatedness between any two tasks can be estimated by comparing their most appropriate patterns. We propose a multiple-XOF system, called mXOF, that can dynamically adapt feature transfer among XOFs. This system utilises the Observed List to estimate the task relatedness. This method enables the automation of transferring features. In terms of knowledge discovery, the resemblance estimation provides insightful relations among multiple data. We experimented mXOF on various scenarios, e.g. representative Hierarchical Boolean problems, classification of distinct classes in the UCI Zoo dataset, and unrelated tasks, to validate its abilities of automatic knowledgetransfer and estimating task relatedness. Results show that mXOF can estimate the relatedness reasonably between multiple tasks to aid the learning performance with the dynamic feature transferring.

[1]  Mengjie Zhang,et al.  Human-inspired Scaling in Learning Classifier Systems: Case Study on the n-bit Multiplexer Problem Set , 2016, GECCO.

[2]  H. Crichton-Miller Adaptation , 1926 .

[3]  Will N. Browne,et al.  Constructing Complexity-efficient Features in XCS with Tree-based Rule Conditions , 2020, 2021 IEEE Congress on Evolutionary Computation (CEC).

[4]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[5]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[6]  Martin V. Butz,et al.  An Algorithmic Description of XCS , 2000, IWLCS.

[7]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[8]  Larry Bull,et al.  A Genetic Programming-based Classifier System , 1999, GECCO.

[9]  Yew-Soon Ong,et al.  Multifactorial Evolution: Toward Evolutionary Multitasking , 2016, IEEE Transactions on Evolutionary Computation.

[10]  Manuela M. Veloso,et al.  Layered Learning , 2000, ECML.

[11]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[12]  Will N. Browne,et al.  Investigating Scaling of an Abstracted LCS Utilising Ternary and S-Expression Alphabets , 2007, IWLCS.

[13]  Mengjie Zhang,et al.  Reusing learned functionality in XCS: code fragments with constructed functionality and constructed features , 2014, GECCO.

[14]  Martin V. Butz,et al.  An algorithmic description of XCS , 2000, Soft Comput..

[15]  Mengjie Zhang,et al.  Improvement of code fragment fitness to guide feature construction in XCS , 2019, GECCO.

[16]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[17]  Will N. Browne,et al.  Introduction to Learning Classifier Systems , 2017, SpringerBriefs in Intelligent Systems.

[18]  D. Hassabis,et al.  Neuroscience-Inspired Artificial Intelligence , 2017, Neuron.

[19]  Luca Lanzi Pier,et al.  Extending the Representation of Classifier Conditions Part II: From Messy Coding to S-Expressions , 1999 .

[20]  Sebastian Thrun,et al.  Lifelong robot learning , 1993, Robotics Auton. Syst..

[21]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[22]  Martin V. Butz,et al.  Rule-Based Evolutionary Online Learning Systems - A Principled Approach to LCS Analysis and Design , 2006, Studies in Fuzziness and Soft Computing.

[23]  Mengjie Zhang,et al.  Online Feature-Generation of Code Fragments for XCS to Guide Feature Construction , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[24]  Mengjie Zhang,et al.  Reusing Building Blocks of Extracted Knowledge to Solve Complex, Large-Scale Boolean Problems , 2014, IEEE Transactions on Evolutionary Computation.

[25]  Abhishek Gupta,et al.  Multifactorial Evolutionary Algorithm With Online Transfer Parameter Estimation: MFEA-II , 2020, IEEE Transactions on Evolutionary Computation.

[26]  Stewart W. Wilson Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.