Learning with Configurable Operators and RL-Based Heuristics

In this paper, we push forward the idea of machine learning systems for which the operators can be modified and finetuned for each problem. This allows us to propose a learning paradigm where users can write (or adapt) their operators, according to the problem, data representation and the way the information should be navigated. To achieve this goal, data instances, background knowledge, rules, programs and operators are all written in the same functional language, Erlang. Since changing operators affect how the search space needs to be explored, heuristics are learnt as a result of a decision process based on reinforcement learning where each action is defined as a choice of operator and rule. As a result, the architecture can be seen as a …system for writing machine learning systems' or to explore new operators.

[1]  Jiuyong Li.  AI 2010: Advances in Artificial Intelligence - 23rd Australasian Joint Conference, Adelaide, Australia, December 7-10, 2010. Proceedings , 2011, Australasian Conference on Artificial Intelligence.

[2]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[3]  Thomas Gärtner,et al.  Kernels for structured data , 2008, Series in Machine Perception and Artificial Intelligence.

[4]  Saso Dzeroski,et al.  Towards a General Framework for Data Mining , 2006, KDID.

[5]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.

[6]  José Hernández-Orallo,et al.  Incremental Learning of Functional Logic Programs , 2001, FLOPS.

[7]  Marco Colombetti,et al.  What Is a Learning Classifier System? , 1999, Learning Classifier Systems.

[8]  John Langford,et al.  Search-based structured prediction , 2009, Machine Learning.

[9]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[10]  Gordon Plotkin,et al.  A Note on Inductive Generalization , 2008 .

[11]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[12]  Francesco Bonchi,et al.  Knowledge Discovery in Inductive Databases, 4th International Workshop, KDID 2005, Porto, Portugal, October 3, 2005, Revised Selected and Invited Papers , 2006, KDID.

[13]  Thomas Hofmann,et al.  Support vector machine learning for interdependent and structured output spaces , 2004, ICML.

[14]  José Hernández-Orallo,et al.  Web Categorisation Using Distance-Based Decision Trees , 2005, Electron. Notes Theor. Comput. Sci..

[15]  David L. Dowe,et al.  Refinements of MDL and MML Coding , 1999, Comput. J..

[16]  Emanuel Kitzelmann,et al.  Inductive Programming: A Survey of Program Synthesis Techniques , 2009, AAIP.

[17]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[18]  Daphne Koller,et al.  Hierarchically Classifying Documents Using Very Few Words , 1997, ICML.

[19]  Thomas G. Dietterich,et al.  Structured machine learning: the next ten years , 2008, Machine Learning.

[20]  Hal Daumé,et al.  Unsupervised search-based structured prediction , 2009, ICML '09.

[21]  José Hernández-Orallo,et al.  Approaches and Applications of Inductive Programming, Third International Workshop, AAIP 2009, Edinburgh, UK, September 4, 2009. Revised Papers , 2010, AAIP.

[22]  Stephen Muggleton,et al.  Inductive Logic Programming: Issues, Results and the Challenge of Learning Language in Logic , 1999, Artif. Intell..

[23]  José Hernández-Orallo,et al.  Similarity Functions for Structured Data. An Application to Decision Trees , 2006, Inteligencia Artif..

[24]  W. Marsden.  I and J , 2012 .

[25]  C. Ferri,et al.  Bridging the Gap between Distance and Generalisation , 2008 .

[26]  Stephen Muggleton,et al.  A Genetic Algorithms Approach to ILP , 2002, ILP.

[27]  Stewart W. Wilson,et al.  Learning classifier systems: New models, successful applications , 2002, Inf. Process. Lett..

[28]  José Hernández-Orallo,et al.  Distance-Based Generalisation Operators for Graphs , 2006 .

[29]  John W. Lloyd,et al.  Knowledge Representation, Computation, and Learning in Higher-order Logic , 2002 .

[30]  Joe Armstrong,et al.  A history of Erlang , 2007, HOPL.

[31]  Tom Schrijvers,et al.  Functional and Logic Programming , 2012, Lecture Notes in Computer Science.

[32]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[33]  Larry Bull,et al.  Learning Classifier Systems , 2002, Annual Conference on Genetic and Evolutionary Computation.

[34]  José Hernández-Orallo,et al.  Newton Trees , 2010, Australasian Conference on Artificial Intelligence.

[35]  S. Džeroski,et al.  Relational Data Mining , 2001, Springer Berlin Heidelberg.

[36]  F. R. A. Hopgood,et al.  Machine Intelligence 5 , 1971, The Mathematical Gazette.

[37]  Robert Givan,et al.  Relational Reinforcement Learning: An Overview , 2004, ICML 2004.

[38]  José Hernández-Orallo,et al.  BRIDGING THE GAP BETWEEN DISTANCE AND GENERALIZATION , 2014, Comput. Intell..

[39]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[40]  Bernhard Schölkopf,et al.  Some kernels for structured data , 2001 .

[41]  Jürgen Schmidhuber,et al.  Optimal Ordered Problem Solver , 2002, Machine Learning.

[42]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[43]  Ludovic Denoyer,et al.  Structured prediction with reinforcement learning , 2009, Machine Learning.

[44]  Kurt Driessens,et al.  Relational Reinforcement Learning , 1998, Machine-mediated learning.