Ontology-Based Meta-Mining of Knowledge Discovery Workflows

This chapter describes a principled approach to meta-learning that has three distinctive features. First, whereas most previous work on meta-learning focused exclusively on the learning task, our approach applies meta-learning to the full knowledge discovery process and is thus more aptly referred to as meta-mining. Second, traditional meta-learning regards learning algorithms as black boxes and essentially correlates properties of their input (data) with the performance of their output (learned model). We propose to tear open the black box and analyse algorithms in terms of their core components, their underlying assumptions, the cost functions and optimization strategies they use, and the models and decision boundaries they generate. Third, to ground meta-mining on a declarative representation of the data mining (dm) process and its components, we built a DM ontology and knowledge base using the Web Ontology Language (owl).

[1]  Peter A. Flach,et al.  Decision Tree-Based Data Characterization for Meta-Learning , 2002 .

[2]  Katharina Morik,et al.  The MiningMart Approach to Knowledge Discovery in Databases , 2004 .

[3]  Vladimir Cherkassky,et al.  Model complexity control and statistical learning theory , 2002, Natural Computing.

[4]  Norbert Jankowski,et al.  Versatile and Efficient Meta-Learning Architecture: Knowledge Representation and Management in Computational Intelligence , 2007, 2007 IEEE Symposium on Computational Intelligence and Data Mining.

[5]  Ricardo Vilalta,et al.  Introduction to the Special Issue on Meta-Learning , 2004, Machine Learning.

[6]  Takahira Yamaguchi Specifying and Learning Inductive Learning Systems Using Ontologies , 1998 .

[7]  Peter L. Bartlett,et al.  For Valid Generalization the Size of the Weights is More Important than the Size of the Network , 1996, NIPS.

[8]  Ricardo Vilalta,et al.  Metalearning - Applications to Data Mining , 2008, Cognitive Technologies.

[9]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[10]  Tom M. Mitchell,et al.  The Need for Biases in Learning Generalizations , 2007 .

[11]  Pavel Brazdil,et al.  Predicting relative performance of classifiers from samples , 2005, ICML '05.

[12]  Mohammed J. Zaki Efficiently mining frequent trees in a forest: algorithms and applications , 2005, IEEE Transactions on Knowledge and Data Engineering.

[13]  Ning Zhong,et al.  Intelligent Technologies for Information Analysis , 2004, Springer Berlin Heidelberg.

[14]  Hilan Bensusan,et al.  Discovering Task Neighbourhoods Through Landmark Learning Performances , 2000, PKDD.

[15]  Joost N. Kok,et al.  Knowledge Discovery in Databases: PKDD 2007, 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, Warsaw, Poland, September 17-21, 2007, Proceedings , 2007, PKDD.

[16]  Abraham Bernstein,et al.  Data mining workflow templates for intelligent discovery assistance and auto-experimentation , 2010 .

[17]  Kate Smith-Miles,et al.  Cross-disciplinary perspectives on meta-learning for algorithm selection , 2009, CSUR.

[18]  Cullen Schaffer,et al.  A Conservation Law for Generalization Performance , 1994, ICML.

[19]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[20]  Ivan Janciak,et al.  Ontology-Based Construction of Grid Data Mining Workflows , 2008 .

[21]  Wlodzislaw Duch,et al.  Meta-learning via Search Combined with Parameter Optimization , 2002, Intelligent Information Systems.

[22]  Francesco Bonchi,et al.  Knowledge Discovery in Inductive Databases, 4th International Workshop, KDID 2005, Porto, Portugal, October 3, 2005, Revised Selected and Invited Papers , 2006, KDID.

[23]  Alexandros Kalousis,et al.  Algorithm selection via meta-learning , 2002 .

[24]  Steven Skiena,et al.  Implementing discrete mathematics - combinatorics and graph theory with Mathematica , 1990 .

[25]  Rodica Potolea,et al.  Evolutional meta-learning framework for automatic classifier selection , 2009, 2009 IEEE 5th International Conference on Intelligent Computer Communication and Processing.

[26]  João Gama,et al.  On Data and Algorithms: Understanding Inductive Performance , 2004, Machine Learning.

[27]  Hendrik Blockeel,et al.  Experiment Databases , 2007, Inductive Databases and Constraint-Based Data Mining.

[28]  David Tcheng,et al.  MORE ROBUST CONCEPT LEARNING USING DYNAMICALLY – VARIABLE BIAS , 1987 .

[29]  Peter A. Flach,et al.  Improved Dataset Characterisation for Meta-learning , 2002, Discovery Science.

[30]  Hilan Bensusan,et al.  A Higher-order Approach to Meta-learning , 2000, ILP Work-in-progress reports.

[31]  Melanie Hilario,et al.  Representational Issues in Meta-Learning , 2003, ICML.

[32]  Shusaku Tsumoto,et al.  Foundations of Intelligent Systems, 15th International Symposium, ISMIS 2005, Saratoga Springs, NY, USA, May 25-28, 2005, Proceedings , 2005, ISMIS.

[33]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[34]  Ricardo Vilalta,et al.  Using Meta-Learning to Support Data Mining , 2004, Int. J. Comput. Sci. Appl..

[35]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery: An Overview , 1996, Advances in Knowledge Discovery and Data Mining.

[36]  João Gama,et al.  Characterizing the Applicability of Classification Algorithms Using Meta-Level Learning , 1994, ECML.

[37]  Mario Cannataro,et al.  A Data Mining Ontology for Grid Programming , 2003 .

[38]  Sara Reese Hedberg The Twenty-Second AAAI Conference: Continuing the Content-Rich Tradition in Beautiful Vancouver, British Columbia , 2007, AI Mag..

[39]  Gregory Piatetsky-Shapiro Data Mining and Knowledge Discovery: The Third Generation (Extended Abstract) , 1997, ISMIS.

[40]  Rajkumar Roy,et al.  Advances in Soft Computing , 2018, Lecture Notes in Computer Science.

[41]  Kate Smith-Miles,et al.  A meta-learning approach to automatic kernel selection for support vector machines , 2006, Neurocomputing.

[42]  Melanie Hilario,et al.  Fusion of Meta-knowledge and Meta-data for Case-Based Model Selection , 2001, PKDD.

[43]  David W. Aha,et al.  Special Issue on Lazy Learning , 1997 .

[44]  Alexey Tsymbal,et al.  Arbiter Meta-Learning with Dynamic Selection of Classifiers and Its Experimental Investigation , 1999, ADBIS.

[45]  J. Ross Quinlan,et al.  Improved Use of Continuous Attributes in C4.5 , 1996, J. Artif. Intell. Res..

[46]  M. Hilario,et al.  A Data Mining Ontology for Algorithm Selection and Meta-Mining , 2009 .

[47]  Alexander Schliep,et al.  Ranking and selecting clustering algorithms using a meta-learning approach , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[48]  Saso Dzeroski,et al.  OntoDM: An Ontology of Data Mining , 2008, 2008 IEEE International Conference on Data Mining Workshops.

[49]  Saso Dzeroski,et al.  Towards a General Framework for Data Mining , 2006, KDID.

[50]  Luc De Raedt,et al.  Machine Learning: ECML-94 , 1994, Lecture Notes in Computer Science.

[51]  T. Ho,et al.  Data Complexity in Pattern Recognition , 2006 .

[52]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[53]  Xindong Wu,et al.  10 Challenging Problems in Data Mining Research , 2006, Int. J. Inf. Technol. Decis. Mak..

[54]  Larry A. Rendell,et al.  The Feature Selection Problem: Traditional Methods and a New Algorithm , 1992, AAAI.

[55]  Charles C. Taylor,et al.  Meta-Analysis: From Data Characterisation for Meta-Learning to Meta-Regression , 2000 .

[56]  Carlos Soares,et al.  Zoomed Ranking: Selection of Classification Algorithms Based on Relevant Performance Information , 2000, PKDD.

[57]  Daren Ler Utilising Regression-based Landmarkers within a Meta-learning Framework for Algorithm Selection , 2005 .

[58]  Steffen Staab,et al.  SEAL-II - The Soft Spot between Richly Structured and Unstructured Knowledge , 2001, J. Univers. Comput. Sci..

[59]  Judea Pearl,et al.  Heuristics : intelligent search strategies for computer problem solving , 1984 .

[60]  Johannes Fürnkranz,et al.  An Evaluation of Landmarking Variants , 2001 .

[61]  Ramakrishnan Srikant,et al.  Mining Sequential Patterns: Generalizations and Performance Improvements , 1996, EDBT.

[62]  Nada Lavrac,et al.  Automating Knowledge Discovery Workflow Composition Through Ontology-Based Planning , 2011, IEEE Transactions on Automation Science and Engineering.

[63]  P. Ut Goff,et al.  Machine learning of inductive bias , 1986 .

[64]  Pedro M. Domingos A Unified Bias-Variance Decomposition for Zero-One and Squared Loss , 2000, AAAI/IAAI.

[65]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[66]  Ljup Co Todorovski,et al.  Experiments in Meta-level Learning with Ilp , 1999 .

[67]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[68]  Tin Kam Ho,et al.  Measures of Geometrical Complexity in Classification Problems , 2006 .

[69]  Björn Bringmann,et al.  Matching in frequent tree discovery , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[70]  Hilan Bensusan,et al.  Meta-Learning by Landmarking Various Learning Algorithms , 2000, ICML.

[71]  Marie desJardins,et al.  Evaluation and selection of biases in machine learning , 1995, Machine Learning.

[72]  Jan Komorowski,et al.  Principles of Data Mining and Knowledge Discovery , 2001, Lecture Notes in Computer Science.

[73]  Tim Oates,et al.  A Review of Recent Research in Metareasoning and Metalearning , 2007, AI Mag..

[74]  Alan L. Rector,et al.  Modularisation of domain ontologies implemented in description logics and related formalisms including OWL , 2003, K-CAP '03.

[75]  John R. Rice,et al.  The Algorithm Selection Problem , 1976, Adv. Comput..

[76]  Norbert Jankowski,et al.  Building meta-learning algorithms basing on search controlled by machine complexity , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[77]  San Cristóbal Mateo,et al.  The Lack of A Priori Distinctions Between Learning Algorithms , 1996 .

[78]  Irena Koprinska,et al.  A Proposed Meta-learning Framework for Algorithm Selection Utilising Regression-based Landmarkers , 2006 .

[79]  Larry A. Rendell,et al.  Layered Concept-Learning and Dynamically Variable Bias Management , 1987, IJCAI.

[80]  Carlos Soares,et al.  A Meta-Learning Method to Select the Kernel Width in Support Vector Regression , 2004, Machine Learning.

[81]  Abraham Bernstein,et al.  Toward intelligent assistance for a data mining process: an ontology-based approach for cost-sensitive classification , 2005, IEEE Transactions on Knowledge and Data Engineering.

[82]  Ricardo Vilalta,et al.  A Perspective View and Survey of Meta-Learning , 2002, Artificial Intelligence Review.

[83]  Joaquin Vanschoren,et al.  Exposé: An ontology for data mining experiments , 2010 .

[84]  Hiroki Arimura,et al.  Efficient Algorithms for Mining Frequent and Closed Patterns from Semi-structured Data , 2008, PAKDD.

[85]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[86]  Huan Liu,et al.  A Probabilistic Approach to Feature Selection - A Filter Solution , 1996, ICML.