Extracting Hidden Context

Concept drift due to hidden changes in context complicates learning in many domains including financial prediction, medical diagnosis, and communication network performance. Existing machine learning approaches to this problem use an incremental learning, on-line paradigm. Batch, off-line learners tend to be ineffective in domains with hidden changes in context as they assume that the training set is homogeneous. An off-line, meta-learning approach for the identification of hidden context is presented. The new approach uses an existing batch learner and the process of contextual clustering to identify stable hidden contexts and the associated context specific, locally stable concepts. The approach is broadly applicable to the extraction of context reflected in time and spatial attributes. Several algorithms for the approach are presented and evaluated. A successful application of the approach to a complex flight simulator control task is also presented.

[1]  J. Rissanen.  A UNIVERSAL PRIOR FOR INTEGERS AND ESTIMATION BY MINIMUM DESCRIPTION LENGTH , 1983 .

[2]  R. Michalski.  How to Learn Imprecise Concepts: A Method for Employing a Two-Tiered Knowledge Representation in Learning , 1987 .

[3]  Miroslav Kubat.  Floating approximation in time-varying knowledge bases , 1989, Pattern Recognit. Lett..

[4]  Haym Hirsh,et al.  Incremental Batch Learning , 1989, ML.

[5]  Edwin P. D. Pednault,et al.  Some Experiments in Applying Inductive Inference Principles to Surface Reconstruction , 1989, IJCAI.

[6]  Alan J. Katz,et al.  Robust Classifiers without Robust Features , 1990, Neural Computation.

[7]  Ryszard S. Michalski,et al.  Learning flexible concepts: fundamental ideas and a method based on two-tiered representation , 1990 .

[8]  Jack Mostow,et al.  Direct Transfer of Learned Information Among Neural Networks , 1991, AAAI.

[9]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[10]  Miroslav Kubat.  A machine learning-based approach to load balancing in computer networks , 1992 .

[11]  Lorien Y. Pratt,et al.  Discriminability-Based Transfer between Neural Networks , 1992, NIPS.

[12]  Marcos Salganicoff,et al.  Density-Adaptive Learning and Forgetting , 1993, ICML.

[13]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[14]  Peter D. Turney.  Exploiting Context When Learning to Classify , 1993, ECML.

[15]  Gerhard Widmer,et al.  Effective Learning in Dynamic Environments by Explicit Context Tracking , 1993, ECML.

[16]  Fredrik Kilander,et al.  COBBIT - A Control Procedure for COBWEB in the Presence of Concept Drift , 1993, ECML.

[17]  Larry A. Rendell,et al.  Improving the Design of Induction Methods by Analyzing Algorithm Functionality and Data-Based Concept Complexity , 1993, IJCAI.

[18]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[19]  Gerhard Widmer,et al.  Adapting to Drift in Continuous Domains (Extended Abstract) , 1995, ECML.

[20]  M. Harries.  Detecting Concept Drift in Financial Time Series Prediction using Symbolic Machine Learning , 1995 .

[21]  Stefanos Manganaris.  Classifying sensor data with Calchas , 1996 .

[22]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery: An Overview , 1996, Advances in Knowledge Discovery and Data Mining.

[23]  Michael Bonnell Harries,et al.  Learning stable concepts in a changing world , 1996, PRICAI Workshops.

[24]  Gerhard Widmer,et al.  Recognition and Exploitation of Contextual CLues via Incremental Meta-Learning , 1996, ICML.

[25]  Miroslav Kubat.  Second Tier for Decision Trees , 1996, ICML.

[26]  Michael Bonnell Harries.  Learning Stable Concepts in Domains With Hidden Changes in Context , 1996, ICML 1996.

[27]  S. Pattinson,et al.  Learning to fly. , 1998 .

[28]  JefI’rty C. Schlirrlrrer.  Beyond incremental processing : Tracking concept drift , 1999 .

[29]  Peter D. Turney.  Robust Classification with Context-Sensitive Features , 2002, ArXiv.

[30]  J. C. Schlimmer,et al.  Incremental learning from noisy data , 2004, Machine Learning.

[31]  Peter D. Turney,et al.  Contextual normalization applied to aircraft gas turbine engine diagnosis , 2004, Applied Intelligence.

[32]  Pedro M. Domingos.  Control-Sensitive Feature Selection for Lazy Learners , 1997, Artificial Intelligence Review.

[33]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[34]  D. Kibler,et al.  Instance-based learning algorithms , 2004, Machine Learning.

[35]  Gerhard Widmer,et al.  Tracking Context Changes through Meta-Learning , 1997, Machine Learning.

[36]  Gerhard Widmer,et al.  Learning in the Presence of Concept Drift and Hidden Contexts , 1996, Machine Learning.