Predicting Future Decision Trees from Evolving Data

Recognizing and analyzing change is an important human virtue because it enables us to anticipate future scenarios and thus allows us to act pro-actively. One approach to understand change within a domain is to analyze how models and patterns evolve. Knowing how a model changes over time is suggesting to ask: Can we use this knowledge to learn a model in anticipation, such that it better reflects the near-future characteristics of an evolving domain? In this paper we provide an answer to this question by presenting an algorithm which predicts future decision trees based on a model of change. In particular, this algorithm encompasses a novel approach to change mining which is based on analyzing the changes of the decisions made during model learning. The proposed approach can also be applied to other types of classifiers and thus provides a basis for future research. We present our first experimental results which show that anticipated decision trees have the potential to outperform trees learned on the most recent data.

[1]  KlinkenbergRalf.  Learning drifting concepts: Example selection vs. example weighting , 2004 .

[2]  Gerhard Widmer,et al.  Learning in the presence of concept drift and hidden contexts , 2004, Machine Learning.

[3]  Geoff Hulten,et al.  Mining time-changing data streams , 2001, KDD '01.

[4]  Myra Spiliopoulou,et al.  MONIC: modeling and monitoring cluster transitions , 2006, KDD '06.

[5]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[6]  Xindong Wu,et al.  Combining proactive and reactive predictions for data streams , 2005, KDD '05.

[7]  L. Lovász.  Matching Theory (North-Holland mathematics studies) , 1986 .

[8]  Clifford M. Hurvich,et al.  Regression and time series model selection in small samples , 1989 .

[9]  Ralph Kimball,et al.  The Data Warehouse Toolkit: Practical Techniques for Building Dimensional Data Warehouses , 1996 .

[10]  Giuseppe Psaila,et al.  Active Data Mining , 1995, Encyclopedia of GIS.

[11]  David R. Anderson,et al.  Multimodel Inference , 2004 .

[12]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[13]  Philip Bille,et al.  A survey on tree edit distance and related problems , 2005, Theor. Comput. Sci..

[14]  Yiming Ma,et al.  Analyzing the interestingness of association rules from the temporal dimension , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[15]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[16]  L. Breiman.  Heuristics of instability and stabilization in model selection , 1996 .

[17]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[18]  H. Akaike.  A new look at the statistical model identification , 1974 .

[19]  Detlef D. Nauck,et al.  Towards a Framework for Change Detection in Data Sets , 2006, SGAI Conf..

[20]  Philip E. Gill,et al.  Practical optimization , 1981 .

[21]  Frank Höppner,et al.  Matching Partitions over Time to Reliably Capture Local Clusters in Noisy Domains , 2007, PKDD.

[22]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[23]  Ralf Klinkenberg,et al.  Learning drifting concepts: Example selection vs. example weighting , 2004, Intell. Data Anal..

[24]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.