Context-Sensitive Attribute Estimation in Regression

One of key issues in both discrete and continuous class prediction and in machine learning in general seems to be the problem of estimating the quality of attributes. Heuris-tic measures mostly assume independence of attributes so their use is non-optimal in domains with strong dependencies between attributes. For the same reason they are also mostly unable to recognize context dependent features. Relief and its extension Re-liefF are statistical methods capable of correctly estimating the quality of attributes in classiication problems with strong dependencies between attributes. By exploiting local information provided by diierent contexts they provide a global view and recognize contextual attributes. After the analysis of ReliefF we have extended it to continuous class problems. Regressional ReliefF (RReliefF) and ReliefF provide a uniied view on estimating attribute quality. The experiments show that RReliefF correctly estimates the quality of attributes, recognizes the con-textual attributes and can be used for non-myopic learning of the regression trees.

[1]  Peter D. Turney.  Exploiting Context When Learning to Classify , 1993, ECML.

[2]  J. R. Quinlan.  Learning With Continuous Classes , 1992 .

[3]  Carla E. Brodley,et al.  Automatic Selection of Split Criterion during Tree Growing Based on Node Location , 1995, ICML.

[4]  Aram Karalic,et al.  Employing Linear Regression in Regression Tree Leaves , 1992, ECAI.

[5]  Francesco Ricci,et al.  Learning a Local Similarity Metric for Case-Based Reasoning , 1995, ICCBR.

[6]  Larry A. Rendell,et al.  The Feature Selection Problem: Traditional Methods and a New Algorithm , 1992, AAAI.

[7]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[8]  Gerhard Widmer,et al.  Effective Learning in Dynamic Environments by Explicit Context Tracking , 1993, ECML.

[9]  Philip J. Stone,et al.  Experiments in induction , 1966 .

[10]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[11]  Tapio Elomaa,et al.  A Geometric Approach to Feature Selection , 1994, ECML.

[12]  Se June Hong,et al.  Use of Contextaul Information for Feature Ranking and Discretization , 1997, IEEE Trans. Knowl. Data Eng..

[13]  Miroslav Kubat.  Conceptual Inductive Learning: The Case of Unreliable Teachers , 1992, Artif. Intell..

[14]  Gerhard Widmer,et al.  Recognition and Exploitation of Contextual CLues via Incremental Meta-Learning , 1996, ICML.

[15]  David J. Lubinsky.  Increasing the Performance and Consistency of Classification Trees by Using the Accuracy Criterion at the Leaves , 1995, ICML.