Rule learning with monotonicity constraints

In classification with monotonicity constraints, it is assumed that the class label should increase with increasing values on the attributes. In this paper we aim at formalizing the approach to learning with monotonicity constraints from statistical point of view. Motivated by the statistical analysis, we present an algorithm for learning rule ensembles. The algorithm first "monotonizes" the data using a nonparametric classification procedure and then generates a rule ensemble consistent with the training set. The procedure is justified by a theoretical analysis and verified in a computational experiment.

[1]  P. Anglin,et al.  SEMIPARAMETRIC ESTIMATION OF A HEDONIC PRICE FUNCTION , 1996 .

[2]  Wojciech Kotlowski,et al.  Stochastic dominance-based rough set model for ordinal classification , 2008, Inf. Sci..

[3]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[4]  Wojciech Kotlowski,et al.  Ensemble of Decision Rules for Ordinal Classification with Monotonicity Constraints , 2008, RSKT.

[5]  Tim Robertson,et al.  Nonparametric, isotonic discriminant procedures , 1999 .

[6]  Ling Li,et al.  Ordinal Regression by Extended Binary Classification , 2006, NIPS.

[7]  Leon Sterling,et al.  Learning and classification of monotonic ordinal concepts , 1989, Comput. Intell..

[8]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[9]  Salvatore Greco,et al.  Rough sets theory for multicriteria decision analysis , 2001, Eur. J. Oper. Res..

[10]  Ian Witten,et al.  Data Mining , 2000 .

[11]  A. J. Feelders,et al.  Classification trees for problems with monotonicity constraints , 2002, SKDD.

[12]  Yoshua Bengio,et al.  Inference for the Generalization Error , 1999, Machine Learning.

[13]  Joseph Sill,et al.  Monotonic Networks , 1997, NIPS.

[14]  V. Koltchinskii,et al.  Empirical margin distributions and bounding the generalization error of combined classifiers , 2002, math/0405343.

[15]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[16]  David Gamarnik.  Efficient learning of monotone concepts via quadratic optimization , 1998, COLT' 98.

[17]  A. Ben-David.  Monotonicity Maintenance in Information-Theoretic Machine Learning Algorithms , 1995, Machine Learning.

[18]  Varghese S. Jacob,et al.  Isotonic Separation , 2005, INFORMS J. Comput..

[19]  H. Daniels,et al.  Application of MLP Networks to Bond Rating and House Pricing , 1999, Neural Computing & Applications.

[20]  Wojciech Kotowski,et al.  Statistical Approach to Ordinal Classification with Monotonicity Constraints , 2008 .

[21]  A. J. Feelders,et al.  Nonparametric Monotone Classification with MOCA , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[22]  Ayhan Demiriz,et al.  Linear Programming Boosting via Column Generation , 2002, Machine Learning.

[23]  Salvatore Greco,et al.  Customer satisfaction analysis based on rough set approach , 2007 .

[24]  Klaus Obermayer,et al.  Regression Models for Ordinal Data: A Machine Learning Approach , 1999 .

[25]  Eibe Frank,et al.  A Simple Approach to Ordinal Classification , 2001, ECML.