Ensemble of Decision Rules for Ordinal Classification with Monotonicity Constraints

Ordinal classification problemswithmonotonicity constraints (also referred to as multicriteria classification problems) often appear in real-life applications, however they are considered relatively less frequently in theoretical studies than regular classification problems. We introduce a rule induction algorithm based on forward stagewise additive modeling that is tailored for this type of problems. The algorithm monotonizes the dataset (excludes highly inconsistent objects) using Dominance-based Rough Set Approach and generates monotone rules. Experimental results indicate that taking into account the knowledge about order and monotonicity constraints in the classifier can improve the prediction accuracy.

[1]  C. Zopounidis.  Operational tools in the management of financial risks , 1997 .

[2]  F. T. Wright,et al.  Order restricted statistical inference , 1988 .

[3]  Ian Witten,et al.  Data Mining , 2000 .

[4]  Salvatore Greco,et al.  Rough sets theory for multicriteria decision analysis , 2001, Eur. J. Oper. Res..

[5]  Wojciech Kotlowski,et al.  Statistical Model for Rough Set Approach to Multicriteria Classification , 2007, PKDD.

[6]  Michael Doumpos,et al.  Developing and Testing Models for Replicating Credit Ratings: A Multicriteria Approach , 2005 .

[7]  Yoram Singer,et al.  A simple, fast, and effective rule learner , 1999, AAAI 1999.

[8]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[9]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[10]  Roman Słowiński,et al.  Ensembles of decision rules , 2006 .

[11]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[12]  Roman Słowiński,et al.  A New Rough Set Approach to Evaluation of Bankruptcy Risk , 1998 .

[13]  Sholom M. Weiss,et al.  Lightweight Rule Induction , 2000, ICML.

[14]  Joost N. Kok,et al.  Knowledge Discovery in Databases: PKDD 2007, 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, Warsaw, Poland, September 17-21, 2007, Proceedings , 2007, PKDD.

[15]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[16]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[17]  Young U. Ryu,et al.  DATA CLASSIFICATION USING THE ISOTONIC SEPARATION TECHNIQUE : APPLICATION TO BREAST CANCER PREDICTION , 2004 .

[18]  Sadaaki Miyamoto,et al.  Rough Sets and Current Trends in Computing , 2012, Lecture Notes in Computer Science.

[19]  A. J. Feelders,et al.  Classification trees for problems with monotonicity constraints , 2002, SKDD.

[20]  Robert Susmaga,et al.  Generation of Exhaustive Set of Rules within Dominance-based Rough Set Approach , 2003, RSKD.

[21]  Varghese S. Jacob,et al.  Computing, Artificial Intelligence and Information Management Breast cancer prediction using the isotonic separation technique , 2007 .

[22]  Bogdan E. Popescu,et al.  PREDICTIVE LEARNING VIA RULE ENSEMBLES , 2008, 0811.1679.

[23]  H. Daniels,et al.  Application of MLP Networks to Bond Rating and House Pricing , 1999, Neural Computing & Applications.

[24]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[25]  Leon G. Higley,et al.  Forensic Entomology: An Introduction , 2009 .

[26]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[27]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[28]  Z. Pawlak.  Rough Sets: Theoretical Aspects of Reasoning about Data , 1991 .

[29]  Salvatore Greco,et al.  An Algorithm for Induction of Decision Rules Consistent with the Dominance Principle , 2000, Rough Sets and Current Trends in Computing.

[30]  Gary Koop,et al.  Analysis of Economic Data , 2000 .